Automatic performance test platform and test process thereof

Abstract

The invention relates to the technical field of automatic testing, and discloses an automatic performance testing platform and a testing process thereof.The testing platform comprises a task creating module, a plurality of task scheduling modules and a task testing module, the task creating module is arranged on a front-end server, and the task scheduling modules are arranged on the front-end server; each task scheduling module and each task testing module are respectively arranged on each slave server; the task creation module is used for creating a test task and configuring a test object, and the test object is a to-be-tested App on at least one mobile terminal; and the task scheduling module performs scheduling based on the test task and the test object of the task creation module, determines a target test task and sends the target test task to the corresponding task test module, so that the task test module performs an App performance test according to the target test task to obtain a corresponding test result. According to the test platform, the whole process of App performance test can be automatically executed, the comprehensiveness and effectiveness of performance test are improved, and the user experience is enhanced while the test quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and in particular to an automated performance testing platform and a testing process thereof. Background Art

[0002] After a mobile application is developed, relevant personnel need to conduct a lot of tests and make adjustments based on the test results before it can be put into commercial use. In the field of App testing, the testing of mobile apps is basically done manually, but manual testing is time-consuming and laborious, so automation tools related to corresponding testing have emerged.

[0003] Currently, although there are automation tools such as appium and Fastboot, there is a lack of in-depth research on App performance testing in such automation tools. In addition, testers often conduct evaluation tests on App performance, such as response time and memory usage, based on their own subjective experience. Due to the differences in subjective cognition among different testers, the test accuracy is poor, making it difficult to ensure the stability and quality of App performance testing, which in turn results in poor user experience and makes it difficult to achieve efficient automated performance testing of Apps. Summary of the invention

[0004] In view of this, the present invention provides an automated performance testing platform and its testing process to solve the problem that the prior art lacks a comprehensive, effective and high-quality automated testing solution for App performance, resulting in poor stability and efficiency of App performance testing, which seriously affects user experience.

[0005] In a first aspect, the present invention provides an automated performance testing platform, the testing platform comprising a front-end server and a back-end server, the front-end server is used to provide a human-computer interaction interface, the back-end server comprises a master server and multiple slave servers, wherein the master server is used to schedule each slave server, and each slave server is used to test corresponding Apps on multiple mobile terminals, and the testing platform also comprises:

[0006] A task creation module, multiple task scheduling modules and a task testing module, wherein the task creation module is arranged on the front-end server, and each task scheduling module and task testing module is arranged on each slave server respectively;

[0007] The task creation module is used to create a test task and configure a test object, where the test object is an App to be tested on at least one mobile terminal;

[0008] The task scheduling module schedules the test tasks and test objects based on the task creation module, determines the target test tasks and sends them to the corresponding task testing modules, so that the task testing modules perform App performance tests according to the target test tasks and obtain corresponding test results.

[0009] The automated performance testing platform of the present invention realizes the separation of the front-end server and the back-end server, and designs a master-multiple-slave server architecture on the back-end server, wherein each slave server is used to test the corresponding Apps on multiple mobile terminals; each slave server is provided with a task scheduling module and a task testing module, wherein the task scheduling module schedules the test tasks and configured test objects through the test creation module set on the front-end server, determines the target test tasks and sends them to the corresponding task testing module, so that the task testing module performs App performance testing according to the target test tasks and obtains corresponding test results, which not only realizes the fully automatic execution of App performance testing, but also ensures the stability and quality of App performance testing, comprehensively improves the testing efficiency and user experience, and meets the needs of App automated performance testing.

[0010] In an optional implementation, the task creation module includes: a first creation unit, a second creation unit, a first management unit, and a second management unit;

[0011] The first creation unit is used to create a random stress test task and configure the App to be tested on the corresponding mobile terminal;

[0012] The second creation unit is used to create a stress test task for a specified path and configure the App to be tested on the corresponding mobile terminal;

[0013] The first management unit is used to manage all servers;

[0014] The second management unit is used to manage all mobile terminals.

[0015] The present invention creates random stress testing tasks or specified path stress testing tasks and configures corresponding apps to be tested by designing a task creation module including a first creation unit, a second creation unit, a first management unit and a second management unit, and performs corresponding management on all servers and mobile terminals on the platform. This can improve the test platform's flexible control over test tasks and included devices, and further enhance the performance stability testing experience of the APP.

[0016] In an optional implementation, the task scheduling module includes: a state monitoring unit, a state recording unit, a timing configuration unit, a first association unit, and a second association unit;

[0017] Among them, the status monitoring unit is used to monitor the current test status of all mobile terminals and obtain status monitoring results;

[0018] The status recording unit is used to record the script running log;

[0019] The timing configuration unit is used to configure the execution time of the test task;

[0020] The first association unit is used to associate the test object of the task creation module with the corresponding mobile terminal according to the status monitoring result of the status monitoring unit;

[0021] The second association unit is used to associate the slave server corresponding to the mobile terminal associated with the real machine with the front-end server.

[0022] The present invention designs a task scheduling module including a status monitoring unit, a status recording unit, a timing configuration unit, a first association unit and a second association unit. The module realizes effective scheduling of all mobile terminals on the test platform in combination with the created test tasks, and generates corresponding reasonable target test tasks for subsequent APP performance testing. The task scheduling module provides a comprehensive, effective and high-quality automated testing management and control solution for App performance, which greatly ensures the stability of App performance testing and further enhances user experience.

[0023] In an optional implementation, the task test module includes: an installation unit and a test unit;

[0024] The installation unit is used to provide a test package corresponding to the target test task, and install the test package to the corresponding mobile terminal to obtain the target mobile terminal;

[0025] The testing unit performs App performance testing on the App to be tested according to the test package of the target mobile terminal and obtains corresponding test results.

[0026] The present invention realizes automatic and effective App performance testing of the App to be tested by designing a task testing module including an installation unit and a testing unit, which can comprehensively improve the quality and efficiency of App automated performance testing.

[0027] In an optional implementation, a preset performance component is built into the target mobile terminal, which is used to collect and analyze performance data during the App performance testing process, and generate corresponding test results based on the analysis results, wherein the preset performance component includes at least a stability component, a performance inspection component, a code coverage component and a data analysis component. The stability component is used to monitor App crashes and obtain corresponding crash data; the performance inspection component is used to collect preset performance indicators and obtain corresponding inspection data; and the code coverage component is used to collect code coverage.

[0028] The present invention collects and analyzes the performance data of the App performance testing process through multiple performance components built into the target mobile terminal, and generates corresponding test results based on the analysis results, which not only helps to ensure the comprehensiveness of the test results, but also improves the accuracy and quality of the performance test.

[0029] In a second aspect, the present invention provides a test process of an automated performance testing platform, which is applied to the automated performance testing platform in the first aspect or any corresponding embodiment thereof, and the test process includes:

[0030] Create test tasks and configure test objects;

[0031] Schedule based on test tasks and test objects to determine target test tasks;

[0032] Perform App performance testing according to the target test tasks and obtain corresponding test results.

[0033] The testing process of the automated performance testing platform of the present invention automatically schedules the created test tasks and test objects to obtain corresponding target test tasks, and performs App performance testing according to the target test tasks to obtain corresponding test results. This can realize the fully automatic execution of the App performance testing process, help to ensure the stability and quality of App performance testing, thereby comprehensively improving testing efficiency and user experience, while meeting the needs of App efficient performance testing.

[0034] In an optional implementation, creating a test task and configuring a test object includes:

[0035] The tester creates a test task and configures the test object through the human-computer interaction interface. The test task includes a random stress test task and its corresponding first preset test package and a specified path stress test task and its corresponding second preset test package.

[0036] The present invention designs two types of test tasks, including random stress testing tasks and specified path stress testing tasks, which can comprehensively cover various performance testing requirements of the App and greatly satisfy the user experience.

[0037] In an optional implementation, scheduling is performed based on the test task and the test object to determine the target test task, including:

[0038] When the test task is a random stress test task, monitor the current test status of all mobile terminals, screen candidate mobile terminals corresponding to the idle state of the current test status, determine the to-be-tested page of the target App from all candidate mobile terminals based on the whitelist, and determine the target test task based on the to-be-tested page of the target App and the first preset test package, wherein the whitelist is a list of to-be-tested pages in the to-be-tested App configured in the random stress test task;

[0039] When the test task is a specified path stress testing task, determine the candidate mobile terminal corresponding to the specified path stress testing task, determine its corresponding target App according to the test task when the candidate mobile terminal is in an idle state, and determine the target test task based on the target App and the second preset test package.

[0040] The present invention designs corresponding scheduling processes for different test task types to determine target test tasks, which not only improves the accuracy of target test tasks, but also realizes effective scheduling of all mobile terminals on the test platform, while improving the rationality of target test tasks, which helps to ensure the stability of subsequent App performance tests and further enhances user experience.

[0041] In an optional implementation, the App performance test is performed according to the target test task to obtain corresponding test results, including:

[0042] Install the first preset test package or the second preset test package included in the target test task to the mobile terminal corresponding to the target App to obtain the target mobile terminal;

[0043] The target mobile terminal performs an App performance test on the target App according to the first preset test package or the second preset test package to obtain corresponding test results, wherein the test results at least include analysis results corresponding to crash data, physical examination data, and code coverage.

[0044] The present invention installs the first preset test package or the second preset test package included in the target test task to the target mobile terminal corresponding to the target App, and performs App performance test on the target App according to the installed test package, and obtains corresponding test results composed of crash data, physical examination data and corresponding analysis results of code coverage, which not only improves the comprehensiveness and accuracy of the test results, but also improves the quality of App performance testing, thereby enhancing user experience.

[0045] In an optional implementation, before performing App performance testing according to the target test task and obtaining corresponding test results, the test process of the automated performance testing platform further includes:

[0046] Check whether the execution time of the test task is configured;

[0047] If yes, then the target execution time corresponding to the test task is obtained, and when the target execution time is met, the App performance test is performed according to the target test task to obtain the corresponding test result process;

[0048] If not, execute the process of performing App performance testing according to the target test task and obtain the corresponding test results.

[0049] The present invention determines whether a user has configured the scheduled execution of a test task by detecting whether the execution time of the test task is configured, which can improve the flexible execution of the test task and further enhance the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 is a schematic diagram of the structure of an automated performance testing platform according to an embodiment of the present invention;

[0052] Figure 2 is a schematic diagram of the structure of another automated performance testing platform according to an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a test flow of an automated performance testing platform according to an embodiment of the present invention;

[0054] Figure 4 is a test flow diagram of another automated performance testing platform according to an embodiment of the present invention;

[0055] Figure 5 It is the structural diagram of the automated performance testing platform;

[0056] Figure 6 It is a schematic diagram of test task creation;

[0057] Figure 7 This is a test result query diagram. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] In this embodiment, an automated performance testing platform is provided. Figure 1 Schematic diagram of the structure of the automated performance testing platform according to an embodiment of the present invention. Figure 1As shown, the test platform includes a front-end server and a back-end server. The front-end server is used to provide a human-computer interaction interface. The back-end server includes a master server and multiple slave servers, wherein the master server is used to schedule each slave server, and each slave server is used to test the corresponding Apps on multiple mobile terminals. The test platform also includes: a task creation module 1, multiple task scheduling modules 2 and a task testing module 3. The task creation module 1 is set on the front-end server, and each task scheduling module 2 and task testing module 3 are respectively set on each slave server.

[0060] In this embodiment, the task creation module 1 is used to create a test task and configure a test object, where the test object is an App to be tested on at least one mobile terminal.

[0061] It should be noted that the specific contents of the test tasks and the App to be tested in this embodiment are not limited here, and can be adaptively determined according to actual needs.

[0062] In this embodiment, the task scheduling module 2 performs scheduling based on the test tasks and test objects of the task creation module 1, determines the target test task and sends it to the corresponding task testing module 3, so that the task testing module 3 performs App performance testing according to the target test task and obtains corresponding test results.

[0063] It should be noted that the task scheduling module 2 corresponding to the scheduling function and the realization of the App performance testing function designed by the automated performance testing platform in this embodiment can be completed with the help of relevant tools, such as the Jenkins tool. It should be noted that Jenkins is an open source continuous integration and deployment tool that provides a friendly operation interface. It is mainly used for continuous and automatic building / testing of software projects and monitoring the operation of external tasks. The application of Jenkins in continuous integration / continuous delivery includes configuring PHP code testing, packaging, and integration processes with GitLab, Docker, and Kubernetes.

[0064] The automated performance testing platform in the embodiment of the present invention realizes the separation of the front-end server and the back-end server, and designs a master-multiple-slave server architecture on the back-end server, wherein each slave server is used to test the corresponding Apps on multiple mobile terminals; each slave server is provided with a task scheduling module and a task testing module, wherein the task scheduling module schedules the test tasks and configured test objects through the test creation module set on the front-end server, determines the target test tasks and sends them to the corresponding task testing module, so that the task testing module performs App performance testing according to the target test tasks and obtains corresponding test results, which not only realizes the fully automatic execution of App performance testing, but also ensures the stability and quality of App performance testing, comprehensively improves the testing efficiency and user experience, and meets the needs of App automated performance testing.

[0065] In a specific embodiment, Figure 2 FIG. 1 is a schematic diagram of the structure of another automated performance testing platform according to an embodiment of the present invention. Figure 2 It can be seen that the task creation module 1 includes: a first creation unit 11 , a second creation unit 12 , a first management unit 13 and a second management unit 14 .

[0066] In this embodiment, the first creation unit 11 is used to create a random stress test task and configure the App to be tested on the corresponding mobile terminal.

[0067] It should be noted that performance testing is to test the various performance indicators of the test object by simulating a variety of normal, peak and abnormal load conditions through automated testing tools. Stress testing belongs to performance testing, which is to obtain the maximum service level that the test object can provide by determining the bottleneck or unacceptable performance points of the test object.

[0068] In this embodiment, the stress random test (i.e., random stress test task, or random stress test) refers to the behavior of the automated program simulating human gestures such as clicks, slides, and drags to perform random execution on the APP page. It can simulate many complex test scenarios, and each execution may enter any page. Among them, the stress random test mainly uses related test tools, such as the probability model of Fastboot to guide GUI testing (i.e., testing of the graphical user interface). Specifically, by designing random stress tests, the robustness of APP automated performance testing can be improved.

[0069] It should be noted that Fastboot is a model-based testing tool used to model GUI transitions. It simulates user operations and detects the response and performance of applications under different operations to discover stability issues of applications on the user interface. Its working principle is based on a probabilistic model. It models the operations that users may perform and simulates user behaviors in various scenarios, including normal operations and abnormal operations. Through these simulated operations, Fastboot can predict the performance of applications in different situations and thus discover potential problems. Fastboot also combines machine learning and reinforcement learning technologies to continuously learn and optimize test results, aiming to improve the accuracy and efficiency of the test.

[0070] In this embodiment, the second creation unit 12 is used to create a stress testing task for a specified path and configure an App to be tested on the corresponding mobile terminal.

[0071] It should be noted that the designated path test (i.e., designated path stress test task, or designated path stress test) in this embodiment refers to the developer writing the designated automation code and sending it to the application (i.e., the App to be tested), and the App to be tested performs the corresponding automated test according to the established test steps. Specifically, the designated path test can also be implemented with the help of relevant testing tools, such as the appium tool.

[0072] It should be noted that appium is an open source tool for automating mobile applications. It supports iOS and Android platforms. Developers can use appium to write test scripts in various programming languages ​​(such as Java, Python, Ruby, etc.) to automate the testing of mobile application functions and user interfaces. In addition, since appium uses the WebDriver protocol to drive various mobile devices, it can be integrated with various testing frameworks and tools, such as JUnit, TestNG, appium Desktop, etc.

[0073] In this embodiment, the first management unit 13 is used to manage all servers.

[0074] In this embodiment, the second management unit 14 is used to manage all mobile terminals.

[0075] It should be noted that the management of the server or mobile terminal in this embodiment includes but is not limited to the addition and deletion of devices. Specifically, this embodiment uses the job in the Jenkins tool to manage the mobile terminal (such as a mobile phone), where one mobile phone corresponds to one job, and the job function can be used to manage the mobile phone, such as adding, modifying, deleting, queuing, etc.

[0076] In the embodiment of the present invention, a task creation module including a first creation unit, a second creation unit, a first management unit and a second management unit is designed to create random stress testing tasks or specified path stress testing tasks and configure corresponding apps to be tested, as well as corresponding management of all servers and mobile terminals on the platform. This can improve the test platform's flexible control over test tasks and included devices, and further enhance the performance stability testing experience of the APP.

[0077] In a specific embodiment, Figure 2 FIG. 1 is a schematic diagram of the structure of another automated performance testing platform according to an embodiment of the present invention. Figure 2 It can be seen that the task scheduling module 2 includes: a state monitoring unit 21 , a state recording unit 22 , a timing configuration unit 23 , a first association unit 24 and a second association unit 25 .

[0078] In this embodiment, the state monitoring unit 21 is used to monitor the current test states of all mobile terminals to obtain state monitoring results.

[0079] It should be noted that the current test status of the mobile terminal in this embodiment is essentially the task scheduling status at the current moment, that is, whether a mobile terminal is in a test state or an idle state at the current moment. The specific detection method can refer to the adaptive setting of the conventional data monitoring method in this field.

[0080] In this embodiment, the state recording unit 22 is used to record the script execution log.

[0081] It should be noted that, in this embodiment, the state recording unit 22 is mainly used to record relevant operation logs during the operation of different scripts, which can be used for subsequent relevant analysis of the test results.

[0082] In this embodiment, the timing configuration unit 23 is used to configure the execution time of the test task.

[0083] It should be noted that, in this embodiment, the timing configuration unit 23 is mainly used to configure the timing task, that is, the specific execution time of the test task.

[0084] In this embodiment, the first associating unit 24 is used to associate the test object of the task creation module with the corresponding mobile terminal according to the status monitoring result of the status monitoring unit.

[0085] It should be noted that, in this embodiment, the first association unit 24 actually refers to the real machine job association, that is, the mobile terminal (that is, the real machine, or mobile phone) corresponding to the App to be tested set in the test task is associated with its corresponding job (using the job management in the Jenkins tool, one job corresponds to at least one real machine).

[0086] In this embodiment, the second associating unit 25 is used to associate the slave server corresponding to the mobile terminal associated with the real machine with the front-end server.

[0087] It should be noted that, in this embodiment, the second association unit 25 substantially refers to the association between the front-end server (also called the front-end system) and the slave server (in which the Jenkins tool is integrated) corresponding to the mobile terminal associated with the real machine in the back-end server.

[0088] In the embodiment of the present invention, a task scheduling module including a status monitoring unit, a status recording unit, a timing configuration unit, a first association unit and a second association unit is designed. The module realizes effective scheduling of all mobile terminals on the test platform in combination with the created test tasks, and generates corresponding reasonable target test tasks for subsequent APP performance testing. The task scheduling module provides a comprehensive, effective and high-quality automated testing management and control solution for App performance, which greatly ensures the stability of App performance testing and further enhances the user experience.

[0089] In a specific embodiment, Figure 2 FIG. 1 is a schematic diagram of the structure of another automated performance testing platform according to an embodiment of the present invention. Figure 2 It can be seen that the task test module 3 includes: an installation unit 31 and a test unit 32 .

[0090] In this embodiment, the installation unit 31 is used to provide a test package corresponding to the target test task, and install the test package to the corresponding mobile terminal to obtain the target mobile terminal.

[0091] It should be noted that, in this embodiment, the installation unit 31 provides a test package link to indicate the download path of the test package.

[0092] In this embodiment, the testing unit 32 performs an App performance test on the App to be tested according to the test package of the target mobile terminal to obtain a corresponding test result.

[0093] It should be noted that the content included in the test results in this embodiment is not limited here and can be adaptively adjusted according to actual needs. For example, the test results include response time, memory usage, CPU (Central Processing Unit) occupancy, etc., which is only for exemplary description.

[0094] It should be noted that the test results in this embodiment need to be obtained through relevant components to obtain the corresponding performance parameters and analyze them. Therefore, this embodiment has a preset performance component built into the target mobile terminal, which is used to collect and analyze the performance data in the App performance test process, and generate corresponding test results based on the analysis results, wherein the preset performance component at least includes a stability component, a performance physical examination component, a code coverage component and a data analysis component. The stability component is used to monitor the App crash and obtain the corresponding crash data; the performance physical examination component is used to collect preset performance indicators and obtain corresponding physical examination data; the code coverage component is used to collect code coverage. It should be noted that the specific contents of the preset performance indicators and preset performance components in this embodiment are not limited here, and can be adaptively adjusted according to actual needs. Specifically, by collecting and analyzing the performance data of the App performance test process through a variety of performance components built into the target mobile terminal, and generating corresponding test results based on the analysis results, it not only helps to ensure the comprehensiveness of the test results, but also improves the accuracy and quality of the performance test.

[0095] In the embodiment of the present invention, an automatic and effective App performance test of the App to be tested is achieved by designing a task test module including an installation unit and a test unit, which can comprehensively improve the quality and efficiency of App automated performance testing.

[0096] In an embodiment of the present invention, a test process embodiment of an automated performance testing platform is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0097] In this embodiment, a test process of an automated performance testing platform is provided, which is applied to the automated performance testing platform of this embodiment. Figure 3 Schematic diagram of the test flow of the automated performance testing platform according to an embodiment of the present invention. Figure 3 As shown, the test process includes the following steps:

[0098] Step S301, create a test task and configure a test object.

[0099] It should be noted that in this embodiment, the creation and configuration of specific test tasks are completed through the human-computer interaction interface provided by the front-end system. The specific content of the human-computer interaction interface is adaptively set according to the actual project requirements. For example, the human-computer interaction interface includes a login interface, a task creation interface, a task configuration interface, etc.

[0100] Step S302: Scheduling based on the test task and the test object to determine the target test task.

[0101] In this embodiment, the target test task includes the App to be tested and its corresponding test content.

[0102] Step S303, perform App performance test according to the target test task and obtain corresponding test results.

[0103] The embodiment of the present invention automatically schedules the created test tasks and test objects to obtain corresponding target test tasks, and performs App performance testing according to the target test tasks to obtain corresponding test results. This can realize the fully automatic execution of the App performance testing process, help to ensure the stability and quality of the App performance testing, thereby comprehensively improving the testing efficiency and user experience, and at the same time meeting the needs of efficient App performance testing.

[0104] In this embodiment, a test process of an automated performance testing platform is provided, which is applied to the automated performance testing platform of this embodiment. Figure 4 FIG. 1 is a schematic diagram of a test flow of another automated performance test platform according to an embodiment of the present invention. Figure 4 As shown, the test process includes the following steps:

[0105] Step S401, create a test task and configure a test object.

[0106] Specifically, the above step 401 includes:

[0107] In step S4011, the tester creates a test task and configures a test object through a human-computer interaction interface. The test task includes a random stress test task and its corresponding first preset test package and a specified path stress test task and its corresponding second preset test package.

[0108] In this embodiment, the first preset test package and the second preset test package can obtain corresponding installation packages through the set corresponding download addresses.

[0109] The embodiment of the present invention designs two types of test tasks, including random stress testing tasks and specified path stress testing tasks, which can comprehensively cover various performance testing requirements of the App and greatly satisfy the user experience.

[0110] Step S402: Scheduling based on the test task and the test object to determine the target test task.

[0111] It should be noted that the test tasks of this embodiment include random stress test tasks and specified path stress test tasks, and different scheduling processes need to be designed accordingly to determine the target test tasks.

[0112] Specifically, the above step S402 includes:

[0113] Step S4021, when the test task is a random stress test task, monitor the current test status of all mobile terminals, screen the candidate mobile terminals corresponding to the idle state of the current test status, determine the page to be tested of the target App from all candidate mobile terminals based on the whitelist, and determine the target test task based on the page to be tested of the target App and the first preset test package, wherein the whitelist is the list of pages to be tested in the App to be tested configured in the random stress test task.

[0114] In this embodiment, the whitelist refers to the list of pages in the APP that need to be tested. For example, if the "Personal Center Page" is set as the whitelist, other pages will be avoided during the actual test process, and only the "Personal Center Page" will be accessed and tested accordingly.

[0115] It should be noted that the relevant content of the random stress test task in this embodiment is referred to the previous text and will not be repeated here.

[0116] Step S4022, when the test task is a specified path stress testing task, determine the candidate mobile terminal corresponding to the specified path stress testing task, determine its corresponding target App according to the test task when the candidate mobile terminal is in an idle state, and determine the target test task based on the target App and the second preset test package.

[0117] It should be noted that the relevant content of the specified path stress testing task in this embodiment can be found in the previous text and will not be repeated here.

[0118] The embodiment of the present invention designs corresponding scheduling processes for different test task types, which not only improves the accuracy of determining the target test tasks, but also realizes the effective scheduling of all mobile terminals on the test platform, improves the rationality of the target test tasks, ensures the stability of subsequent App performance tests, and further enhances the user experience.

[0119] Step S403, perform App performance test according to the target test task and obtain corresponding test results.

[0120] Specifically, the above step S403 includes:

[0121] Step S4031: Install the first preset test package or the second preset test package included in the target test task to the mobile terminal corresponding to the target App to obtain the target mobile terminal.

[0122] In this embodiment, the specific installation process of the first preset test package and the second preset test package is implemented according to a conventional installation method in the art.

[0123] Step S4032: The target mobile terminal performs an App performance test on the target App according to the first preset test package or the second preset test package to obtain corresponding test results, wherein the test results at least include analysis results corresponding to crash data, physical examination data, and code coverage.

[0124] In this embodiment, crash data, physical examination data and code coverage are obtained through relevant components. For example, the BI component used to monitor App crash data is a self-developed collection tool focusing on mobile application crash data, supporting multiple platforms such as Android and iOS, and has the function of automatically collecting crash logs; through this component, developers can quickly collect crash data to locate the cause of the crash in time, which can greatly ensure the quality of the software. It should be noted that the relevant components of the physical examination data are determined according to the adaptability of the content it contains, such as the physical examination data contains startup time, page loading time, network traffic consumption (that is, the network traffic usage during the test App use, including the amount of data uploaded and downloaded), CPU occupancy (such as idle, medium, and high load scenarios corresponding to the CPU usage), memory leaks, large files and other physical examination data, and the corresponding components are determined according to each specific element. The code coverage rate of this embodiment essentially refers to the test coverage rate of the code, which is an indicator that measures the scope of test execution from the code level. It specifically measures the completeness of the test by counting how many / which codes are executed in the test. According to different statistical criteria, it is often divided into multiple test coverage types such as statement coverage, branch coverage, modified condition / branch coverage, function coverage and function call coverage. Specifically, the test coverage rate of the code can provide a more accurate quantitative indicator of test completeness.

[0125] In an embodiment of the present invention, the first preset test package or the second preset test package included in the target test task is installed on the target mobile terminal corresponding to the target App, and the target App is subjected to App performance testing according to the installed test package, so as to obtain corresponding test results including crash data, physical examination data and corresponding analysis results of code coverage, which not only improves the comprehensiveness and accuracy of the test results, but also improves the quality of App performance testing, thereby enhancing the user experience.

[0126] It should be noted that the execution time of the mobile terminal App automated test can be set according to the needs, that is, it specifies what the program does at what time, such as executing the test task at night, which can make full use of the night time for automated testing, thereby significantly reducing the labor cost of mobile terminal testing for the enterprise. In this embodiment, the automated test of the mobile terminal App can be configured with a scheduled task through the Jenkins tool. Therefore, before performing the App performance test according to the target test task and obtaining the corresponding test results, the test process of the automated performance testing platform of this embodiment also includes:

[0127] Step A1: Check whether the execution time of the test task is configured.

[0128] Step A2: If yes, then obtain the target execution time corresponding to the test task, and when the target execution time is met, perform the App performance test according to the target test task to obtain the corresponding test result process.

[0129] Step A3: If not, execute the process of performing App performance test according to the target test task to obtain the corresponding test result.

[0130] In the embodiment of the present invention, by detecting whether the execution time of the test task is configured, determining whether the user has configured the scheduled execution of the test task, the flexible execution of the test task can be improved, thereby improving the user experience.

[0131] In a specific embodiment, in view of the lack of comprehensive solutions to the quality, performance and user experience problems in App performance testing, an automated performance testing platform is provided to integrate vertical capabilities such as performance monitoring, crash analysis, and code coverage into automated testing. Specifically, in terms of platform construction, the front end of the platform adopts the popular Vue framework, which can support access from a variety of terminal devices; the back end uses technologies such as PHP and MySQL, and is deployed through Docker, with good migration capabilities; in terms of continuous integration, the platform uses the jenkins tool for scheduling management, realizes multi-functional expansion, can adapt to various complex and diverse scenarios, and supports the management of multiple nodes and multiple mobile devices; in terms of underlying technology, the platform is based on tools such as appium, Fastboot, tidevice, XCTest, etc., and encapsulates Python scripts for issuing automated task data. In the App, data in the automation process is collected through coverage components, performance inspection components, and BI components, and finally a complete automated task report is generated.

[0132] It should be noted that the platform deploys automation functions to multiple Jenkins machines, and uniformly schedules each Jenkins machine for automated testing. Each Jenkins machine can connect to multiple mobile devices to support multi-node automated machine deployment. Specifically, the platform corresponds to a server architecture of a master server (non-Jenkins machine, here is the PHP backend) + multiple Jenkins machines. The master server schedules each Jenkins machine; each Jenkins machine can contain multiple jobs, one of which can correspond to the APP test of a mobile phone.

[0133] It should be noted that tidevice is an iOS automation testing tool that can provide functions such as taking screenshots, obtaining mobile phone information, installing and uninstalling test packages, starting and stopping applications, and obtaining performance data of specified applications. It can provide a complete set of automatic testing solutions and support cross-platform use, including Windows, MacOS and Linux operating systems, which makes it possible to use it in various development environments; this tool is implemented in pure Python through the reverse iOS communication protocol, which enables it to not only perform performance collection and UI automation testing of iOS applications, but also perform iOS automation testing without Mac, greatly improving testing efficiency and convenience.

[0134] It should be noted that XCTest is a testing framework that can be used for testing iOS applications. It is not only used for unit testing of business logic, but also for UI testing that simulates user operations, that is, unit testing, performance testing, and user interface testing. It can provide a hierarchical testing method that allows developers to perform detailed checks on each function in the App to ensure that they can run as expected, thereby improving code quality and maintainability.

[0135] In a specific embodiment, Figure 5 This is the structural diagram of the automated performance testing platform. Figure 5It should be noted that this embodiment is for iOS App testing (adaptably adjusted to Android App testing). Appium and tidevice tools are installed on each Jenkins machine respectively, and Fastboot, XCTest, UITest, BI components, coverage components, performance check components, Easter egg components and tracking components are installed on the mobile phone respectively. Specifically, the appium tool is used to install the App test package on the mobile phone and execute some customized automation process scripts; the tidevice tool is used to delete the App on the mobile phone, determine the device connection status, kill the App process, etc.; XCTest is Apple's native test control, which is used to start the App's automated testing function, and UITest is a test tool for testing the user interface of mobile applications, which is used to provide corresponding test programs; the BI component is used to monitor App crash data; the performance check component is used to output the CPU, memory, and fps (Frame Per Second, smoothness, which is used to measure the smoothness of the application's screen, that is, the number of frames transmitted per second, to evaluate the number of frames of animation or video, to ensure smooth animation or video playback) and other performance data; the coverage component is used to produce code coverage in the automated execution process, and then analyze code stability and program operation stability (that is, to control the code self-test process, detect whether the test code is correct or analyze the program's operation stability in combination with crash data, which is specifically achieved through self-developed coverage scripts, that is, to write corresponding coverage scripts in Python, which are used to perform various automated operations, call various tools, call interfaces, etc., to string together the overall test process); the Easter egg component integrates various development tools / component functions, such as performance collection tools (that is, performance examination components) and coverage tools (that is, coverage components); the tracking component mainly tests the relevant data of the collection process and uses it for data analysis, such as page browsing time, element clicks, page browsing, etc., and analyzes it to obtain corresponding test results.

[0136] In a specific embodiment, based on the above-mentioned automated performance testing platform, vertical capabilities such as performance monitoring, crash analysis, and code coverage are integrated into automated testing to provide a set of testing solutions that improve quality, performance, and user experience, and improve R&D efficiency. The specific process includes:

[0137] In this embodiment, click Automated Test - Random Stress Test - Create Task on the interface of the automation platform, enter "Task Name", "Test Package Link", "Branch Name", "APP Options", "Stress Test Time", "White List", "Real Machine Selection" and "Whether Login is Required" and other information, click OK to generate an automated stress test task, and execute the test task accordingly. Figure 6This is a schematic diagram of test task creation. After the test task is automated, you can view the complete automation report.

[0138] In this embodiment, if you need to create a specified path stress test task in actual application, you only need to click Automation Test-Specify Path-New Task on the interface of the automation platform, and refer to Figure 6 After configuring the relevant information of the test task, it is used to generate the corresponding specified path automation task and automatically execute it to obtain the corresponding test results.

[0139] In this embodiment, Figure 7 This is a test result query diagram. You can query the automated stress test report (i.e., the complete automated report) by clicking on the relevant interface. Figure 7 It can be seen that the automated report covers performance inspection data such as startup time, frame rate / stuttering, CPU, network, UI level, memory / memory leaks, page opening time, large files, and subroutine UI inspection. It should be noted that the automated report also includes code coverage and crash data.

[0140] In summary, the automated performance testing platform realizes the separation of the front-end, back-end, task scheduling and scripts, and has good portability and scalability. Based on this platform, the fully automatic execution process of performance testing can be realized, and integrity report data including code coverage, performance examination data, crash data, etc. can be generated, which is simple and convenient to operate; the test report has the significant advantage of wide coverage.

[0141] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An automated performance testing platform, comprising a front-end server and a back-end server, wherein the front-end server is used to provide a human-computer interaction interface, characterized in that: The backend server includes a master server and multiple slave servers, wherein the master server is used to schedule each slave server, and each slave server is used to test corresponding apps on multiple mobile terminals. The test platform also includes: A task creation module, a plurality of task scheduling modules and a task testing module, wherein the task creation module is arranged on the front-end server, and each task scheduling module and task testing module is arranged on each slave server respectively; The task creation module is used to create a test task and configure a test object, wherein the test object is an App to be tested on at least one mobile terminal; The task scheduling module performs scheduling based on the test tasks and test objects of the task creation module, determines the target test tasks and sends them to the corresponding task testing modules, so that the task testing modules perform App performance testing according to the target test tasks and obtain corresponding test results.

2. The automated performance testing platform according to claim 1, characterized in that: The task creation module includes: a first creation unit, a second creation unit, a first management unit and a second management unit; The first creation unit is used to create a random stress test task and configure the App to be tested on the corresponding mobile terminal; The second creation unit is used to create a stress testing task for a specified path and configure the App to be tested on the corresponding mobile terminal; The first management unit is used to manage all servers; The second management unit is used to manage all mobile terminals.

3. The automated performance testing platform according to claim 1, characterized in that: The task scheduling module includes: a state monitoring unit, a state recording unit, a timing configuration unit, a first association unit and a second association unit; Wherein, the state monitoring unit is used to monitor the current test state of all mobile terminals and obtain the state monitoring result; The state recording unit is used to record the script running log; The timing configuration unit is used to configure the execution time of the test task; The first association unit is used to associate the test object of the task creation module with the corresponding mobile terminal according to the status monitoring result of the status monitoring unit; The second association unit is used to associate the slave server corresponding to the mobile terminal associated with the real machine with the front-end server.

4. The automated performance testing platform according to claim 1, characterized in that: The task test module includes: an installation unit and a test unit; The installation unit is used to provide a test package corresponding to the target test task, and install the test package to the corresponding mobile terminal to obtain the target mobile terminal; The testing unit performs an App performance test on the App to be tested according to the test package of the target mobile terminal to obtain a corresponding test result.

5. The automated performance testing platform according to claim 4, characterized in that: The target mobile terminal has a built-in preset performance component, which is used to collect and analyze performance data during the App performance test and generate corresponding test results based on the analysis results. The preset performance component includes at least a stability component, a performance inspection component, a code coverage component and a data analysis component. The stability component is used to monitor App crashes and obtain corresponding crash data; the performance inspection component is used to collect preset performance indicators and obtain corresponding inspection data; the code coverage component is used to collect code coverage.

6. A test process of an automated performance test platform, applied to the automated performance test platform as claimed in any one of claims 1 to 5, characterized in that: The testing process includes: Create test tasks and configure test objects; Scheduling based on the test task and the test object to determine a target test task; Perform App performance testing according to the target test tasks and obtain corresponding test results.

7. The test process of the automated performance testing platform according to claim 6 is characterized in that: The step of creating a test task and configuring a test object includes: The tester creates a test task and configures a test object through a human-computer interaction interface. The test task includes a random stress test task and its corresponding first preset test package and a specified path stress test task and its corresponding second preset test package.

8. The test process of the automated performance testing platform according to claim 7 is characterized in that: The scheduling based on the test task and the test object to determine the target test task includes: When the test task is a random stress test task, the current test status of all mobile terminals is monitored, candidate mobile terminals corresponding to the idle state of the current test status are screened, and the pages to be tested of the target App are determined from all candidate mobile terminals based on the whitelist, and the target test task is determined based on the pages to be tested of the target App and the first preset test package, wherein the whitelist is a list of pages to be tested in the App to be tested configured in the random stress test task; When the test task is a specified path stress testing task, determine the candidate mobile terminal corresponding to the specified path stress testing task, determine its corresponding target App according to the test task when the candidate mobile terminal is in an idle state, and determine the target test task based on the target App and the second preset test package.

9. The test process of the automated performance testing platform according to claim 8 is characterized in that: The App performance test is performed according to the target test task to obtain corresponding test results, including: Installing the first preset test package or the second preset test package included in the target test task to the mobile terminal corresponding to the target App to obtain the target mobile terminal; The target mobile terminal performs an App performance test on the target App according to the first preset test package or the second preset test package to obtain corresponding test results, wherein the test results at least include analysis results corresponding to crash data, physical examination data, and code coverage.

10. The test process of the automated performance testing platform according to claim 7, characterized in that: Before performing the App performance test according to the target test task and obtaining the corresponding test result, the test process also includes: Check whether the execution time of the test task is configured; If yes, then obtain the target execution time corresponding to the test task, and execute the App performance test according to the target test task when the target execution time is met to obtain the process of the corresponding test result; If not, execute the process of performing App performance testing according to the target test task to obtain corresponding test results.

Images