Application testing method, device and equipment and readable storage medium

Abstract

The invention provides an application testing method, device and equipment and a readable storage medium, and the method comprises the steps: considering a process resource competition relation between tested applications and the common dependence of the tested applications on an underlying service or a basic service of tested equipment; performance data generated by related processes of all tested applications in tested equipment in the testing process are obtained in parallel through multiple threads, and performance data of the tested applications are obtained on the basis of the performance data of the tested applications and the related processes of other tested applications having a process resource competition relation with the tested applications in combination with set performance threshold values of all the related processes. According to the method, the testing result of each tested application is determined by analyzing the interdependence and influence relationship of the tested applications, so that the interdependence and influence among the tested applications can be accurately evaluated, the performance condition of a single tested application is more comprehensively evaluated, and the testing precision and the testing efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of software application technology, and in particular to an application testing method, device, equipment and readable storage medium. Background Art

[0002] At present, in actual applications, such as on all-in-one devices (for example, a learning machine that integrates multiple applications), there is a tight coupling relationship between multiple applications and the underlying operating system and basic services. They are usually developed by the same development team and share the underlying services. This development architecture leads to potential interdependence and influence between different applications. When resource competition occurs or the underlying service resources are tight, it will have obvious and varying impacts on the performance of a single application.

[0003] However, existing technologies lack an effective way to comprehensively evaluate the impact of complex interactions between multiple applications in a complete device on a single application and the performance of the entire system. Traditional performance testing solutions ignore complex dependencies and cannot accurately evaluate the comprehensive impact of specific operating scenarios on single application tests. Existing methods have obvious shortcomings in understanding and solving performance bottlenecks and problems that applications may encounter in actual use. Summary of the invention

[0004] In view of this, in order to solve the above technical problems, the present application provides an application testing method, device, equipment and readable storage medium.

[0005] Specifically, the present application is implemented through the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, there is provided an application testing method, the method comprising:

[0007] Using a multi-threaded approach, the performance data generated by the related processes of N tested applications in the tested device during the test process are obtained in parallel; wherein there is a process resource competition relationship between at least two tested applications;

[0008] For a single application under test, the test result of the application under test is determined based on the performance data of the application under test and its related processes that compete for process resources, combined with the performance thresholds set for each related process.

[0009] Optionally, the related processes include at least an application process created by the device under test and used to run the application under test; and the parallel acquisition of performance data generated by the related processes of N applications under test in the device under test during the test process includes:

[0010] According to the application process name of each application under test, identify the application processes and process identifiers of N applications under test from all processes created by the device under test;

[0011] The performance data of the application processes of N tested applications are collected in parallel at the set collection frequency.

[0012] Optionally, the related processes also include the first K processes that are most occupied by the system of the device under test during the test; and the parallel acquisition of performance data generated by the related processes of N tested applications in the device under test during the test includes:

[0013] Before the set collection frequency is reached each time to trigger performance data collection, the process IDs of the top K processes that occupy the most system resources are obtained;

[0014] According to the process identifiers of the first K processes, when the collection frequency is reached, the parallel collection of the performance data of the first K processes is triggered.

[0015] Optionally, the method is applied to a test device, the test device supports configuration of an Android debug bridge ADB; the ADB establishes a communication connection with the device under test; and the parallel acquisition of performance data generated by related processes of N tested applications in the device under test during the test process includes:

[0016] According to the process identifiers of the related processes of the N tested applications, a thread is created for each related process, and the thread is associated with a corresponding ADB command; the ADB command is used to obtain performance data of the related process;

[0017] When the set collection frequency is reached, the associated ADB commands are executed in parallel using the created multiple threads to obtain the performance data of the related processes.

[0018] Optionally, determining the test result of the tested application includes:

[0019] For the current application under test for which test results are to be generated, when the performance data of the related processes of the application under test that competes for process resources is lower than its performance threshold:

[0020] Generate a first test result of the current application under test, which is used to indicate that the application under test with process resource competition will not affect the current application under test;

[0021] If the performance data of the related process of the current application under test is higher than its performance threshold, the first test result is also used to indicate that the failure of the performance data of the current application under test is caused by the program logic or resource management within the application.

[0022] Optionally, determining the test result of the tested application includes:

[0023] For the current application under test for which test results are to be generated, when the performance data of the related process of the application under test that competes for process resources is higher than its performance threshold:

[0024] Generate a second test result, which indicates that it is necessary to further analyze the reasons for the high occupancy of the performance data of the processes related to the tested application with process resource competition, and analyze whether it is caused by too many requests of the current tested application or other process activities;

[0025] If the performance data of the related process of the current tested application is higher than its performance threshold, the second test result is also used to indicate that further analysis is required to determine whether other operations or process anomalies affect the related process of the current tested application.

[0026] Optionally, determining the test result of the tested application includes:

[0027] Detect whether the top K processes that occupy the most space in the system of the device under test include abnormal processes; the abnormal processes represent processes that should be in a non-calling state or a dormant state under the current test requirements;

[0028] If so, the test results of the tested application also include a third test result, which is used to indicate that there is a problem with the inter-system call of the tested device, and further analysis is needed to determine whether it is caused by a process call error, abnormal behavior of the abnormal process itself, or other potential defects in the system.

[0029] According to a second aspect of an embodiment of the present application, there is provided an application testing device, the device comprising:

[0030] The performance data collection module is used to acquire the performance data generated by the related processes of N tested applications in the tested device in parallel during the test process in a multi-threaded manner; wherein there is a process resource competition relationship between at least two tested applications;

[0031] The test result determination module is used to determine the test result of a single tested application based on the performance data of the tested application and its related processes that have a process resource competition relationship with it, combined with the performance thresholds set for each related process.

[0032] Optionally, the related process includes at least an application process created by the device under test and used to run the application under test; and the performance data collection module is specifically used to:

[0033] According to the application process name of each application under test, identify the application processes and process identifiers of N applications under test from all processes created by the device under test;

[0034] The performance data of the application processes of N tested applications are collected in parallel at the set collection frequency.

[0035] Optionally, the related processes also include the top K processes that are most occupied by the tested device system during the test; the performance data collection module is specifically used to:

[0036] Before the set collection frequency is reached each time to trigger performance data collection, the process IDs of the top K processes that occupy the most system resources are obtained;

[0037] According to the process identifiers of the first K processes, when the collection frequency is reached, the parallel collection of the performance data of the first K processes is triggered.

[0038] Optionally, the method is applied to a test device, the test device supports configuration of an Android debug bridge ADB; the ADB establishes a communication connection with the device under test; and the performance data collection module is specifically used for:

[0039] According to the process identifiers of the related processes of the N tested applications, a thread is created for each related process, and the thread is associated with a corresponding ADB command; the ADB command is used to obtain performance data of the related process;

[0040] When the set collection frequency is reached, the associated ADB commands are executed in parallel using the created multiple threads to obtain the performance data of the related processes.

[0041] Optionally, the test result determination module is specifically used to:

[0042] For the current application under test for which test results are to be generated, when the performance data of the related processes of the application under test that competes for process resources is lower than its performance threshold:

[0043] Generate a first test result of the current application under test, which is used to indicate that the application under test with process resource competition will not affect the current application under test;

[0044] If the performance data of the related process of the current application under test is higher than its performance threshold, the first test result is also used to indicate that the failure of the performance data of the current application under test is caused by the program logic or resource management within the application.

[0045] Optionally, the test result determination module is specifically used to:

[0046] For the current application under test for which test results are to be generated, when the performance data of the related process of the application under test that competes for process resources is higher than its performance threshold:

[0047] Generate a second test result, which indicates that it is necessary to further analyze the reasons for the high occupancy of the performance data of the processes related to the tested application with process resource competition, and analyze whether it is caused by too many requests of the current tested application or other process activities;

[0048] If the performance data of the related process of the current tested application is higher than its performance threshold, the second test result is also used to indicate that further analysis is required to determine whether other operations or process anomalies affect the related process of the current tested application.

[0049] Optionally, the test result determination module is specifically used to:

[0050] Detect whether the top K processes that occupy the most space in the system of the device under test include abnormal processes; the abnormal processes represent processes that should be in a non-calling state or a dormant state under the current test requirements;

[0051] If so, the test results of the tested application also include a third test result, which is used to indicate that there is a problem with the inter-system call of the tested device, and further analysis is needed to determine whether it is caused by a process call error, abnormal behavior of the abnormal process itself, or other potential defects in the system.

[0052] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the above-mentioned application testing method by calling the computer program.

[0053] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned application testing method is implemented.

[0054] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0055] In the technical solution provided by the present application, by considering the process resource competition relationship between N tested applications and their common dependence on the underlying services or basic services of the tested device, the performance data generated by the relevant processes of all tested applications in the tested device during the test process are obtained in parallel through multi-threading. Based on the relevant process performance data of the tested application and other tested applications that have a process resource competition relationship with the tested application, and in combination with the performance thresholds set for each related process, the mutual dependence and influence relationship between multiple tested applications are analyzed to determine the test result of each tested application. The mutual dependence and influence between the tested applications can be accurately evaluated, thereby more comprehensively evaluating the performance status of a single tested application, thereby improving the test accuracy and test efficiency.

[0056] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present application. In addition, any embodiment in the present application does not need to achieve all the above effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] Figure 1 It is a schematic diagram of an application testing method flow chart shown in an exemplary embodiment of the present application;

[0059] Figure 2 It is a flowchart of obtaining the test result of the tested application according to the performance data analysis shown in an exemplary embodiment of the present application;

[0060] Figure 3 It is an overall flow chart of an application testing method using an all-in-one device as an example shown in an exemplary embodiment of the present application;

[0061] Figure 4 is a structural schematic diagram of an application testing device shown in an exemplary embodiment of the present application;

[0062] Figure 5 It is a hardware schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0063] Current testing tools and methods usually test a single application (app) in terms of performance evaluation. When multiple applications need to be performance tested, testers usually adopt the strategy of testing each application separately. The time complexity of this method is as high as O(n), that is, testing n applications requires n independent testing processes, and its testing efficiency is low. Moreover, as the number of applications increases, the time and resource costs required will also increase linearly, increasing the overall burden of testing.

[0064] At present, in actual applications, such as on all-in-one devices (for example, learning machines that integrate multiple applications), there is a close coupling relationship between multiple applications and the underlying operating system and basic services. They are usually developed by the same development team and share the underlying services. This development architecture leads to potential interdependence and influence between different applications. When resource competition occurs or the underlying service resources are tight, it will have a significant and varying impact on the performance of a single application.

[0065] However, the existing technology lacks an effective way to comprehensively evaluate the impact of complex interactions between multiple applications in the whole device on a single application and the performance of the entire system. Traditional performance testing solutions ignore complex dependencies and cannot accurately evaluate the comprehensive impact of specific operating scenarios on single application testing. When it comes to understanding and solving performance bottlenecks and problems that applications may encounter in actual use, existing methods have obvious shortcomings. There is an urgent need for a testing method that can accurately evaluate performance impacts, ensure the stability of all-in-one devices and improve user experience.

[0066] In view of this, the present application proposes an application testing method suitable for simultaneous testing of multiple applications where there are mutual dependencies and calls between the multiple tested applications. This method can not only be applied to application testing in all-in-one devices that integrate multiple applications, such as learning machines, smart terminals, and other devices that require multiple applications to work together, but can also be applied to application testing scenarios where there are complex dependencies / interactions between multiple applications and at least two tested applications share the same underlying services or resources. It can be understood that this method can also be applied to any other applicable testing scenarios. The above is only an example of an exemplary scenario and does not constitute a limitation on the applicable scenarios of the application testing method of this application.

[0067] The application testing method provided in this application can be executed by a testing device or a testing platform, see Figure 1 The schematic diagram of a process of applying a test method is exemplarily shown, and the implementation process thereof may at least include the following steps:

[0068] S101, using a multi-threaded method, in parallel, obtaining performance data generated by related processes of N tested applications in the tested device during the test process; wherein there is a process resource competition relationship between at least two tested applications;

[0069] The device under test refers to the device that needs to be tested for performance, such as a computer, mobile terminal, etc. For example, the device under test can be an all-in-one device that integrates multiple applications. The application under test is installed on the device under test, and the specific application can be determined by the tester according to the test requirements or test scenarios; for example, the current test scenario is a payment process test of placing an order to purchase goods and completing payment in a shopping software, then the application under test can include shopping software and payment software; for another example, the current test scenario is to jump to the course live broadcast application to watch the live broadcast through the live broadcast reminder on the learning software, then the application under test can include learning software and course live broadcast application.

[0070] Related processes refer to a set of processes related to the application under test and running during the test process. They are created when the application under test is started or running, and can be created by the application under test or by other services or components in the device under test that the application under test depends on.

[0071] In an embodiment of the present application, the related process may include an application process created by the startup or running process of the tested application, which is used to execute or process various tasks or services related to the tested application, or may also include a system basic service process that supports the operation of N tested applications in the tested device system, and the system basic service process is used to provide underlying system services, such as memory management, file management, network communication, etc. In addition, in order to better understand the system performance of the tested device in the current test scenario, the related process may also include the top K processes that occupy the most of the entire system of the tested device during the test process, K is a preset integer, and the specific value can be determined according to the test requirements. By monitoring and analyzing these processes, the performance of the tested application in the current test scenario and the system stability of the tested device in supporting the operation of the tested application can be more accurately evaluated. When determining the top K processes, they can be sorted according to different performance indicators, such as CPU usage, memory usage, disk I / O, etc. The tester can select the most appropriate performance indicator to determine the top K processes according to the test objectives and requirements.

[0072] The determination of the relevant processes of the application under test can be achieved by setting the process name or configuring the monitoring rule parameters according to the monitoring requirements. That is, for the performance data of the processes that need to be monitored by the application under test, for static processes, that is, processes that always exist and have fixed names during the startup or operation of the application under test, a collection file containing the process name can be pre-configured. The collection file contains the process names that will be called during the operation of each application under test to ensure that the test tool can accurately identify and monitor the performance data of these processes. For dynamic processes, that is, processes that are dynamically created or destroyed according to actual needs or test progress during the operation of the application under test, such as the top K processes that occupy the most system resources, which are dynamically changing processes, monitoring can be achieved by configuring monitoring rules in the performance testing tool or system monitoring software. The monitoring rules can be defined based on certain characteristics or behavior patterns of the process, such as CPU usage, memory occupancy, communication relationships between processes, etc. By configuring these rules, the test tool can automatically screen and monitor the dynamic processes that meet the conditions, thereby obtaining their performance data.

[0073] The performance test of an application may include multiple aspects, such as response time, throughput, resource occupancy such as CPU, memory, stability, etc. Testers will design corresponding test cases and test scenarios to simulate various situations that users may encounter in actual use, so as to evaluate the performance of the application under test. Based on this, the performance data described in this application represents quantitative information describing the running state of the process, which may include at least one of the CPU usage rate and memory occupancy of a single process, or may further include the total CPU usage rate or total memory occupancy of the entire system of the device under test corresponding to a single process. That is, the performance data may include the CPU usage rate of a single process and the total CPU usage rate of the entire system, or may include the memory occupancy of a single process and the total memory occupancy of the entire system, or may include CPU and memory related performance data at the same time. In addition, the performance data may also include, but is not limited to, the frequency and speed of the process's read and write operations on the disk (disk I / O), the network data transmission rate of the process (network throughput), the time required for the process to process a request or task (response time), etc., which may be dynamically determined according to the current test requirements for N applications under test, and this application does not limit this.

[0074] Process resource competition refers to the competition state generated when multiple processes try to access or use the same resource such as CPU, memory, disk, etc. at the same time. In this embodiment, at least two of the N tested applications compete for the same resource such as CPU usage during the test. The performance of the tested applications with resource competition in the current test scenario will respond to and interfere with each other. Therefore, when performing performance testing, the impact of resource competition needs to be fully considered. For example, the high CPU usage of an application may cause the CPU usage of other applications to be limited, thereby affecting their performance; similarly, an application's frequent access to memory or disk may occupy a large amount of I / O bandwidth, causing other applications to be delayed or blocked when accessing these resources.

[0075] In this embodiment, for the N tested applications that need to be performance tested in the specified test scenario of the tested device, the test cases constructed for the N tested applications in the test scenario are obtained, and then the N tested applications are started and the test cases are run. When the test cases start to run and execute the test, the relevant processes that need to be monitored are identified from the running processes according to the process names or monitoring rules of the relevant processes that need to be monitored for each tested application, and the performance data of the relevant processes of the N tested applications can be collected in a multi-threaded parallel manner at a set collection frequency, that is, the performance data of the relevant processes of the N tested applications are collected at the same time each time until the test process ends. Among them, the test case includes a combination of a series of operations and expected results for verifying whether the performance of the tested application under specific conditions meets the expected results, which can include test titles, test steps, expected results, preconditions, test items, case numbers, importance levels and other elements, and will be designed in detail for the specific functions and performance requirements of each tested application.

[0076] This embodiment uses a multi-threaded approach to implement the parallel collection of performance data of multiple related processes during the test process. Multi-threaded libraries or frameworks in programming languages, such as Java's Thread class and Python's threading module, can be used to create and manage threads. Each thread can be set to be responsible for the performance data collection task of a tested application or a related process, and the thread life cycle and concurrency can be controlled through a thread pool or thread scheduler.

[0077] See also Figure 1 The exemplary parallel collection of related process data shows that after identifying the related processes of N tested applications in the tested device, the data is collected once at a set interval for the multiple related processes whose performance data need to be monitored during the test, and the performance data of all related processes are obtained in parallel at one time. The performance data of each related process obtained each time can be written into a storage file such as a log file according to the collection timestamp to facilitate subsequent test analysis.

[0078] S102, for a single application under test, based on the performance data of the application under test and related processes of the application under test that have a process resource competition relationship with the application under test, combined with the performance thresholds set for each related process, determine the test result of the application under test.

[0079] The performance threshold is set for each relevant process. The performance threshold of each relevant process may include upper limits set for different performance indicators such as CPU usage and memory occupancy, which are used to determine whether the application under test or the relevant process has performance anomalies. When the performance indicator of the relevant process reaches or exceeds its corresponding performance threshold, it means that the activity of the relevant process or the application under test associated with the relevant process may have performance anomalies, and further analysis and processing of the causes of the anomalies, such as resource competition, code efficiency, call anomalies, etc., are required to resolve performance bottlenecks and ensure the stability and response speed of the application under test, as well as the stability of the device system running the application under test. The specific value of the performance threshold of the relevant process of the application under test can be dynamically set according to the expected operating status and expected resource occupancy of the application under test during development, and / or set through expert evaluation or industry standards.

[0080] Based on the fact that there are at least two tested applications in the present application that have a process resource competition relationship, therefore, by monitoring the performance indicators of related processes, the dependencies and mutual influences between them can be evaluated. By judging whether there are performance anomalies in the related processes, it can be used to judge whether the activities of the related processes will have a significant impact on the activities of other processes that are dependent on or mutually influential with them. By considering the mutual dependencies and influences between the tested applications, accurate testing of the tested applications can be achieved.

[0081] In the process of obtaining the test results of a single application under test based on the collected performance data analysis, the performance data of the application under test and the related processes that have a process resource competition relationship with the application under test are compared with their respective preset performance thresholds based on the key indicator of the performance threshold of each related process. Figure 1 As shown, by evaluating whether other processes have an impact on the operation of the current application under test (i.e., external impact analysis), and evaluating whether the current application under test will affect its own performance or other processes due to its own problems (i.e., internal impact and self-impact analysis of the application under test), the results of the external impact analysis and the internal impact analysis are comprehensively considered to comprehensively and accurately evaluate the performance of a single application under test, and generate a test result. The test result is used to indicate whether there are problems in the test operation of the application under test, and the overall detection direction for further analysis when there are problems. The test result is provided to relevant personnel who develop the application under test so that developers can perform specific problem location and resolution work based on the test results, thereby promptly discovering and resolving potential performance bottlenecks and problems, ensuring the stability and response speed of the application under test, and the stability of the device system running the application under test.

[0082] In the disclosed embodiment, the process resource competition relationship between N tested applications and their common dependence on the underlying services or basic services of the tested device are taken into consideration, and the performance data generated by the relevant processes of all tested applications in the tested device during the test process are obtained in parallel through multi-threading. Based on the relevant process performance data of the tested application and other tested applications that have a process resource competition relationship with the tested application, and in combination with the performance thresholds set for each related process, the mutual dependence and influence relationship between the multiple tested applications are analyzed to determine the test result of each tested application. This can not only accurately evaluate the mutual dependence and influence between the tested applications due to the shared underlying services, thereby more comprehensively analyzing the degree of influence on each tested application in different situations, such as when the underlying services are fully occupied, but also pay attention to the performance data of a single application itself and the impact of its relationship with other applications on its performance, so as to more accurately determine the performance status of a single tested application, thereby improving the test accuracy and test efficiency.

[0083] In some embodiments, for the N related processes of the tested application described in the above embodiments, the related processes may at least include the application process created by the tested device and used to run the tested application, that is, the application process pointed to by the process name that is the same as the application package name of the tested application or prefixed with the package name, which is created when the tested application is started and the tested device starts running. For example, if the package name of application A is com.tencent.q1, then when the application is started, the application process com.tencent.q1 or the application process com.tencent.q1:some_suffix with the package name as the prefix is ​​created.

[0084] In the Android system environment, the package name is the unique identifier of the Android application package. The package name is defined in AndroidManifest.xml. The package name is determined when the Android application project is created. When the Android application is started, it runs in its own application process. The process name of the application process is the same as the package name or contains the package name as a prefix.

[0085] Each application under test corresponds to an application process, which contains various elements required for the application to run, such as memory space allocation, permission settings, etc., so that the application can run smoothly in this specific environment. It is responsible for executing the main functions of the application under test, including but not limited to the functional business logic, resource management, sub-process management, interaction with other processes, etc. in the application under test. The application process can also serve as a parent process and include one or more sub-processes. The cover process is created by the application process when needed to perform specific tasks or process specific data.

[0086] Based on this, when the performance data generated by the relevant processes of N tested applications in the tested device during the test are obtained in parallel, the process name can be determined according to the package name of each tested application, and then the application process to be monitored can be identified. That is, according to the application process name of each tested application, the application process and process identifier of each of the N tested applications can be identified from all the processes created by the tested device; the performance data of the application process of each of the N tested applications can be collected in parallel at the set collection frequency.

[0087] Since the application under test is started before the test case is run, the application process of the application under test is created when it is started. Therefore, the process name and process representation of the application process can be determined based on the package name of the application under test before the test case is run for testing. Regarding obtaining the application process name of the application under test through the application package name, taking the Android environment as an example, the ApplicationInfo of all application information installed in the device under test can be obtained through the PackageManager instance to determine the package name of the application under test. Next, ActivityManager is used to obtain all running processes, and the process name and process identifier of the application process of the application under test are obtained by matching the corresponding process information according to the package name.

[0088] In the embodiments of the present disclosure, by detecting the application process of the application under test, the performance data of the application under test during the execution of the test case, such as CPU usage, memory usage, response time, etc., can be directly obtained. These data can most intuitively reflect the actual performance of the application and can eliminate the interference of other irrelevant processes or system processes, thereby improving the accuracy of the test results, helping to more accurately locate performance bottlenecks or problems, and facilitating developers to quickly locate the cause of the problem, shortening the troubleshooting time and thus improving development efficiency.

[0089] In some embodiments, the related processes of the N tested applications described in the aforementioned embodiments may include not only the application processes of each tested application, but also the top K processes that are occupied by the tested device system the most during the test and / or the underlying / basic service processes provided by the tested device system and shared by at least two tested applications. For underlying services or basic service processes, since the process names are fixed and known, a static collection file can be defined and the process names of the underlying / basic service processes that need to be monitored in the current test scenario can be written. During the test, the process names in the static collection file can be traversed, and each underlying / basic service process can be identified from the running processes to obtain performance data regularly.

[0090] In the case where the relevant processes include the top K processes that occupy the most system resources of the device under test during the test, for the parallel acquisition of performance data generated by the relevant processes of N tested applications in the device under test during the test as described in the aforementioned step S102, based on the fact that the specific processes of the top K processes will dynamically change with the progress of the test, in order to ensure the accuracy and relevance of performance data collection, before each time the set collection frequency is reached to trigger performance data collection, the process identifiers of the top K processes that occupy the most current system resources are first obtained; based on the process identifiers of the top K processes, when the collection frequency is reached, the parallel collection of performance data of the top K processes is triggered.

[0091] The set collection frequency indicates the collection frequency of performance data set according to the test requirements. It can be a fixed time interval such as every 1 second, or it can be triggered based on a specific time such as when a test case is executed. Since the resource usage of the process changes in real time, the first K process information obtained before triggering performance data collection may have certain timeliness issues. In order to reduce this impact, the collection frequency can be set as short as possible.

[0092] In this embodiment, before each collection frequency is reached, detailed information of all currently running processes is obtained through system commands or APIs, including process ID PID, process name, CPU usage, memory occupancy, etc.; based on the obtained process information, the top K processes with the highest resource usage such as CPU usage and memory usage are screened out, and their process IDs are recorded. The screening process can be based on a comprehensive consideration of multiple performance indicators or only on a single indicator, and can be flexibly set according to test requirements.

[0093] In the embodiment of the present disclosure, during the test process, the top K processes that occupy the most resources of the tested device system are included in the scope of performance data monitoring. By monitoring the top K processes that occupy the most resources, the overall performance of the tested device during the test process can be obtained, so that performance bottlenecks that may be caused by system-level problems, rather than just application-level problems, can be identified; at the same time, if the top K processes include non-tested applications or system services, it indicates that there is resource competition or system-level conflicts that affect the performance of the tested application. By monitoring the top K processes, potential system conflicts can be identified, and the performance of the tested application in a real environment can be more accurately evaluated, thereby making the test results more comprehensive and accurate, and providing auxiliary information for troubleshooting and positioning, which helps to shorten troubleshooting time and improve test efficiency.

[0094] Regarding the collection of performance data of the relevant processes of the tested application described in the aforementioned step S101, system monitoring tools such as top, htop, vmstat, etc. can be used to obtain performance data in real time, or the built-in monitoring function of the application or the use of special performance testing tools such as JMeter, LoadRunner, etc. can be used to test and collect data. In this embodiment, when the test device is an Android system or supports Android environment configuration or supports configuration of Android debug bridge ADB, based on Android debug bridge ADB as a powerful Android device debugging and management tool, ADB allows developers to communicate with Android devices, including installing and debugging applications, accessing the file system on the device, and obtaining real-time performance data, etc. Therefore, regarding the multi-threaded parallel acquisition described in step S101 in the aforementioned embodiment, ADB can be configured in the test device, and a communication connection between ADB and the tested device can be established. After the communication connection is established, the ADB command can be used to obtain the performance data of multiple related processes in real time. In order to further improve the efficiency of data collection, the multi-threaded parallel mode of ADB is used to collect data from multiple processes at the same time, thereby ensuring that as many performance details as possible can be captured during the test process.

[0095] Based on this, with regard to the parallel acquisition of performance data generated by related processes of N tested applications in the tested device during the test process, an independent thread can be created for each related process according to the process identifiers of the related processes of the N tested applications, and the thread can be associated with the corresponding ADB command. The thread is responsible for collecting the performance data of the related processes associated with each other in parallel with other threads during the test process. The ADB command is used to obtain the performance data of the related processes associated with the thread. When the set collection frequency is reached, the multiple threads that have been created are used to execute the associated ADB commands in parallel, thereby realizing the simultaneous collection of performance data from multiple related processes and obtaining the performance data of the related processes.

[0096] Use different ADB commands for different performance data. For example, you can use adbshell top -p to check CPU usage. <pid>The command obtains the CPU usage of the specified process. The CPU usage of each process can be obtained by parsing the command output. The total CPU usage of the entire system of the device under test can be parsed from the output of adb shell top to obtain the CPU usage summary information or load average field information, thereby obtaining the CPU load of the entire system. For MEM usage, you can use adb shell dumpsys meminfo <pid>command to get the memory usage of the specified process.

[0097] For the first K processes described in the above embodiment, before reaching the set acquisition frequency each time, it is necessary to first obtain the process IDs of the first K processes that occupy the most system resources. For newly appeared processes (i.e., processes that were not in the first K processes before, but now enter the list of the first K processes due to increased resource usage), it is necessary to create a new thread for it and associate the corresponding ADB command; for processes that are no longer in the list of the first K processes, if the corresponding thread is not used again within a period of time such as the set time, the thread resources will be automatically recycled to release system resources. If the process is destroyed, its corresponding thread will also be automatically terminated and recycled by the operating system.

[0098] It can be understood that this embodiment takes the Android environment as an example and adopts ADB combined with multi-threading to realize parallel acquisition of performance data of related processes of N tested applications. For other operating system environments such as IOS systems, functions similar to ADB such as Xcode and Instruments can be used to collect CPU, memory, disk, network and other performance data. This application does not limit this.

[0099] In the disclosed embodiment, by executing ADB commands in parallel, performance data can be collected from multiple related processes at the same time, satisfying the mutual impact analysis when multiple applications are running at the same time, significantly shortening the data collection time, and considering the impact of other applications when analyzing the performance of a single tested application. Combined with the efficiency and comprehensiveness of data collection, the performance of a single tested application can be accurately analyzed. In addition, real-time data collection helps to discover and locate performance problems in a timely manner, which can reduce the time cost of troubleshooting and repairing performance problems.

[0100] In some embodiments, for the single tested application described in the aforementioned step S102, based on the performance data of the tested application and the related processes of the tested application that have a process resource competition relationship with it, combined with the performance thresholds set for each related process, the test result of the tested application is determined. This embodiment takes a current tested application to generate a test result, that is, one of the N tested applications, as an example to explain the process of determining the test result of the tested application. It can be understood that the test result analysis of other tested applications is the same as that of the current tested application in the same manner and principle.

[0101] In this embodiment, at least two of the N tested applications are competing for system resources. Therefore, when performing performance testing and analysis, it is necessary to pay attention to the performance of a single tested application itself, and consider the mutual influence between the current tested application and other applications, such as applications with which it has a process resource competition relationship. Process resource competition may involve key resources such as CPU, memory, disk I / O, and network bandwidth.

[0102] Based on this, see Figure 2 The schematic diagram of the performance data analysis and test result generation process is exemplified. When evaluating a single application under test (i.e., the current application under test), it is possible to first analyze whether there are other processes that may have a significant impact on the process activities of the current application under test, that is, execute step S201 to analyze the relationship between the performance data of the related processes of the application under test that competes with it for process resources and the performance threshold of the related processes of the application under test that competes for process resources.

[0103] When the performance data of the related process of the application under test that competes with it for process resources is lower than its performance threshold, it can be considered that the competing application has not caused a significant performance impact on the current application under test. In this case, step S202a can be executed, that is, generating a first test result for the current application under test. The first test result is used to clearly indicate that the application under test that competes for process resources will not affect the current application under test. On this basis, it can be further pointed out whether its performance is qualified and the possible reasons based on the performance data of the current application under test itself.

[0104] That is, when the performance data of the related process of the tested application that competes with it for process resources is lower than its performance threshold, step S203 is executed to further determine the performance data of the current tested application itself. If the performance data of the related process of the current tested application is also lower than its performance threshold, it indicates that the performance of the current tested application is normal in the absence of external competition interference, that is, the first test result includes analysis content indicating that the performance of the current tested application is normal (step S204a), such as the performance data of the current tested application is qualified or performs normally; if the performance data of the related process of the current tested application is higher than its performance threshold, based on the absence of external competition interference at this time, the performance of the current tested application is normal. The performance problem of the previous application under test may be caused by improper program logic or resource management within the application. Therefore, step S205a is executed to add performance problem analysis content about the current application under test to the first test result that has been generated. That is, the first test result can also include analysis content for problems with the performance of the current application under test and for indicating that the performance data of the current application under test is unqualified due to program logic or resource management within the application, thereby providing possible directions / targeted suggestions for troubleshooting for performance optimization of the current application under test, so that R&D personnel can quickly locate the root cause of the problem and optimize it based on the first test result, thereby improving testing and optimization efficiency.

[0105] For the current application under test for which test results are to be generated, when the performance data of the related processes of the application under test that competes with it for process resources is higher than its performance threshold, it means that the competing applications are occupying more system resources, which will have an adverse impact on the performance of the current application under test. In this case, step S202b can be executed to generate a second test result. The main purpose of the second test result is to indicate that R&D-related personnel need to further analyze the reasons for the high occupancy of the performance data of the related processes of these competing applications. That is, the second test result includes analysis content used to indicate the need to further analyze the reasons for the high occupancy of the performance data of the related processes of the application under test that competes for process resources, and to analyze whether it is caused by too many requests from the current application under test or other process activities, so as to provide analysis direction for R&D-related personnel.

[0106] Next, execute step S203 to further determine the performance data of the current application under test. If the performance data of the related process of the current application under test is lower than its performance threshold, it indicates that the performance of the current application under test is normal in the presence of external resource competition interference, and the second test result may also include analysis content indicating that the performance of the current application under test is normal (step S204b); if the performance data of the related process of the current application under test is also higher than its performance threshold at this time, it is impossible to determine whether the abnormal performance data of the current application under test is due to the competing application occupying too many system resources or due to the internal logic of the application or other process abnormalities. Therefore, step S205b can be executed to improve the second test result, so that the second test result can also be used to indicate the need for further analysis of whether there are other operations or process abnormalities affecting the related processes of the current application under test, so as to instruct R&D personnel to deeply analyze the root cause of the performance abnormality of the current application under test and solve the problem.

[0107] Among them, when the performance data of the relevant processes of the current application under test and other applications under test with which there is resource competition are both higher than their performance thresholds, it is also possible to further determine whether the abnormal performance data of the relevant processes of the current application under test and other applications under test with which there is resource competition is caused by the abnormal process activity of the other party by analyzing the time difference of the abnormal time point according to the abnormal time point when the performance data of the relevant processes of the current application under test and other applications under test with which there is resource competition is caused by the abnormal process activity of the other party. For example, if the performance abnormality of the relevant processes of the current application under test and other applications under test with which there is resource competition occurs almost simultaneously or very close in time, it indicates that the resource competition between the two is the common cause of the performance degradation; if the performance abnormality of one application under test occurs before the abnormality of another application under test and lasts for a long time, it can be inferred that the performance abnormality of the latter application may be caused by the abnormal activity of the former application (such as a sudden increase in resource usage).

[0108] It can be understood that, for the aforementioned embodiment, the performance data of the relevant process of the tested application / current tested application that competes with the existing process resources is higher than its performance threshold, it can be understood that among all the relevant processes of the tested application / current tested application that has the resource process, there is at least one relevant process whose performance data is higher than the performance threshold set for the performance indicator of the relevant process in one of the performance indicators of the performance data. For example, the performance data of the relevant process of the current tested application includes two performance indicator dimensions, namely, CPU usage rate and response time. If the response time of the relevant process is higher than its performance threshold and lasts for a set period of time, it is considered that the performance data of the relevant process of the current tested application is higher than its performance threshold.

[0109] In this embodiment, the analysis step of whether the performance data of the current application under test and the processes related to the application under test that compete with the application under test for process resources are higher than the performance threshold is as follows: Figure 2 What is shown is only an exemplary description of the order in which the steps are executed. The present application does not limit the order in which the two steps are executed. The performance data of the relevant processes of the current application under test may be analyzed first to see if they are higher than their performance thresholds, and the corresponding first or second test results may be generated. The corresponding first or second test results may be further supplemented based on the fact that the performance data of the relevant processes of the application under test that competes with the application for process resources are higher than their performance thresholds.

[0110] In the disclosed embodiment, by analyzing whether other tested applications that are in a resource competition relationship with the current tested application, i.e., competing applications, have a significant impact on the current tested application, a comprehensive evaluation is made of the impact of external factors on the performance of the current tested application, and the root cause of the abnormal performance of the current tested application is inferred, providing direction for R&D personnel to locate performance abnormalities, so that R&D personnel can conduct more targeted testing and avoid blindly testing all possible influencing factors, which not only saves testing time and resources, but also improves testing efficiency and ensures the accuracy and reliability of test results.

[0111] In some embodiments, when the performance data of the relevant processes collected include the performance data of the top K processes that occupy the most of the system of the device under test and are collected regularly at a set collection frequency during the test, the behavior patterns and resource occupancy of these key processes can be further analyzed based on these data, so as to more accurately evaluate the performance of the application under test and its impact on system resources. For the test result of the application under test as described in the aforementioned step S102, it can also include a step of detecting whether the top K processes that occupy the most of the system of the device under test include abnormal processes, wherein the abnormal process represents a process that should be in a non-calling state or a dormant state under the current test requirements, and the abnormal activity of these processes may indicate improper allocation of system resources or conflicts between applications.

[0112] For example, in the test scenario of video playback, if a voice call process appears among the top K processes that occupy the most system resources, and the test requirement at this time is to play the video and does not involve the voice call function, the voice call process can be regarded as an abnormal process. The appearance of the voice call process means that the operating system of the device under test mistakenly allocates resources to unnecessary processes, or there is an improper calling relationship between processes, resulting in incorrect occupation of resources.

[0113] When the top K processes that occupy the most space in the system include abnormal processes, that is, when a voice call process that should not appear appears in the above video playback scenario, the test result of the application under test may also include a third test result, which is used to indicate that there is a problem with the inter-system call of the device under test, indicating that the relevant R&D personnel need to further analyze whether it is caused by a process call error or abnormal behavior of the abnormal process itself or other potential defects in the system. When generating the third test result, the third test result may exist at the same time as the aforementioned first test result and belong to the test result of the application under test, or may also exist at the same time as the second test result and belong to the test result of the application under test.

[0114] In the embodiment of the present disclosure, by detecting whether the top K processes that occupy the most space in the system of the device under test include abnormal processes, potential problems that are difficult to discover using conventional testing methods can be obtained. Since abnormal processes represent improper allocation of system resources, conflicts between applications, or process call errors, this detection method can significantly improve the accuracy and depth of the test, helping R&D personnel to have a more comprehensive understanding of the performance of the application under test.

[0115] In some embodiments, after using multiple threads to parallelly acquire the performance data of the related processes, when the performance data of the process to be monitored is obtained, the performance data of each process to be monitored can be written into a log file according to the time point of acquiring the performance data, so that the performance data of the related processes of each tested application can be clearly identified and distinguished; after the test is completed, the data in the log file is converted into a format that is easier to analyze or visualize, such as including converting the data from the original log format to CSV, JSON or database table format, etc., and the log file can be traversed in the order of acquisition time points, and each performance data can be integrated according to the acquisition time point, the tested application and process identifier, and the data format of the performance indicator to obtain the final data file, which contains the CPU, memory and other performance data of all tested applications during the test, and the data should be sorted and organized according to chronological order and App identifier.

[0116] After obtaining the integrated performance data, the integrated performance data can be converted into chart data of the data display type according to the data display type; the chart can be rendered and displayed on the display page of the test device to achieve a visual display of the performance data. For example, the performance data of each process can be converted into a line chart in the order of the test time and then displayed. If the same process contains multiple performance indicators, it will contain multiple line charts. Through the chart display, the resource usage changes of each related process can be more intuitively observed.

[0117] To facilitate storage and backup, the integrated performance data, i.e. the final data file mentioned above, can be reported and stored to the cloud service and stored locally on the device under test according to the cloud reporting path and local storage path, respectively, to facilitate local quick browsing or further data analysis and processing.

[0118] In order to enable those skilled in the art to better understand the application testing method provided by the present application, the method is exemplified below by taking the testing of N tested applications in a scenario where an all-in-one device integrates multiple APPs (Applications).

[0119] When testing a scenario of the all-in-one device process, the pre-test work includes creating test cases, connecting devices, and configuring related processes that need to be monitored by the tested application. Figure 3 As shown, after creating the test cases and tasks, they can be distributed to the all-in-one device through the test device or test platform, and ADB can be configured for device connection and failure retry mechanism.

[0120] The relevant processes that need to be monitored by the configured application under test can be composed of two parts: the first part is the top K processes that currently occupy the most resources in the system of the all-in-one device; the second part is the application process created when the application under test is started and is dedicated to running the application under test, as well as other processes that the tester assesses to be of high importance, such as the underlying / basic service process that supports the operation of multiple applications under test.

[0121] Continue to refer Figure 3 As shown, when collecting the performance data of the relevant processes during the test, the performance data of the above two parts of the processes are collected in parallel in a multi-threaded manner according to the set collection frequency, such as once every two seconds. The collection frequency of the two parts can be the same or change within a certain time range. Since the processes in the first part are dynamic processes, the first K processes will continue to change with the progress of the test. Therefore, when collecting the first K processes during the test, it is necessary to determine which processes are the first K processes of the current system before reaching the set collection frequency each time, and then trigger a collection of the performance data of the newly determined first K processes when the set collection frequency is reached. The processes in the second part are static processes. The tester can write the process name fixedly before starting the test through the collection file, so as to read the collection file during the test and collect the performance data of each process specified by the collection file in parallel according to the set collection frequency.

[0122] When obtaining the performance data of the relevant processes that need to be monitored by the tested application, the performance data may include but is not limited to the CPU usage and memory usage of a single process. In order to grasp the overall system performance of the tested device at the same time, the total memory usage of the entire system can also be collected. Among them, if the ADB command is used to obtain performance data, the CPU usage and total CPU usage can be obtained through the adb top command, the memory usage of a single process can be obtained through the adbdumpys meminfo command, and the total memory usage can be obtained through the cat procmeminfo command.

[0123] After the test is completed, the performance data of the relevant processes collected and stored in the log file is scattered data. You can traverse the log file and integrate the data in the form of time-app and process-performance data (such as CPU usage-memory usage-total CPU usage-total memory usage) through the dimensions of time and app / process. Get the final data file such as a JSON file and upload it to the cloud storage. At the same time, you can combine the test equipment or test platform capabilities to display the performance data in the form of a visual chart.

[0124] There are mutual influences and dependencies between multiple tested applications integrated based on the all-in-one device during operation. Therefore, in the process of testing and analyzing the performance data collected during the test process to construct the test results, it is necessary to consider the resource competition and dependencies between different tested applications. For example, multiple tested applications share the resources of an underlying service process a of the all-in-one device. The performance of the underlying service process affects the operating efficiency of multiple tested applications. If one tested application occupies more resources of the underlying service process, it will inevitably affect other processes calling the underlying service process, such as causing blocking and response delays.

[0125] Therefore, performance thresholds can be set for the performance data of the relevant processes monitored in this test, and the test results of the tested application can be constructed by analyzing the relationship between the performance data of the tested application and other tested applications that compete with the tested application for resources and their performance thresholds.

[0126] For example, a threshold value x% can be set for the underlying service process a as a reference standard for evaluating the performance of other processes. If the resource usage of process a is less than x%, it can be considered that it will not have a significant impact on other processes. In this case, if the performance data of an application (such as app1) is unqualified, the test results include analysis content indicating that the performance anomaly problem lies in the program logic or resource management within app1.

[0127] However, if the resource usage of process a exceeds x%, the test results include an indication that the high usage needs further analysis, such as whether it is due to too many requests from app1 or the activities of other processes. In this case, even if the performance data of app1 is unqualified, it cannot be directly concluded that the performance of the internal program of app1 is unqualified. Further investigation is needed to see whether there are other operations or processes that abnormally affect process a, thereby affecting the performance of app1.

[0128] For a process b with a specific function, such as only responsible for voice recognition, in a scenario where no voice operation is performed, the resource usage of process b should be minimal, or there should be no activity at all. If abnormal resource usage of process b is detected, the test results can also include analysis content indicating that there is a problem with the inter-system call, and can further provide suggestions for possible cause analysis directions, such as possible causes including incorrect process calls, abnormal behavior of process b (such as failure to end normally or recycle resources), or other potential defects in the system. In this case, the test results are submitted to R&D personnel to enable them to view system logs, track inter-process communications, analyze process code logic, etc., conduct in-depth investigation and analysis to determine the root cause of abnormal resource usage, and take corresponding optimization measures, such as fixing code defects, optimizing resource management strategies, adjusting process scheduling priorities, etc., to ensure the stability and efficiency of the system.

[0129] Through the above steps, the test result performance of a single app and the impact of the interaction between the single app and its related processes on the performance of the entire system can be systematically analyzed and evaluated, which helps to identify and resolve performance bottlenecks and provides a powerful tool for optimizing application performance and improving user experience, thereby improving the performance and reliability of the entire system.

[0130] Corresponding to the above-mentioned embodiment of the application test method, see Figure 4 As shown, the present application also provides an embodiment of an application test device, the device comprising:

[0131] The performance data collection module 401 is used to acquire, in parallel, the performance data generated by the related processes of N tested applications in the tested device during the test process in a multi-threaded manner; wherein there is a process resource competition relationship between at least two tested applications;

[0132] The test result determination module 402 is used to determine the test result of a single tested application based on the performance data of the tested application and its related processes that have a process resource competition relationship with the tested application, combined with the performance thresholds set for each related process.

[0133] In some embodiments, the related process includes at least an application process created by the device under test and used to run the application under test; the performance data collection module is specifically used to:

[0134] According to the application process name of each application under test, identify the application processes and process identifiers of N applications under test from all processes created by the device under test;

[0135] The performance data of the application processes of N tested applications are collected in parallel at the set collection frequency.

[0136] In some embodiments, the related processes also include the top K processes that are occupied most by the tested device system during the test; the performance data collection module is specifically used to:

[0137] Before the set collection frequency is reached each time to trigger performance data collection, the process IDs of the top K processes that occupy the most system resources are obtained;

[0138] According to the process identifiers of the first K processes, when the collection frequency is reached, the parallel collection of the performance data of the first K processes is triggered.

[0139] In some embodiments, the method is applied to a test device, the test device supports configuration of an Android debug bridge ADB; the ADB establishes a communication connection with the device under test; the performance data collection module is specifically used to:

[0140] According to the process identifiers of the related processes of the N tested applications, a thread is created for each related process, and the thread is associated with a corresponding ADB command; the ADB command is used to obtain performance data of the related process;

[0141] When the set collection frequency is reached, the associated ADB commands are executed in parallel using the created multiple threads to obtain the performance data of the related processes.

[0142] In some embodiments, the test result determination module is specifically used to:

[0143] For the current application under test for which test results are to be generated, when the performance data of the related processes of the application under test that competes for process resources is lower than its performance threshold:

[0144] Generate a first test result of the current application under test, which is used to indicate that the application under test with process resource competition will not affect the current application under test;

[0145] If the performance data of the related process of the current application under test is higher than its performance threshold, the first test result is also used to indicate that the failure of the performance data of the current application under test is caused by the program logic or resource management within the application.

[0146] In some embodiments, the test result determination module is specifically used to:

[0147] For the current application under test for which test results are to be generated, when the performance data of the related process of the application under test that competes for process resources is higher than its performance threshold:

[0148] Generate a second test result, which indicates that it is necessary to further analyze the reasons for the high occupancy of the performance data of the processes related to the tested application with process resource competition, and analyze whether it is caused by too many requests of the current tested application or other process activities;

[0149] If the performance data of the related process of the current tested application is higher than its performance threshold, the second test result is also used to indicate that further analysis is required to determine whether other operations or process anomalies affect the related process of the current tested application.

[0150] In some embodiments, the test result determination module is specifically used to:

[0151] Detect whether the top K processes that occupy the most space in the system of the device under test include abnormal processes; the abnormal processes represent processes that should be in a non-calling state or a dormant state under the current test requirements;

[0152] If so, the test results of the tested application also include a third test result, which is used to indicate that there is a problem with the inter-system call of the tested device, and further analysis is needed to determine whether it is caused by a process call error, abnormal behavior of the abnormal process itself, or other potential defects in the system.

[0153] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0154] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application. Ordinary technicians in this field can understand and implement it without creative work.

[0155] The present application also provides an electronic device. The structural diagram of the electronic device is as follows: Figure 5 As shown, the electronic device 500 includes at least one processor 501, a memory 502 and a bus 503, and at least one processor 501 is electrically connected to the memory 502; the memory 502 is configured to store at least one computer-executable instruction, and the processor 501 is configured to execute the at least one computer-executable instruction, thereby executing the steps of any application testing method provided in any embodiment or any optional implementation manner in the present application.

[0156] Furthermore, the processor 501 may be a Field-Programmable Gate Array (FPGA) or other devices with logic processing capabilities, such as a Microcontroller Unit (MCU) or a Central Process Unit (CPU).

[0157] An embodiment of the present application also provides another readable storage medium storing a computer program, which is used to implement the steps of any application testing method provided in any embodiment or any optional implementation manner in the present application when executed by a processor.

[0158] The readable storage medium provided in the embodiments of the present application includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the readable storage medium includes any medium that can store or transmit information in a readable form by a device (e.g., a computer).

[0159] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0160] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.< / pid> < / pid>

Claims

1. An application testing method, characterized in that: The method comprises: Using a multi-threaded approach, the performance data generated by the related processes of N tested applications in the tested device during the test process are obtained in parallel; wherein there is a process resource competition relationship between at least two tested applications; For a single application under test, the test result of the application under test is determined based on the performance data of the application under test and its related processes that compete for process resources, combined with the performance thresholds set for each related process.

2. The method according to claim 1, characterized in that The related processes at least include an application process created by the device under test and used to run the application under test; The parallel acquisition of performance data generated by related processes of N tested applications in the tested device during the test process includes: According to the application process name of each application under test, identify the application processes and process identifiers of N applications under test from all processes created by the device under test; The performance data of the application processes of N tested applications are collected in parallel at the set collection frequency.

3. The method according to claim 1 or 2, characterized in that: The related processes also include the top K processes that are most occupied by the system of the device under test during the test; The parallel acquisition of performance data generated by related processes of N tested applications in the tested device during the test process includes: Before the set collection frequency is reached each time to trigger performance data collection, the process IDs of the top K processes that occupy the most system resources are obtained; According to the process identifiers of the first K processes, when the collection frequency is reached, the parallel collection of the performance data of the first K processes is triggered.

4. The method according to claim 1, characterized in that The method is applied to a test device, which supports configuration of an Android debug bridge ADB; the ADB establishes a communication connection with the device under test; The parallel acquisition of performance data generated by related processes of N tested applications in the tested device during the test process includes: According to the process identifiers of the related processes of the N tested applications, a thread is created for each related process, and the thread is associated with the corresponding ADB command; The ADB command is used to obtain performance data of related processes; When the set collection frequency is reached, the associated ADB commands are executed in parallel using the created multiple threads to obtain the performance data of the related processes.

5. The method according to claim 1, characterized in that Determining the test result of the tested application includes: For the current application under test for which test results are to be generated, when the performance data of the related processes of the application under test that competes for process resources is lower than its performance threshold: Generate a first test result of the current application under test, which is used to indicate that the application under test with process resource competition will not affect the current application under test; If the performance data of the related process of the current application under test is higher than its performance threshold, the first test result is also used to indicate that the failure of the performance data of the current application under test is caused by the program logic or resource management within the application.

6. The method according to claim 1 or 5, characterized in that: Determining the test result of the tested application includes: For the current application under test for which test results are to be generated, when the performance data of the related process of the application under test that competes for process resources is higher than its performance threshold: Generate a second test result, which indicates that it is necessary to further analyze the reasons for the high occupancy of the performance data of the processes related to the tested application with process resource competition, and analyze whether it is caused by too many requests of the current tested application or other process activities; If the performance data of the related process of the current tested application is higher than its performance threshold, the second test result is also used to indicate that further analysis is required to determine whether other operations or process anomalies affect the related process of the current tested application.

7. The method according to claim 3, characterized in that Determining the test result of the tested application includes: Detect whether the top K processes that occupy the most space in the system of the device under test include abnormal processes; the abnormal processes represent processes that should be in a non-calling state or a dormant state under the current test requirements; If so, the test results of the tested application also include a third test result, which is used to indicate that there is a problem with the inter-system call of the tested device, and further analysis is needed to determine whether it is caused by a process call error, abnormal behavior of the abnormal process itself, or other potential defects in the system.

8. An application testing device, characterized in that: The device comprises: The performance data collection module is used to acquire the performance data generated by the related processes of N tested applications in the tested device in parallel during the test process in a multi-threaded manner; wherein there is a process resource competition relationship between at least two tested applications; The test result determination module is used to determine the test result of a single tested application based on the performance data of the tested application and its related processes that have a process resource competition relationship with it, combined with the performance thresholds set for each related process.

9. An electronic device, characterized in that: include: Memory, processor; The memory is used to store computer programs; The processor is used to call the computer program to implement the method according to any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images