Test method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a test method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a test set corresponding to a to-be-tested terminal, and enabling the test set to comprise at least one test subset in a first operation time sequence, each test subset comprises at least one to-be-tested function module in a second operation time sequence; and testing the to-be-tested terminal according to the test set to obtain at least one test result. By testing the plurality of to-be-tested function modules at the same time, the testing efficiency is improved. According to the method, a unified joint test mode is carried out based on the plurality of test subsets and the plurality of to-be-tested function modules, so that simulation of a scene of multi-application and multi-function joint use is facilitated, analysis of comprehensive performance problems in the scene of multi-application and multi-function use is facilitated, and test effectiveness and reference of test results are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of testing technology, and specifically to a testing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Currently, when testing Android devices (electronic devices such as mobile phones, tablets, and televisions whose operating systems are Android), customized scripts are basically executed to achieve this.

[0003] These scripts are usually developed for fixed modules to be tested. For example, when a test requirement for a module to be tested is generated, a test script for the module to be tested is developed separately based on the test requirement, such as opening specific applications one by one to test the startup time of each application, or executing specific commands to obtain system status information in real time.

[0004] However, since the current tests are mainly aimed at a single module to be tested, the test efficiency is low, and each module to be tested is tested separately. There is a lack of test analysis of the linkage usage between multiple modules to be tested, resulting in limited reference value of the test results. Summary of the Invention

[0005] The embodiments of the present application provide a testing method, apparatus, electronic device, and computer-readable storage medium, which can improve testing efficiency and enhance the reference value of test results.

[0006] In a first aspect, an embodiment of the present application provides a testing method, the method comprising:

[0007] Acquire a test set corresponding to the terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each of the test subsets includes at least one functional module to be tested in a second operating sequence;

[0008] A test operation is performed on the terminal to be tested according to the test set to obtain at least one test result.

[0009] In a second aspect, an embodiment of the present application further provides a testing device, the device comprising:

[0010] an acquisition module, configured to acquire a test set corresponding to the terminal to be tested, the test set including at least one test subset in a first operating sequence, and each of the test subsets including at least one functional module to be tested in a second operating sequence;

[0011] The testing module is configured to perform a testing operation on the terminal to be tested according to the test set to obtain at least one test result.

[0012] Optionally, in some embodiments of the present application, obtaining a test set corresponding to the terminal to be tested includes:

[0013] Obtain performance anomaly feedback information;

[0014] Identifying at least one scenario to be tested from the performance abnormality feedback information through feature extraction and determining a first operating timing corresponding to each scenario to be tested;

[0015] Determining at least one functional module to be tested included in each of the scenarios to be tested and determining a second operating sequence corresponding to each functional module to be tested, wherein each scenario to be tested corresponds to one test subset;

[0016] The test set is generated according to each of the functional modules to be tested in each of the scenarios to be tested, as well as the first operating timing and the second operating timing.

[0017] Optionally, in some embodiments of the present application, determining at least one functional module to be tested included in each of the scenarios to be tested includes:

[0018] Determining at least one type of function module to be tested included in each of the scenarios to be tested and determining module parameters corresponding to each type of function module to be tested;

[0019] For each type of the functional module to be tested, the functional module to be tested is determined according to the type of the functional module to be tested and the corresponding module parameters.

[0020] Optionally, in some embodiments of the present application, obtaining a test set corresponding to the terminal to be tested includes:

[0021] In response to a first interface screening operation on the single sequence list interface, determining a single sequence corresponding to at least one scenario to be tested and determining an initial running timing corresponding to each single sequence, wherein each single sequence corresponds to one of the test subsets;

[0022] In response to the sorting operation on the single sequence, obtaining a first running timing after adjustment of the initial running timing;

[0023] A test set is obtained according to each of the single sequences and the first running timing corresponding to each of the single sequences.

[0024] Optionally, in some embodiments of the present application, before determining, in response to the first interface screening operation, the single sequences corresponding to the at least one scenario to be tested, and determining the initial running timing corresponding to each of the single sequences, the method further includes:

[0025] In response to the second interface screening operation on the module type list interface, determining the type of the functional module to be tested, and displaying the module configuration interface corresponding to the type of the functional module to be tested;

[0026] In response to an interface configuration operation on the module configuration interface, determining module parameters;

[0027] Determining a function module to be tested for the type of function module to be tested according to the module parameters;

[0028] A single sequence is generated according to at least one of the functional modules to be tested and the second operating timing of each of the functional modules to be tested.

[0029] Optionally, in some embodiments of the present application, performing a test operation on the terminal to be tested according to the test set to obtain at least one test result includes:

[0030] Performing a test operation on the terminal to be tested according to the test set to obtain a comprehensive test result;

[0031] And, if the test set further includes an associated test subset, generating an associated test result for the associated test subset, wherein the associated test subset includes at least two test subsets having an associated relationship;

[0032] The comprehensive test result and the associated test result are respectively used as test results.

[0033] Optionally, in some embodiments of the present application, after performing a test operation on the terminal to be tested according to the test set and obtaining at least one test result, the method further includes:

[0034] generating a third operating timing according to the first operating timing, wherein the third operating timing is different from the first operating timing;

[0035] generating a reference set according to each of the test subsets and the third running timing;

[0036] Determining a reference result corresponding to the reference set;

[0037] The performance evaluation results of the functional modules to be tested are determined according to the test results and the reference results.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the above-mentioned test method are implemented.

[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned testing method are implemented.

[0040] In a fifth aspect, embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in the embodiments of the present application.

[0041] An embodiment of the present application obtains a test set corresponding to the terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence. A test operation is performed on the terminal to be tested according to the test set to obtain at least one test result.

[0042] Among them, the test efficiency is improved by testing multiple functional modules to be tested at the same time.

[0043] Among them, the unified joint testing method based on multiple test subsets and multiple functional modules to be tested helps to simulate the scenario of joint use of multiple applications and multiple functions, helps to analyze the comprehensive performance problems under the scenario of multi-application and multi-function usage, and improves the effectiveness of the test and the reference value of the test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 This is a schematic diagram of a scenario in which a terminal device according to an embodiment of the present application executes the test method;

[0046] Figure 2 Schematic diagram of the test method provided in the embodiment of the present application;

[0047] Figure 3 This is a schematic diagram of a single sequence list interface provided in an embodiment of the present application;

[0048] Figure 4 This is a schematic diagram of the interface of a single sequence provided in an embodiment of the present application;

[0049] Figure 5This is a schematic diagram of the module type list interface provided in an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of a module configuration interface provided in an embodiment of the present application;

[0051] Figure 7 is a schematic structural diagram of a testing device provided in an embodiment of the present application;

[0052] Figure 8 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] The embodiments of the present application provide a test method, device, electronic device and computer-readable storage medium. Specifically, the embodiments of the present application provide a test device suitable for electronic equipment, for improving test efficiency and improving the referenceability of test results. Specifically, the electronic device includes a terminal device or a server, and the terminal device includes but is not limited to a mobile phone, a computer, a television and other main control devices with command receiving and issuing functions. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms and other basic cloud computing services, etc. The server can be directly or indirectly connected via wired or wireless communication.

[0055] See also Figure 1 , Figure 1 : is a schematic diagram of a scenario in which a terminal device according to an embodiment of the present application executes the test method. The specific execution process of the terminal device executing the test method is as follows:

[0056] The terminal device 10 obtains a test set corresponding to the terminal to be tested 11, which test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence. The terminal device 10 performs a test operation on the terminal to be tested 11 according to the test set to obtain at least one test result.

[0057] For example, a user establishes a communication connection between the terminal device 10 and the terminal to be tested 11, and the user operates the terminal device 10 to initiate a test operation, and then the terminal device 10 obtains the test set corresponding to the terminal to be tested 11, and then the terminal device 10 tests the terminal device based on the test set to obtain the test results of each functional module to be tested in the test set.

[0058] In summary, the embodiments of the present application improve testing efficiency by simultaneously testing multiple functional modules to be tested. The unified joint testing method based on multiple test subsets and multiple functional modules to be tested helps simulate scenarios where multiple applications and multiple functions are used together, helps analyze comprehensive performance issues in these scenarios, and improves the effectiveness of the test and the reference value of the test results.

[0059] The following are detailed descriptions. It should be noted that the order of description of the following embodiments does not limit the priority order of the embodiments. Figure 2 , Figure 2 A flowchart of a test method provided in an embodiment of the present application. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the flowchart. Specifically, the process of the test method specifically includes:

[0060] 101. Obtain a test set corresponding to a terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence.

[0061] That is, the test set includes multiple test subsets, each test subset has a first running sequence, and each test subset includes multiple functional modules to be tested, each functional module to be tested has a second running sequence.

[0062] The terminals to be tested include devices such as televisions, refrigerators, air conditioners or mobile phones.

[0063] It should be emphasized that the test subset is the sum of a series of functional modules to be tested, that is, a test subset includes multiple functional modules to be tested. In an embodiment of the present application, the multiple functional modules to be tested are usually combined into a usage scenario. For example, for the scenario of sending a message through the XX communication tool, the scenario corresponds to a test subset. Accordingly, the test subset includes multiple functional modules such as opening the XX communication tool, responding to the edit to obtain the information to be sent, and sending the information to be sent. The usage scenario or scenario is also called the scenario to be tested, that is, the scenario to be tested corresponding to the test subset of the embodiment of the present application. In addition, the usage scenario can also include a scenario of closing the current application and starting another application, etc., wherein the scenario can be a collection of multiple functional modules or functional operations. For example, multiple functional modules in a short period of time are combined to obtain a usage scenario.

[0064] Furthermore, a test set is a collection of usage scenarios, that is, a collection of scenarios to be tested. For example, this test set includes four scenarios: opening settings, adjusting screen brightness, sending messages via XX communication tool, and sliding the terminal desktop. It should be noted that this test set is usually based on a combination of multiple scenarios used continuously for a short period of time during terminal use.

[0065] The first and second operating sequences refer to the order of execution, and in the present embodiment, they primarily refer to the order of test execution. The purpose of the first and second operating sequences is to simulate real-world usage conditions to test the terminal under test, so as to effectively analyze problems encountered by the terminal under test during use, such as performance issues, logic issues, security issues, and compatibility issues.

[0066] 102. Perform a test operation on the terminal to be tested according to the test set to obtain at least one test result.

[0067] Among them, since it involves testing multiple test subsets or multiple functional modules to be tested at the same time, multiple test results can be flexibly generated based on the test requirements, for example, test results corresponding to the entire test set, test results of a single functional module to be tested, or test results corresponding to a single test subset, etc.

[0068] In summary, the embodiments of the present application improve testing efficiency by simultaneously testing multiple functional modules to be tested. The unified joint testing method based on multiple test subsets and multiple functional modules to be tested helps simulate scenarios where multiple applications and multiple functions are used together, helps analyze comprehensive performance issues in these scenarios, and improves the effectiveness of the test and the reference value of the test results.

[0069] It is understandable that the terminal test requirements can be generated based on performance feedback. For example, based on feedback received about performance anomalies, corresponding test requirements are generated, and a test set is determined based on the test requirements. That is, optionally, in some embodiments of the present application, the step of "obtaining a test set corresponding to the terminal to be tested" includes:

[0070] Obtain performance anomaly feedback information;

[0071] Identifying at least one scenario to be tested from the performance abnormality feedback information through feature extraction and determining a first operating timing corresponding to each scenario to be tested;

[0072] Determining at least one functional module to be tested included in each scenario to be tested and determining a second running sequence corresponding to each functional module to be tested, wherein each scenario to be tested corresponds to a test subset;

[0073] A test set is generated according to each functional module to be tested in each scenario to be tested, as well as the first running timing and the second running timing.

[0074] Among them, performance abnormality feedback information refers to some performance abnormality problems reported by the terminal to be tested during use. The performance abnormality feedback information is usually obtained from actual users or testers. For example, the performance abnormality feedback information includes "Enter the settings, adjust the screen brightness to the maximum, then open the XX communication tool, and send a message to a friend, then return to the terminal desktop, slide the terminal desktop back and forth, and there will be a freeze" or "When switching from the XX communication tool to the video player, the video playback freezes or the video player restarts" and so on.

[0075] Among them, since the performance abnormality feedback information contains keywords of the usage scenario, for example, the aforementioned performance abnormality feedback information contains scene-related keywords such as "enter settings", "adjust screen brightness to maximum", "XX communication tool", "return to terminal desktop" and "switch", therefore, the test scenario set in the performance abnormality feedback information can be identified by semantic understanding of the problem description or keyword extraction, and the first operating sequence can be determined by the order of appearance of each test scenario in the performance abnormality feedback information.

[0076] Semantic understanding and keyword extraction can be processed by machine learning models or deep learning models. Using machine learning models or deep learning models to achieve semantic understanding and keyword extraction can help improve the accuracy of scene recognition, thereby improving the effectiveness of the terminal test.

[0077] Among them, after determining the test scenario corresponding to the performance anomaly feedback information, the test functional modules corresponding to each test scenario can be determined. For example, based on the description of each test scenario recorded in the performance anomaly feedback information, the functional modules involved can be identified. The identification of the functional modules can also be achieved through semantic understanding or keyword recognition.

[0078] In addition, since the types of functional modules to be tested corresponding to each scenario to be tested are generally relatively fixed, it is possible to first identify the type of functional module to be tested, then identify the module parameters from the performance anomaly feedback information, and then comprehensively determine the functional module to be tested based on the type and parameters of the functional module to be tested to improve the accuracy of identifying the functional module to be tested. That is, optionally, in some embodiments of the present application, the step of "determining at least one functional module to be tested included in each scenario to be tested" includes:

[0079] Determining at least one type of function module to be tested included in each of the test scenarios and determining module parameters corresponding to each type of function module to be tested;

[0080] For each type of functional module to be tested, the functional module to be tested is determined according to the type of functional module to be tested and corresponding module parameters.

[0081] For example, in the scenario of launching video player AA and playing video YY, the identification determines that the functional modules to be tested include "application launch" and "video playback," and that the module parameters include AA for "application launch" and YY for "video playback." This results in two functional modules to be tested: "launching application AA" and "playing video YY."

[0082] Among them, in the embodiment of the present application, it is also possible to pre-establish a mapping relationship between each scenario to be tested and each type of functional module to be tested, determine the type of functional module to be tested corresponding to each scenario to be tested based on the mapping relationship, and then determine the specific functional module to be tested based on the type of functional module to be tested in combination with the module parameters identified from the performance abnormality feedback information. That is, according to the mapping relationship, it is determined that the scenario of starting the video player AA and playing the YY video involves the types of functional modules to be tested including "application startup" and "video playback", and the model parameters are identified through semantic understanding or keyword recognition, including AA for "application startup" and YY for "video playback".

[0083] In summary, the embodiments of the present application implement simulated testing of abnormal issues and improve test effectiveness by identifying the scenarios to be tested and the functional modules to be tested from performance anomaly feedback information, and then generating a test set for testing. Furthermore, in the process of generating a test set based on performance anomaly feedback information, automated testing based on performance anomaly feedback information is implemented by identifying the scenarios to be tested and the functional modules to be tested, thereby improving test efficiency.

[0084] When the device to be tested is tested according to the first operating sequence and the second operating sequence, the test scripts of the functional modules to be tested may be called in sequence, and the tests of the functional modules to be tested may be executed in sequence.

[0085] Optionally, in an embodiment of the present application, the test set may also be obtained by providing a GUI interface and through user screening configuration. That is, optionally, in some embodiments of the present application, the step of "obtaining a test set corresponding to the terminal to be tested" includes:

[0086] In response to a first interface screening operation on the single sequence list interface, determining single sequences corresponding to at least one scenario to be tested and determining an initial running time corresponding to each single sequence, wherein each single sequence corresponds to a test subset;

[0087] In response to a sorting operation on a single sequence, a first running time sequence after adjustment of the initial running time sequence is obtained;

[0088] A test set is obtained according to each single sequence and the first running timing corresponding to each single sequence.

[0089] Among them, a single sequence is a sequence obtained by arranging the various functional modules to be tested in the test subset according to the running sequence. The running sequence corresponding to the single sequence is also the running sequence of the corresponding test subset (or test scenario) during the test. For example, single sequence 1 is executed first, and single sequence 2 is executed later.

[0090] Correspondingly, the test set corresponds to a multi-sequence, that is, the multi-sequence is obtained by combining multiple single sequences. For example, see Figure 3 , Figure 3 This is a schematic diagram of the single sequence list interface provided in an embodiment of the present application. It shows multiple single sequences, each with a name such as "ZZ Polling Startup Test," "Cold Start + Conventional Applications," "4 TOP Applications Polling," and "Launch XX Communication Tool While Playing a Movie." These single sequences then form a multi-sequence named "Multi-Scenario Performance Joint Test."

[0091] Accordingly, users can also adjust the order of individual sequences based on test requirements to obtain a multi-sequence with the first runtime timing. For example, in the single sequence list interface, the position of each sequence can be adjusted up or down to adjust the order of each sequence, adjust the initial runtime timing, and ultimately obtain a multi-sequence with the first runtime timing.

[0092] Accordingly, each single sequence can also be obtained through interface configuration. For example, the functional modules to be tested included in each single sequence are obtained through interface configuration, and the single sequence is obtained based on the running timing of each functional module to be tested. That is, optionally, in some embodiments of the present application, after the step of "determining, in response to the first interface screening operation, the single sequence corresponding to at least one scenario to be tested, and determining the initial running timing corresponding to each single sequence", the method further includes:

[0093] In response to the second interface screening operation on the module type list interface, determining the type of the functional module to be tested, and displaying a module configuration interface corresponding to the type of the functional module to be tested;

[0094] In response to an interface configuration operation on a module configuration interface, determining module parameters;

[0095] Determine the function module to be tested according to the module parameters for the type of the function module to be tested;

[0096] A single sequence is generated according to at least one functional module to be tested and the second operating timing of each functional module to be tested.

[0097] For example, see Figure 4 , Figure 4 This is a schematic diagram of the interface of a single sequence provided in an embodiment of the present application. The interface shows a single sequence with the sequence name "Launch XX communication tool while playing a movie". The single sequence is composed of multiple functional modules to be tested, such as "Wait 3 seconds", "Launch application AA" and "Play video YY", and each functional module to be tested can obtain a new running sequence based on the user's up and down adjustments in the interface, and finally obtain a single sequence with a second running sequence.

[0098] In the embodiment of the present application, the user can obtain the function module to be tested by configuring the function module type to be tested and the module parameters corresponding to each function module type to be tested. For example, see Figure 5 , Figure 5 This is a schematic diagram of the module type list interface provided by the embodiment of the present application. The module type list interface shows multiple types of function modules to be tested, such as "Start Application", "Local Video", "Online Video" and "System Restart". Accordingly, the user can select any type of function module to be tested through the second interface screening operation in this interface to configure and obtain the function module to be tested. For example, please refer to Figure 6 , Figure 6This is a schematic diagram of a module configuration interface provided by an embodiment of the present application. Taking the "Launch Application" function module type as an example, in this module configuration interface, the "Launch AA" function module to be tested can be obtained by configuring the "Launch Application AA" module parameters. The module parameters can be selected from a parameter set or customized.

[0099] Among them, since it involves testing of multiple scenarios to be tested or multiple functional modules to be tested, the types of test results are more diverse. For example, an overall test result can be generated, or a test result of part of the scenarios to be tested or a single functional module to be tested can be generated. That is, optionally, in some embodiments of the present application, the step of "performing a test operation on the terminal to be tested according to the test set to obtain at least one test result" includes:

[0100] Execute test operations on the terminal to be tested according to the test set to obtain comprehensive test results;

[0101] And, if the test set further includes an associated test subset, generating an associated test result for the associated test subset, where the associated test subset includes at least two test subsets having an associated relationship;

[0102] The comprehensive test results and the correlation test results are respectively used as test results.

[0103] The comprehensive test result is the test result for the test set as a whole. The associated test result is the test result for test subsets that have an associated relationship. For example, for two test subsets that have an associated relationship, a test result combining the two test subsets is generated. That is, for the test conditions of the two test subsets, a separate test result containing only the two test subsets is generated.

[0104] In this embodiment of the present application, the existence of associations between test subsets can be determined based on multiple dimensions, such as whether the scenario types of the test scenarios corresponding to the test subsets are consistent, whether there is a continuous time sequence relationship, whether the high-frequency activation conditions are close, and whether the occupied resources are consistent. Furthermore, associated test subsets can also be configured based on user needs. For example, two or more associated test subsets can be identified by the same symbol.

[0105] In the embodiment of the present application, if the test of the device under test includes a performance test, the test results can display the performance data of each functional module under test for each resource. Furthermore, in the generated test results, the performance issues of each functional module under test can be compared horizontally based on the performance data of each resource included. For example, the performance of two functional modules under test can be analyzed based on their performance data of CPU, memory, IO, and frame rate.

[0106] Among them, in the embodiment of the present application, since multiple test scenarios or multiple test functional modules are tested according to the first operating sequence and the second operating sequence based on the test set, the test results can reflect the performance changes of each test functional module during use. Compared with the separate test of a single test functional module, the solution of the embodiment of the present application is more likely to discover performance problems under the continuous use of multiple applications and multiple functions.

[0107] In the embodiments of the present application, in order to more fully perform performance testing under the continuous use of multiple applications and multiple functions, reference sets of other runtime sequences may be generated to sequentially simulate various multi-application and multi-function usage scenarios. That is, optionally, in some embodiments of the present application, the step of "performing a test operation on the terminal to be tested according to the test set to obtain at least one test result" includes:

[0108] generating a third running timing according to the first running timing, where the third running timing is different from the first running timing;

[0109] generating a reference set according to each test subset and the third running time sequence;

[0110] Determine a reference result corresponding to the reference set;

[0111] Determine the performance evaluation results of each functional module to be tested based on the test results and reference results.

[0112] For example, the third running sequence is obtained by adjusting the first running sequence, and then the reference set is obtained. For example, if the test set includes 5 single sequences of the first running sequence being "ZZ polling startup test", "cold start + regular application", "4 TOP application polling", "starting XX communication tool during movie playing" and "starting XX communication tool during movie playing", then the reference set can be the 5 single sequences of the third running sequence being "cold start + regular application", "ZZ polling startup test", "4 TOP application polling", "starting XX communication tool during movie playing" and "starting XX communication tool during movie playing", or the 5 single sequences of "starting XX communication tool during movie playing", "ZZ polling startup test", "cold start + regular application", "4 TOP application polling" and "starting XX communication tool during movie playing", etc.

[0113] The performance evaluation result refers to the performance evaluation of the functional module under test, including whether there are performance issues or not. The performance evaluation result is obtained by comparing the test results of the functional module under test at different runtimes. For example, if the performance of the functional module under test does not meet the standard at any runtime, the functional module under test is considered to have a performance issue. Alternatively, if the performance does not meet the standard at any single runtime, the functional module under test is considered to have a performance issue at that runtime.

[0114] By generating a reference set and testing it, performance issues with a low probability of occurring only under certain usage timings can be screened out by comparing the test results of the reference set with those of the test set.

[0115] In summary, the embodiments of the present application improve testing efficiency by simultaneously testing multiple functional modules to be tested. The unified joint testing method based on multiple test subsets and multiple functional modules to be tested helps simulate scenarios where multiple applications and multiple functions are used together, helps analyze comprehensive performance issues in these scenarios, and improves the effectiveness of the test and the reference value of the test results.

[0116] The embodiments of the present application identify the test scenarios and functional modules to be tested from performance anomaly feedback information, and generate a test set for testing, thereby achieving simulated testing of abnormal problems and improving test effectiveness. Furthermore, in the process of generating a test set based on performance anomaly feedback information, automated testing based on performance anomaly feedback information is achieved by identifying the test scenarios and functional modules to be tested, thereby improving test efficiency.

[0117] Among them, by providing a GUI interface and responding to the user's screening configuration, the configuration of the test set is realized and the user's customized test operation is realized.

[0118] Among them, generating related test reports not only improves the diversity of test results, but also helps to comprehensively analyze each functional module under test in the device under test, and helps to locate performance problems.

[0119] Among them, by generating test sets and reference sets for multiple runtimes and testing them separately, it is helpful to compare and analyze the test results of multiple runtimes and identify performance problems that have a low probability of occurring only under specific runtimes.

[0120] To facilitate better implementation of the test method of the present application, the present application also provides a test device based on the above test method. The meanings of the terms are the same as those in the above test method, and the specific implementation details can be referred to the description in the method embodiment.

[0121] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of the test device provided in an embodiment of the present application. The test device can be specifically as follows:

[0122] An acquisition module 201 is configured to acquire a test set corresponding to a terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence;

[0123] The testing module 202 is configured to perform a test operation on the terminal to be tested according to the test set to obtain at least one test result.

[0124] Optionally, in some embodiments of the present application, obtaining a test set corresponding to the terminal to be tested includes:

[0125] Obtain performance anomaly feedback information;

[0126] Identifying at least one scenario to be tested from the performance abnormality feedback information through feature extraction and determining a first operating timing corresponding to each scenario to be tested;

[0127] Determining at least one functional module to be tested included in each scenario to be tested and determining a second running sequence corresponding to each functional module to be tested, wherein each scenario to be tested corresponds to a test subset;

[0128] A test set is generated according to each functional module to be tested in each scenario to be tested, as well as the first running timing and the second running timing.

[0129] Optionally, in some embodiments of the present application, determining at least one functional module to be tested included in each scenario to be tested includes:

[0130] Determining at least one type of function module to be tested included in each of the test scenarios and determining module parameters corresponding to each type of function module to be tested;

[0131] For each type of functional module to be tested, the functional module to be tested is determined according to the type of functional module to be tested and corresponding module parameters.

[0132] Optionally, in some embodiments of the present application, obtaining a test set corresponding to the terminal to be tested includes:

[0133] In response to a first interface screening operation on the single sequence list interface, determining single sequences corresponding to at least one scenario to be tested and determining an initial running time corresponding to each single sequence, wherein each single sequence corresponds to a test subset;

[0134] In response to a sorting operation on a single sequence, a first running time sequence after adjustment of the initial running time sequence is obtained;

[0135] A test set is obtained according to each single sequence and the first running timing corresponding to each single sequence.

[0136] Optionally, in some embodiments of the present application, in response to the first interface screening operation, before determining the single sequences corresponding to at least one scenario to be tested, and determining the initial running timing corresponding to each single sequence, the method further includes:

[0137] In response to the second interface screening operation on the module type list interface, determining the type of the functional module to be tested, and displaying a module configuration interface corresponding to the type of the functional module to be tested;

[0138] In response to an interface configuration operation on a module configuration interface, determining module parameters;

[0139] Determine the function module to be tested according to the module parameters for the type of the function module to be tested;

[0140] A single sequence is generated according to at least one functional module to be tested and the second operating timing of each functional module to be tested.

[0141] Optionally, in some embodiments of the present application, performing a test operation on the terminal to be tested according to the test set to obtain at least one test result includes:

[0142] Execute test operations on the terminal to be tested according to the test set to obtain comprehensive test results;

[0143] And, if the test set further includes an associated test subset, generating an associated test result for the associated test subset, where the associated test subset includes at least two test subsets having an associated relationship;

[0144] The comprehensive test results and the correlation test results are respectively used as test results.

[0145] Optionally, in some embodiments of the present application, after performing a test operation on the terminal to be tested according to the test set and obtaining at least one test result, the method further includes:

[0146] generating a third running timing according to the first running timing, where the third running timing is different from the first running timing;

[0147] generating a reference set according to each test subset and the third running time sequence;

[0148] Determine a reference result corresponding to the reference set;

[0149] Determine the performance evaluation results of each functional module to be tested based on the test results and reference results.

[0150] In the embodiment of the present application, the acquisition module 201 acquires a test set corresponding to the terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence. The testing module 202 performs a test operation on the terminal to be tested according to the test set to obtain at least one test result.

[0151] The embodiments of the present application improve testing efficiency by testing multiple functional modules to be tested simultaneously. The unified joint testing method based on multiple test subsets and multiple functional modules to be tested helps simulate scenarios where multiple applications and multiple functions are used together, helps analyze comprehensive performance issues in these scenarios, and improves the effectiveness of the test and the reference value of the test results.

[0152] In addition, the present application also provides an electronic device, such as Figure 8 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0153] The electronic device may include one or more processors 301 of processing cores, one or more computer-readable storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0154] The processor 301 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 302 and accessing data stored in the memory 302, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 301.

[0155] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0156] The electronic device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 303 can also include one or more DC or AC power supplies, a recharging system, a power supply device debugging circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0157] The electronic device may further include an input unit 304, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0158] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby implementing the steps of any of the testing methods provided in the embodiments of the present application.

[0159] An embodiment of the present application obtains a test set corresponding to the terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each test subset includes at least one functional module to be tested in a second operating sequence. A test operation is performed on the terminal to be tested according to the test set to obtain at least one test result.

[0160] By simultaneously testing multiple functional modules under test, test efficiency is improved. The unified joint testing approach based on multiple test subsets and multiple functional modules under test helps simulate scenarios involving the combined use of multiple applications and multiple functions, facilitating analysis of comprehensive performance issues in these scenarios, and enhancing test effectiveness and the reference value of test results.

[0161] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0162] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0163] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any test method provided in the present application.

[0164] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0165] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0166] Since the instructions stored in the computer-readable storage medium can execute the steps in any test method provided in the present application, the beneficial effects that can be achieved by any test method provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0167] The above is a detailed introduction to a test method, device, electronic device and computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A testing method, characterized in that: include: Acquire a test set corresponding to the terminal to be tested, where the test set includes at least one test subset in a first operating sequence, and each of the test subsets includes at least one functional module to be tested in a second operating sequence; A test operation is performed on the terminal to be tested according to the test set to obtain at least one test result.

2. The testing method according to claim 1, wherein: The obtaining of the test set corresponding to the terminal to be tested includes: Obtain performance anomaly feedback information; Identifying at least one scenario to be tested from the performance abnormality feedback information through feature extraction and determining a first operating timing corresponding to each scenario to be tested; Determining at least one functional module to be tested included in each of the scenarios to be tested and determining a second operating sequence corresponding to each functional module to be tested, wherein each scenario to be tested corresponds to one test subset; The test set is generated according to each of the functional modules to be tested in each of the scenarios to be tested, as well as the first operating timing and the second operating timing.

3. The testing method according to claim 2, wherein: The determining of at least one functional module to be tested included in each of the scenarios to be tested comprises: Determining at least one type of function module to be tested included in each of the test scenarios and determining module parameters corresponding to each type of function module to be tested; For each type of the functional module to be tested, the functional module to be tested is determined according to the type of the functional module to be tested and the corresponding module parameters.

4. The testing method according to claim 1, wherein: The obtaining of the test set corresponding to the terminal to be tested includes: In response to a first interface screening operation on the single sequence list interface, determining a single sequence corresponding to at least one scenario to be tested and determining an initial running timing corresponding to each single sequence, wherein each single sequence corresponds to one of the test subsets; In response to the sorting operation on the single sequence, obtaining a first running timing after adjustment of the initial running timing; A test set is obtained according to each of the single sequences and the first running timing corresponding to each of the single sequences.

5. The testing method according to claim 4, characterized in that: Before determining, in response to the first interface screening operation, the single sequence corresponding to at least one scenario to be tested, and determining the initial running timing corresponding to each of the single sequences, the method further includes: In response to the second interface screening operation on the module type list interface, determining the type of the functional module to be tested, and displaying the module configuration interface corresponding to the type of the functional module to be tested; In response to an interface configuration operation on the module configuration interface, determining module parameters; Determining a function module to be tested for the type of function module to be tested according to the module parameters; A single sequence is generated according to at least one of the functional modules to be tested and the second operating timing of each of the functional modules to be tested.

6. The testing method according to claim 1, wherein: The performing a test operation on the terminal to be tested according to the test set to obtain at least one test result includes: Performing a test operation on the terminal to be tested according to the test set to obtain a comprehensive test result; And, if the test set further includes an associated test subset, generating an associated test result for the associated test subset, wherein the associated test subset includes at least two test subsets having an associated relationship; The comprehensive test result and the associated test result are respectively used as test results.

7. The testing method according to claim 1, wherein: After performing a test operation on the terminal to be tested according to the test set and obtaining at least one test result, the method further includes: generating a third operating timing according to the first operating timing, wherein the third operating timing is different from the first operating timing; generating a reference set according to each of the test subsets and the third running timing; Determining a reference result corresponding to the reference set; The performance evaluation results of the functional modules to be tested are determined according to the test results and the reference results.

8. A testing device, characterized in that: The device comprises: an acquisition module, configured to acquire a test set corresponding to the terminal to be tested, the test set including at least one test subset in a first operating sequence, and each of the test subsets including at least one functional module to be tested in a second operating sequence; The testing module is configured to perform a testing operation on the terminal to be tested according to the test set to obtain at least one test result.

9. An electronic device, characterized in that: The test method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the test method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the testing method according to any one of claims 1 to 7 are implemented.

Images