An application testing method, apparatus, electronic device, and storage medium

Abstract

This disclosure relates to the field of computer technology, and more particularly to an application testing method, apparatus, electronic device, and storage medium. The method involves acquiring test scenario data of an application under test, responding to requests for preset background traffic data, executing each test scenario data, and obtaining corresponding response information. Each response includes a scenario execution result and verification information, the verification information being used to verify the correctness of the execution result of the test scenario data. Based on the request time, scenario execution result, and verification information of the corresponding request, the test result of the application under test is determined. This improves the efficiency and accuracy of the testing.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an application testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] As software application architectures become increasingly complex, various failures may occur during application. When system high availability is insufficient or system stability is poor, these failures can negatively impact the user experience. Therefore, how to test an application's high availability capabilities in handling failures has become a pressing issue.

[0003] In related technologies, fault testing is typically performed manually, resulting in low efficiency. Furthermore, manual observation of test results introduces observational errors, thus reducing the accuracy of the tests. Summary of the Invention

[0004] This disclosure provides an application testing method, apparatus, electronic device, and storage medium to improve testing efficiency and accuracy.

[0005] The specific technical solutions provided in this disclosure are as follows:

[0006] Obtain test scenario data for the application under test. Each test scenario data is generated based on operation content data and operation sequence labels.

[0007] In response to a request for preset background traffic data, each test scenario data is executed to obtain corresponding response information. Each response information includes a scenario execution result and verification information, wherein the verification information is used to verify the correctness of the execution result of the test scenario data.

[0008] The test results of the application under test are determined based on the request time, scenario execution result, and verification information of the corresponding request information.

[0009] Optionally, obtain test scenario data, specifically including:

[0010] Listen for code modification events affecting the application under test;

[0011] In response to the detected code modification event, obtain test scenario data.

[0012] Optionally, obtain test scenario data, specifically including:

[0013] Acquire test scenario data according to the preset test cycle.

[0014] Optional, also includes:

[0015] Obtain operation sequence labels, where each operation sequence label indicates the order of operations among the operation content data contained in the test scenario data; and,

[0016] Obtain basic scenario tags, where each basic scenario tag indicates the execution order of the test scenarios of the basic scenario;

[0017] Test scenario data is generated based on the corresponding operation sequence tags, the basic scenario tags, and the corresponding operation content data.

[0018] Optionally, each of the test scenario data is executed separately to obtain the corresponding response information, specifically including:

[0019] Identify the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data;

[0020] Each second test scenario data is executed separately to obtain the corresponding response information. Each second test scenario data is test scenario data that does not correspond to the basic scenario label.

[0021] The third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution is determined, and the third test scenario data is executed.

[0022] Optionally, each second test scenario data is executed separately to obtain the corresponding response information, specifically including:

[0023] For each of the second test scenario data, perform the following operations:

[0024] According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

[0025] Optionally, obtain the corresponding response information, specifically including:

[0026] For the test scenario data, perform the following operations respectively:

[0027] Since the application under test does not depend on other services, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution results;

[0028] In response to the application under test depending on other services, the verification information corresponding to the test scenario data is added to the request information of the dependent service, and the response information returned in response to the request information is obtained.

[0029] Optionally, the verification information corresponding to the test scenario data can be added to the response information containing the scenario execution results, specifically including:

[0030] For the response information, perform the following operations respectively:

[0031] In response to the obtained response object, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution result;

[0032] In response to the failure to obtain the response object, the verification information is added to the information database. In response to the further acquisition of the response object, the verification information is read from the information database and added to the response information.

[0033] Optionally, the verification information corresponding to the test scenario data is added to the request information of the service that depends on it, and the response information returned in response to the request information is obtained, specifically including:

[0034] For the requested information, perform the following operations respectively:

[0035] In response to obtaining the request object corresponding to the request information of the requested dependent service, the verification information is added to the request information, and the response information returned for the request information is obtained;

[0036] In response to the failure to obtain the requested object, the verification information is added to the information database. In response to the further acquisition of the requested object, the verification information is read from the information database and added to the request information to obtain the response information returned for the request information.

[0037] An application testing apparatus, comprising:

[0038] The acquisition module is used to acquire test scenario data, where each test scenario data is generated based on operation content data and operation sequence tags;

[0039] The processing module is used to respond to the request information for the preset background traffic data, execute each of the test scenario data respectively, and obtain the corresponding response information. Each response information includes the scenario execution result and verification information, and the verification information is used to verify the correctness of the execution result of the test scenario data.

[0040] The verification module is used to determine the test result of the application under test based on the request time, scenario execution result and verification information of the corresponding request information.

[0041] Optionally, the acquisition module is specifically used for:

[0042] Listen for code modification events affecting the application under test;

[0043] In response to the detected code modification event, obtain test scenario data.

[0044] Optionally, the acquisition module is specifically used for:

[0045] Acquire test scenario data according to the preset test cycle.

[0046] Optionally, it also includes a generation module, which is specifically used for:

[0047] Obtain operation sequence labels, where each operation sequence label indicates the order of operations among the operation content data contained in the test scenario data; and,

[0048] Obtain basic scenario tags, where each basic scenario tag indicates the execution order of the test scenarios of the basic scenario;

[0049] Test scenario data is generated based on the corresponding operation sequence tags, the basic scenario tags, and the corresponding operation content data.

[0050] Optionally, when executing each test scenario data to obtain the corresponding response information, the processing module is specifically used for:

[0051] Identify the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data;

[0052] Each second test scenario data is executed separately to obtain the corresponding response information. Each second test scenario data is test scenario data that does not correspond to the basic scenario label.

[0053] The third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution is determined, and the third test scenario data is executed.

[0054] Optionally, when executing each second test scenario data and obtaining the corresponding response information, the processing module is specifically used for:

[0055] For each of the second test scenario data, perform the following operations:

[0056] According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

[0057] Optionally, when obtaining the corresponding response information, the processing module is specifically used for:

[0058] For the test scenario data, perform the following operations respectively:

[0059] Since the application under test does not depend on other services, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution results;

[0060] In response to the application under test depending on other services, the verification information corresponding to the test scenario data is added to the request information of the dependent service, and the response information returned in response to the request information is obtained.

[0061] Optionally, when adding the verification information corresponding to the test scenario data to the response information containing the scenario execution result, the processing module is specifically used for:

[0062] For the response information, perform the following operations respectively:

[0063] In response to the obtained response object, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution result;

[0064] In response to the failure to obtain the response object, the verification information is added to the information database. In response to the further acquisition of the response object, the verification information is read from the information database and added to the response information.

[0065] Optionally, when adding the verification information corresponding to the test scenario data to the request information of the service that depends on it, and obtaining the response information returned for the request information, the processing module is specifically used for:

[0066] For the requested information, perform the following operations respectively:

[0067] In response to obtaining the request object corresponding to the request information of the requested dependent service, the verification information is added to the request information, and the response information returned for the request information is obtained;

[0068] In response to the failure to obtain the requested object, the verification information is added to the information database. In response to the further acquisition of the requested object, the verification information is read from the information database and added to the request information to obtain the response information returned for the request information.

[0069] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the application testing method described above.

[0070] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described application testing method.

[0071] The beneficial effects of the embodiments disclosed herein are as follows:

[0072] The test scenario data of the application under test is acquired. Each test scenario data is generated based on operation content data and operation sequence tags. In response to a request for preset background traffic data, each test scenario data is executed, and corresponding response information is obtained. Each response information includes the scenario execution result and verification information. The verification information is used to verify the correctness of the execution result of the test scenario data. Based on the request time, scenario execution result, and verification information of the corresponding request information, the test result of the application under test is determined. In this way, in response to a request for preset background traffic data, each test scenario data can be executed automatically, thereby realizing automated testing of the application and improving the efficiency of application testing. Furthermore, in this embodiment of the disclosure, since each response information includes the scenario execution result and verification information, the scenario execution result can be verified based on the verification information in the response information, thereby avoiding inaccurate human evaluation or verification errors and improving the accuracy of verification. Attached Figure Description

[0073] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0074] Figure 1 This is a schematic diagram of the process for generating test scenario data in an embodiment of this disclosure;

[0075] Figure 2 This is a schematic diagram illustrating test scenario data examples in the embodiments of this disclosure;

[0076] Figure 3 This is a flowchart illustrating the method for creating test scenario data in an embodiment of this disclosure;

[0077] Figure 4 This is a flowchart of an application testing method in an embodiment of this disclosure;

[0078] Figure 5 This is a flowchart illustrating a first method for obtaining test scenario data in this embodiment of the present disclosure;

[0079] Figure 6 This is a flowchart illustrating the method for obtaining response information in an embodiment of this disclosure;

[0080] Figure 7 This is a flowchart illustrating a method for obtaining response information according to an embodiment of this disclosure;

[0081] Figure 8 This is a flowchart illustrating a method for adding verification information according to an embodiment of this disclosure;

[0082] Figure 9 This is a flowchart of another method for adding verification information in an embodiment of this disclosure;

[0083] Figure 10 This is a schematic diagram of email notification in an embodiment of this disclosure;

[0084] Figure 11 This is a flowchart illustrating an automated verification method according to an embodiment of the present disclosure;

[0085] Figure 12 This is a flowchart illustrating a method for fault scenario testing according to an embodiment of this disclosure;

[0086] Figure 13 This is a schematic diagram illustrating an execution example in an embodiment of this disclosure;

[0087] Figure 14 This is a schematic diagram of the structure of the application testing device in the embodiments of this disclosure;

[0088] Figure 15 This is a schematic diagram of the structure of an electronic device in an embodiment of this disclosure. Detailed Implementation

[0089] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0090] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0091] The embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0092] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0093] For ease of understanding, the technical terms used in the embodiments of this disclosure are explained as follows:

[0094] Fault drills are used to test whether software applications have high availability under certain fault scenarios. Fault scenarios include RPC service failures, RPC service timeouts, middleware failures, middleware timeouts, and infrastructure failures.

[0095] It should be noted that, in this embodiment of the disclosure, fault drills are also referred to as application testing.

[0096] Background traffic data: During the fault simulation, test traffic is applied to the application to simulate its actual operating state.

[0097] Jenkins: An open-source continuous integration tool that can execute specified tasks according to settings. Jenkins Pipeline is a workflow framework that runs on Jenkins, connecting multiple tasks that originally ran independently to orchestrate and visualize complex processes that are difficult to complete with a single task.

[0098] TestNG: An open-source automated testing framework with rich test case orchestration annotations and a comprehensive test data management module.

[0099] Cucumber: An open-source automated testing framework that allows you to describe test scenarios using natural language.

[0100] JavaAgent: A Java code injection method that modifies bytecode without affecting the normal compilation of a Java project. It can intercept and modify bytecode before loading class files, or it can change the bytecode of loaded classes at runtime.

[0101] Test scenario data: also known as fault scenarios, test scenarios are used to verify the high availability of applications. Test scenario data is generated based on operation content data and operation sequence labels.

[0102] Manipulating content data: This represents the specific operations that implement the test steps, such as fault injection or result verification.

[0103] Operation sequence label: Used to indicate the order of operations among the various operation content data contained in the test scenario data. In this embodiment of the disclosure, the operation sequence label can be "if", "when", "and", etc., and this embodiment of the disclosure does not impose any limitations on it.

[0104] Basic scenario label: Used to indicate the execution order of test scenarios based on scenario type. In this embodiment of the disclosure, the basic scenario label can be before or after scenario execution, and this embodiment of the disclosure does not impose any restrictions on this.

[0105] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments.

[0106] The inventors have discovered that with the development of computer technology and the increase in user volume, the architectural complexity of software applications is increasing. Whenever a popular activity is launched in an application, various types of failures are likely to occur. If the application's high availability capability is insufficient, its system stability is poor, or its incident handling plan is lacking, then when a failure occurs, it will panic amidst various system alarms, delaying repair time and causing incalculable economic losses. Furthermore, it will also reduce the user's experience with the software application. Therefore, how to test the high availability capability of an application in handling failures has become an urgent problem to be solved.

[0107] In related technologies, application testing is performed manually. Therefore, this testing method is inefficient. When urgent needs or events arise and time is tight, fault testing needs to be performed according to priority, which may not be able to fully test all fault scenarios.

[0108] Furthermore, since many high-availability capabilities require determining whether to activate protection strategies based on the number or proportion of anomalies, the current approach using related technologies cannot accurately determine whether the number or proportion of anomalies counted when a protection strategy is triggered strictly meets expectations based solely on monitoring data. Manually executing testing procedures introduces observation errors when observing test results, thus reducing the accuracy of the tests. Additionally, to avoid system instability, human error, and monitoring bias, multiple rehearsals and observations are required to draw conclusions.

[0109] To address the aforementioned issues, this disclosure proposes an application testing method, apparatus, electronic device, and storage medium. The method acquires test scenario data for the application under test. Each test scenario data is generated based on operation content data and operation sequence tags. Responding to a request for preset background traffic data, each test scenario data is executed, yielding corresponding response information. Each response includes a scenario execution result and verification information. The verification information verifies the correctness of the execution result of the test scenario data. Based on the request time, scenario execution result, and verification information of the corresponding request, the test result of the application under test is determined. Thus, by acquiring the test scenario data of the application under test and responding to a request for preset background traffic data, automated testing of the application under test can be achieved, thereby improving testing efficiency and reducing labor costs. Furthermore, in this embodiment, the verification information is displayed in the response information, allowing the verification information in the response to verify the scenario execution result, improving the accuracy of the verification.

[0110] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0111] Based on the above embodiments, the process of generating test scenario data in the embodiments of this disclosure will first be described in detail, see reference. Figure 1 The diagram shown illustrates the process of generating test scenario data in this embodiment of the disclosure, specifically including:

[0112] S10: Obtain the operation sequence label.

[0113] Each operation sequence label indicates the order of operations among the various operation content data contained in the test scenario data.

[0114] In this embodiment of the disclosure, a new test project is created, and an operation sequence label file is created in the resource directory of the test project. An operation sequence label is defined in the operation sequence label file to indicate the operation order between different operation content data, thereby obtaining at least one operation sequence label.

[0115] The test project is a Java project, and the type of test project is not limited in this embodiment.

[0116] The resource directory can be, for example, the src directory in a Java project, but this embodiment does not impose any restrictions on it.

[0117] The operation sequence label file can be, for example, a step package, and this disclosure does not limit this.

[0118] The following is a specific example to illustrate S10 in this embodiment. For instance, in a specific implementation, a `step` package is created in the `src` directory of a pre-established Java project. Then, a Java class is created in the `step` package to define the operation order between operation content data. Using the Cucumber testing framework, operation order tags for the operation content data are defined, and a connection is established between the operation order tags and the operation content data. Thus, during application testing, the corresponding operation content data can be executed based on the operation order tags.

[0119] The newly created Java class used to define the operation order between operation content data can be, for example, FaultInjectSetp.java, but this embodiment does not impose any restrictions on it.

[0120] The operation order label can be a related annotation defined using the Cucumber testing framework. The value in the annotation is a regular expression used to define the label name of the operation order label. The operation order label can be, for example, "@Given", "@Then", "@When", "@But", "@And", etc. This disclosure does not impose any restrictions on this.

[0121] The operation content data is a Java Method. The Java Method is used to implement specific operations for application testing, such as background traffic startup, fault injection, fault cancellation, result verification, etc. This disclosure does not limit this.

[0122] S11: Obtain basic scene tags.

[0123] Each basic scenario label indicates the execution order of the test scenarios for the basic scenario.

[0124] In this embodiment of the disclosure, a basic scene tag file is created in the resource directory of the test project, and a new execution order tag for indicating the test scene of type basic scene is created in the basic scene tag file, thereby obtaining at least one basic scene tag.

[0125] The basic scenario tag file can be a testCase package, and this disclosure does not impose any restrictions on it.

[0126] The following is a specific example to illustrate S11 in this embodiment. For example, in a specific implementation, a testCase package is created in the src directory of a pre-established Java project. Then, a base Java class is created in the testCase package to indicate the execution order of test scenarios of type base scenario. This base Java class can be used to improve the test case orchestration capability of the Cucumber framework, enabling it to perform certain operations before and after the execution of some test scenarios.

[0127] First, using the TestNG `@BeforeSuite` annotation, scenarios with the `@before` and `@after` Cucumber tags are retrieved from the Cucumber Feature scenario set before the test scenario execution. The `@before` and `@after` tags define the pre- and post-execution operations for all fault scenarios in the Feature file, such as installing fault injection tools before execution and uninstalling fault tools after execution. Then, combining the TestNG `@BeforeClass` and `@AfterClass` annotations, the retrieved scenarios with the `@before` and `@after` tags are executed before and after the scenario set execution.

[0128] The underlying Java class can be, for example, BaseTest.java, but this embodiment does not impose any restrictions on it.

[0129] The operation order label can be divided into at least the following two label types: the first label type is priority execution, which can be represented as @before in specific implementation; the second label type is non-priority execution, which can be represented as @after in specific implementation. This disclosure does not impose any restrictions on this.

[0130] Additionally, it should be noted that S10 can be executed first, followed by S11, or S11 can be executed first, followed by S10. Of course, S10 and S11 can also be executed simultaneously. In this embodiment, the execution order of S10 and S11 is not restricted.

[0131] S12: Generate test scenario data based on the corresponding operation sequence labels, basic scenario labels, and corresponding operation content data.

[0132] In this embodiment of the disclosure, test scenario data is generated based on the corresponding operation sequence tags, basic scenario tags, and corresponding operation content tags.

[0133] In practice, test scenarios are designed in the test scenario data file. The keyword function is used to define the test scenario data set. A scenario defines a test scenario, and the execution steps of the operation content data in each test scenario data are composed of operation sequence tags such as Given, When, Then, But, and And. The execution steps must uniquely match the regular expression of the defined operation sequence tags.

[0134] Furthermore, the background is used for environment initialization work before the execution of data in each test scenario, such as attaching the fault plugin to the JVM of the truncated system.

[0135] For example, see Figure 2 The diagram shown is a sample of test scenario data in an embodiment of this disclosure. The test scenario data includes basic scenario tags "@before" and "@after". "@before" is used for the installation of fault tools before the scenario set is executed, and "@after" is used for the uninstallation of fault tools after execution.

[0136] The operation sequence labels are "if", "when", "and", and "then", as well as the operation content data, such as "Set background traffic concurrency: 3".

[0137] In this embodiment of the disclosure, test scenario data is generated based on operation content data, operation sequence tags, and basic scenario tags. Thus, during the testing process, the test scenario data can be directly called and executed, thereby automating application testing.

[0138] Based on the above embodiments, the process of creating test scenario data in this disclosure embodiment will be described in detail below using a specific example. (See attached document for details.) Figure 3 The diagram shown is a flowchart illustrating the method for creating test scenario data in this embodiment of the present disclosure, specifically including:

[0139] S301: Create a Java project using the project management tool Maven.

[0140] In this embodiment of the disclosure, a new Java project is created based on the project management tool Maven, and the test framework TestNG and the test framework Cucumber are added to the pom.xml configuration information of the Java project.

[0141] It should be noted that there are dependencies between the test frameworks in this embodiment.

[0142] S302: In a Java project, create a test scenario file.

[0143] In this embodiment of the disclosure, a feature directory is created under the resource directory of the Java project created through S201, and a test scenario data file is created under the feature directory.

[0144] In this embodiment, at least one test scenario data in each test scenario data file constitutes a scenario set, that is, the test scenario data file contains at least one test scenario data; the test scenario data file can be, for example, data on switching test scenarios in a computer room, and this embodiment does not limit this.

[0145] It should be noted that in this embodiment of the disclosure, the test scenario data in the test scenario data file can be classified based on the scenario type, or the test scenario data files in the test scenario data file can be freely combined and arranged according to actual test requirements. This embodiment of the disclosure does not impose any restrictions on this.

[0146] S303: Create a "step" package in the src directory of the Java project's resources, and create a new Java class for operation order in the "step" package, defining the operation order tag under the operation order Java class.

[0147] In this embodiment of the disclosure, a step package is created under the src directory of the Java project, and a new Java class for operation order is created in the step package. The operation order tags @Given, @Then, @When, @But, and @And are defined using the Cucumber testing framework.

[0148] S304: Create a testCase package in the src directory of the Java project, and create a base Java class in the testCase package, defining the base scenario tags under the base Java class.

[0149] In this embodiment of the disclosure, a testCase package is created under the resource src directory of the Java project, and a basic Java class is created in the testCase package. The test case orchestration capability of the Cucumber test framework is improved in the basic Java class, thereby enabling the initialization environment configuration and the installation of fault injection tools before each fault scenario is executed and the uninstallation of fault tools after execution.

[0150] S305: Generate test scenario data based on operation content data, operation sequence labels, and basic scenario labels.

[0151] In this embodiment of the disclosure, operation content data, operation sequence labels, and basic scene labels are written in the test scene data file to generate test scene data.

[0152] It should be noted that, in this embodiment of the disclosure, each operation content data needs to uniquely match the regular expression in the operation sequence tag or the basic scene tag.

[0153] S306: Create a Java class in the testCase package of the Java project that executes the test scenario data file and inherits from the base Java class.

[0154] In this embodiment of the disclosure, a Java class for executing test scenario data files is created in the testCase package of the Java project, and inherits from the base Java class, such as the SwitchRoomTestRunner.java class. This Java class for running test scenario data files will run all test scenario data in the specified test scenario data file.

[0155] S307: Define the configuration file, specifying the Java class of the test scenario data file to be run.

[0156] In this embodiment of the disclosure, an xml directory is created under the resource directory of the Java project, and a testNG xml configuration file is created under the xml directory to specify the Java class of the test scenario data file to be run.

[0157] In this embodiment of the disclosure, operation content data, operation sequence tags, and basic scenario tags are used to generate test scenario data, which enables automatic fault injection and destruction during application testing.

[0158] Based on the above embodiments, see Figure 4The diagram shown is a flowchart of an application testing method according to an embodiment of this disclosure, specifically including:

[0159] S40: Obtain test scenario data for the application to be tested.

[0160] The data for each test scenario is generated based on the operation content data and operation sequence labels.

[0161] In this embodiment of the disclosure, test scenario data of the application to be tested is obtained.

[0162] It should be noted that the test scenario data in this embodiment is generated based on operation content data and operation sequence labels.

[0163] Optionally, in this embodiment of the disclosure, three possible implementation methods are provided for obtaining test scenario data of the application to be tested. The following is a detailed description of the methods for obtaining test scenario data of the application to be tested in this embodiment of the disclosure.

[0164] The first method: event triggering.

[0165] In this embodiment of the disclosure, test scenario data can be obtained by listening to code modification events, see below. Figure 5 The flowchart shown is a first method for obtaining test scenario data in this embodiment of the present disclosure, which specifically includes:

[0166] S401: Listen for code modification events related to the application under test.

[0167] In this embodiment of the disclosure, the system monitors whether any code modification events occur for the application under test.

[0168] Optionally, in this embodiment of the disclosure, a callback link can be configured for the application under test. Once the callback link is configured, code modification events for the application under test can be listened for.

[0169] It should be noted that in this embodiment of the disclosure, code modification events of the application under test can be continuously monitored, or code modification events of the application under test can be monitored according to a preset period. This embodiment of the disclosure does not impose any restrictions on this.

[0170] The code modification event could be, for example, a developer submitting and pushing code to GitLab, and this embodiment of the disclosure does not impose any restrictions on this.

[0171] S402: Responds to detected code modification events and retrieves test scenario data.

[0172] In this embodiment of the disclosure, after a code modification event is detected, test scenario data is obtained in response to the detected code modification event.

[0173] Optionally, in this embodiment of the disclosure, when acquiring test scenario data, the callback link of the application to be tested can be automatically called back in response to the listened code modification event, and the test scenario data associated with the callback link can be obtained based on the obtained callback link.

[0174] The automatic triggering method in this embodiment ensures that the application testing process is started immediately when the project code is modified, and the impact of code changes on the application's high availability capability can be detected in a timely manner.

[0175] The second method: fixed-period triggering.

[0176] In this embodiment of the disclosure, test scenario data can be automatically acquired at fixed intervals.

[0177] Specifically, this includes: acquiring test scenario data according to a preset test cycle.

[0178] In this embodiment of the disclosure, a fixed test cycle is preset, and test scenario data is acquired according to the fixed test cycle, thereby automatically executing the application test process.

[0179] When a fixed test cycle is pre-set, a cron expression can be configured to trigger the test process at regular intervals.

[0180] It should be noted that the method of configuring cron expressions to obtain test scenarios is periodic.

[0181] For example, the configured cron expression is 01***, which means that test scenario data will be automatically retrieved at 1:00 AM every day.

[0182] The third method: manual triggering.

[0183] In this embodiment of the disclosure, after logging into the Jenkins console, the user can manually trigger the acquisition of test scenario data.

[0184] In addition, it should be noted that the embodiments disclosed herein are not limited to the three methods of obtaining test scenario data mentioned above.

[0185] S41: In response to the request information for the preset background traffic data, execute the data for each test scenario and obtain the corresponding response information.

[0186] Each response message includes a scenario execution result and verification information, wherein the verification information is used to verify the correctness of the execution result of the test scenario data.

[0187] In this embodiment of the disclosure, in response to a request for preset background traffic data, each test scenario data is executed to obtain response information generated based on the corresponding test scenario data.

[0188] Optionally, in this embodiment of the disclosure, a possible implementation method is provided to obtain the corresponding response information, see below. Figure 6 The diagram shown is a flowchart illustrating a method for obtaining response information in an embodiment of this disclosure, specifically including:

[0189] S411: Determine the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data.

[0190] In this embodiment, the basic scene tags can be divided into two types: the first type is priority execution, and the second type is non-priority execution. When the tag type of the basic scene tags is priority execution, the first test scene data needs to be executed before the second test scene data. When the tag type of the basic scene tags is non-priority execution, the third test scene data needs to be executed only after all the second test scene data has been executed.

[0191] Therefore, in this embodiment of the disclosure, firstly, based on the tag type corresponding to each of the at least one basic scene tag, the tag type corresponding to the basic scene tag that is executed first is determined from the at least one basic scene tag, and the corresponding first test scene data is determined. Finally, the determined first test scene data is executed.

[0192] In this case, @before represents a base scenario label that is executed first, while @after represents a base scenario label that is not executed first. In this case, the first test scenario data corresponding to @before will be executed first.

[0193] S412: Execute each second test scenario data separately and obtain the corresponding response information.

[0194] Each second test scenario data is test scenario data that does not correspond to the basic scenario label.

[0195] In this embodiment of the disclosure, when the test scenario data does not correspond to the basic scenario label, the test scenario data is determined to be the second test scenario data that can determine the corresponding response information based on the request information. Therefore, each second test scenario data is executed sequentially according to the order of each test scenario data in the test scenario data file, thereby obtaining the corresponding response information.

[0196] For example, when the test scenario data is "a network latency fault is injected into a certain data center service to verify the high availability capability of the data center switchover", it is determined that the test scenario data does not correspond to the basic scenario label and is the second test scenario data. Then, the second test scenario data is executed to obtain the corresponding response information.

[0197] Optionally, in this embodiment of the disclosure, a possible implementation method is provided for executing each second test scenario data separately. The process of obtaining response information is described below using any second test scenario data as an example:

[0198] According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

[0199] In this embodiment of the disclosure, since the operation sequence label is used to indicate the operation order between operation content data, the operation content data corresponding to each operation sequence label is executed according to the operation sequence label corresponding to the second test scenario data, thereby obtaining the corresponding response information.

[0200] For example, if the operation order labels are "if" and "when", the operation content data corresponding to the operation order label "if" will be executed first, and then the operation content data corresponding to the operation order label "when" will be executed.

[0201] Optionally, in this embodiment of the disclosure, a possible implementation method is provided to obtain the corresponding response information. The process of obtaining response information in this embodiment of the disclosure is described in detail below. Figure 7 The diagram shown is a flowchart illustrating a method for obtaining response information according to an embodiment of this disclosure, specifically including:

[0202] S4121: In response that the application under test does not depend on other services, add the verification information corresponding to the test scenario data to the response information containing the scenario execution results.

[0203] In this embodiment of the disclosure, the test scenarios can be divided into the following two types: the first type is without dependence on other services, that is, the application under test can run normally without depending on other services during the running process; the second type is dependent on other services, that is, the application under test needs to depend on other services in order to achieve normal operation during the running process.

[0204] When the type of the application under test is one that does not depend on other services, and the test results of the application under test can be verified by the execution results of the scenario of the application under test, in response to the fact that the application under test does not depend on other services, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution results.

[0205] The scenario execution result of the application under test is the execution result obtained after running the application under test in the test scenario.

[0206] In this way, after obtaining the response information containing the scenario execution results and verification information, the scenario execution results can be verified based on the verification information in the response information, thereby obtaining the corresponding test results.

[0207] For example, when the test scenario is a data center switchover, if the test scenario is executed and the obtained scenario execution result is a network latency of 10ms, and the response information contains the verification information that the network latency is 10ms, then the corresponding test result is determined to be that there is no fault.

[0208] Optionally, in this embodiment of the disclosure, a possible implementation method for adding verification information is provided, see [link to relevant documentation]. Figure 8 The diagram shown is a flowchart illustrating a method for adding verification information according to an embodiment of this disclosure, specifically including:

[0209] S4121-1: In response to obtaining the response object corresponding to the response information, add the verification information corresponding to the test scenario data to the response information containing the scenario execution result.

[0210] In this embodiment of the disclosure, in response to obtaining the response object corresponding to the response information, that is, when the application displays the code location of key information during the test and the response object corresponding to the response information can be obtained, the verification information corresponding to the test scenario data is directly added to the corresponding response information.

[0211] In practice, the code corresponding to the response information can be modified directly.

[0212] S4121-2: In response to not obtaining a response object, add the verification information to the information database; in response to obtaining the response object, read the verification information from the information database and add it to the response information.

[0213] In this embodiment of the disclosure, in response to the failure to obtain a response object, that is, the failure to obtain a response object, the verification information can be added to the information database first, and the system can continuously monitor whether a response object is obtained. When it is determined that a response object has been obtained, in response to the further obtained response object, the verification information is read from the information database and added to the response information.

[0214] The information database can be a Memcache or Redis caching service, and this embodiment does not impose any restrictions on it.

[0215] S4122: In response to the application under test depending on other services, add the verification information corresponding to the test scenario data to the request information of the dependent service, and obtain the response information returned in response to the request information.

[0216] In this embodiment of the disclosure, when the application under test depends on other services, that is, the normal operation of the application under test requires other services, the response information corresponding to the test scenario data is added to the request information of the dependent service, so that after the test scenario data is executed, the response information returned in response to the request information is obtained.

[0217] Optionally, in this embodiment of the disclosure, a possible implementation method for obtaining response information is provided, see [link to relevant documentation]. Figure 9 The diagram shows a flowchart of another method for adding verification information in an embodiment of this disclosure, which specifically includes:

[0218] S4122-1: In response to obtaining the request object corresponding to the request information of the requested dependent service, add the verification information to the request information and obtain the response information returned for the request information.

[0219] In this embodiment of the disclosure, when the application under test exhibits code locations that allow the addition of verification information during the testing process, and can obtain the request object corresponding to the request information of the service it depends on, in response to obtaining the request object corresponding to the request information of the service it depends on, the application directly modifies the request information, adds the verification information to the request information, and thus obtains the response information returned for the request information.

[0220] In practice, the code corresponding to the request information can be directly modified.

[0221] S4122-2: In response to not obtaining the requested object, add the verification information to the information database; in response to further obtaining the requested object, read the verification information from the information database and add it to the request information, and obtain the response information returned for the request information.

[0222] In this embodiment of the disclosure, when the application exhibits code location of critical information during testing and cannot obtain the request object corresponding to the request information of the service it depends on, in response to the failure to obtain the request object, in the above situation, the verification information can be added to the information database first, and the system can continuously monitor whether the request object is obtained. When it is determined that the request object has been obtained, in response to the further obtained request object, the verification information is read from the information database and added to the request information to obtain the response information returned for the request information.

[0223] The information database can be a Memcache or Redis caching service, and this embodiment does not impose any restrictions on it.

[0224] S413: Determine the third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution, and execute the third test scenario data.

[0225] In this embodiment of the disclosure, firstly, based on the tag type corresponding to each of the at least one basic scene tag, the basic scene tag whose tag type is not prioritized for execution is determined from the at least one basic scene tag, and the corresponding third test scene data is determined. Finally, the determined third test scene data is executed.

[0226] In this case, @before indicates that the tag type is the basic scenario tag that is executed first, and @after indicates that the tag type is the basic scenario tag that is not executed first. Then, the third test scenario data corresponding to @after will be executed.

[0227] S42: Determine the test results of the application to be tested based on the request time, scenario execution result, and verification information of the corresponding request information.

[0228] In this embodiment of the disclosure, the test result of the application to be tested is determined based on the request time, scenario execution result and verification information of the corresponding request information.

[0229] In this way, by verifying the execution results of the scenario through the verification information in the response information, the problem of insufficient verification granularity caused by monitoring data collection errors and human observation errors can be avoided, thereby improving the accuracy of result verification.

[0230] Optionally, in this embodiment of the disclosure, TestNG automatically generates an emailable-report.html test report based on the test results in the / target / surefire-reports directory, and sends the test report email notification in conjunction with the Jenkins Email plugin. (See reference...) Figure 10 The diagram shown is a schematic of email notification in an embodiment of this disclosure. The test report shows that in the test scenario of data center switching, there were 12 successful attempts, with a total time of 1,011,496 ms. In the test scenario of node removal, there were 5 successful attempts, with a total time of 859,345 ms.

[0231] In this embodiment, application testing can be automatically triggered by timed events or code changes, thereby automating application testing and improving verification efficiency. Furthermore, returning verification information regarding system performance during and after failures within the background traffic data response information improves verification accuracy.

[0232] Based on the above embodiments, see Figure 11 The diagram shown is a flowchart of an automated verification method according to an embodiment of this disclosure, specifically including:

[0233] 1. Testing Project.

[0234] The test project uses Java multithreading to concurrently request background traffic data from the application under test. At the same time, it records the request time of each request and the corresponding response information.

[0235] 2. Application to be tested.

[0236] The application under test is based on JavaAgent technology. Before loading the class file generated after compiling the test scenario data, the bytecode is intercepted and modified, and the verification information required for application testing is written into the response information and returned to the test project.

[0237] Based on different testing scenarios, the embodiments of this disclosure can be divided into the following four code instrumentation methods.

[0238] The first method:

[0239] When the application under test displays the code location of the verification information during the test, and the response object of the response information can be obtained, the JavaAgent first obtains the verification information, and then the JavaAgent directly adds the verification information to the response information.

[0240] The second method:

[0241] When the application under test exhibits a code location for verification information during testing, and the response object for the response information cannot be obtained, the JavaAgent first obtains the verification information and writes it into the information database. When it is determined that the response object corresponding to the response information can be obtained, the JavaAgent retrieves the verification information from the information database and adds the verification information to the response information.

[0242] The information database can be, for example, a Memcache or Redis caching service.

[0243] The third method:

[0244] When the code location of the application under test that displays verification information during the test is exactly where the request object corresponding to the request information of the service it depends on can be obtained, the JavaAgent first obtains the verification information and directly adds the verification information to the request information.

[0245] The fourth method:

[0246] When the application under test exhibits a code location for verification information during testing, and the request object corresponding to the request information of the dependent service cannot be obtained, the JavaAgent first obtains the verification information and writes it into the information database. When it is determined that the request object corresponding to the request information can be obtained, the JavaAgent retrieves the verification information from the information database and adds the verification information to the request information.

[0247] The information database can be, for example, a Memcache or Redis caching service.

[0248] In this embodiment of the disclosure, based on JavaAgent technology, code instrumentation is performed on the application to be tested, and verification information is added to the response information containing the scenario execution results. Thus, the response information of the background traffic data contains verification information on the performance of the application to be tested during the fault and during the fault recovery, which can improve the accuracy of judging whether the high availability capability of the system meets expectations.

[0249] Based on the above embodiments, the following section uses test scenario data as an example of a fault scenario to elaborate on the application testing method in the embodiments of this disclosure. (See attached document for details.) Figure 12 The diagram shown is a flowchart illustrating a method for fault scenario testing according to an embodiment of this disclosure, specifically including:

[0250] 1. Fault test triggered.

[0251] In this embodiment of the disclosure, fault test triggering can be implemented in the following three ways:

[0252] The first method involves using a webhook to listen for code modification events, thereby automatically acquiring fault scenario data and triggering fault drills.

[0253] The second method is to use cron expressions to automatically acquire fault scenario data according to a preset cycle and trigger fault drills.

[0254] The third method: manually acquire fault scenario data and trigger fault drills.

[0255] 2. Test environment deployment task.

[0256] In this embodiment of the disclosure, the test environment deployment task is implemented remotely and automatically by writing scripts to call the deployment platform OpenAPI or by using Ansible tools.

[0257] 3. Fault scenario data execution tasks.

[0258] In this embodiment, fault scenario data is built based on the TestNG and Cucumber testing frameworks. The project code is pulled in the Jenkins Job that executes the fault scenario, and the command `mvn clean test -Dmaven.test.failure.ignore=true` is executed to carry out automated testing of the fault scenario data and obtain the corresponding response information.

[0259] The fault scenario data execution tasks include background traffic initiation, fault injection and destruction, and test result verification.

[0260] 4. Test result notification task.

[0261] In this embodiment of the disclosure, the test result notification task involves sending an email notification based on the HTML test report output by the TestNG framework and the Jenkins Email plugin after the test is completed. (See [link to documentation]). Figure 13 The diagram shown is a schematic representation of an execution example in an embodiment of this disclosure.

[0262] It should be noted that in this embodiment of the disclosure, the continuous integration tool Jenkins is used to run the entire fault testing process, and the execution steps are run serially using Jenkins Pipeline, including test environment deployment tasks, fault scenario data execution tasks, and test result notification tasks.

[0263] In this embodiment, by automating the triggering mechanism, test environment deployment, fault scenario data execution, and test result notification, the frequency and efficiency of fault testing can be improved. This is more than 10 times higher than customized high-availability assurance activities, increasing the timeliness of testing and reducing manual input costs to virtually zero. Furthermore, the method in this embodiment enables automated and routine high-availability capability assessment of applications, allowing for one or more daily fault regressions. Regressions can also be automatically triggered immediately when code is modified, thus avoiding online stability issues caused by inadequate human assessment.

[0264] Based on the same inventive concept, this disclosure also provides an application testing device, which can be a hardware structure, a software module, or a hardware structure plus a software module. This application testing device embodiment can inherit the content described in the foregoing method embodiments. Based on the above embodiments, see [reference needed]. Figure 14 The diagram shown is a structural schematic of the application testing device in an embodiment of this disclosure, specifically including:

[0265] The acquisition module 1400 is used to acquire test scenario data, wherein each test scenario data is generated based on operation content data and operation sequence labels;

[0266] The processing module 1401 is used to respond to the request information for the preset background traffic data, execute each of the test scenario data respectively, and obtain the corresponding response information. Each response information includes the scenario execution result and the verification information, and the verification information is used to verify the correctness of the execution result of the test scenario data.

[0267] The verification module 1402 is used to determine the test result of the application to be tested based on the request time, scenario execution result and verification information of the corresponding request information.

[0268] Optionally, the acquisition module 1400 is specifically used for:

[0269] Listen for code modification events affecting the application under test;

[0270] In response to the detected code modification event, obtain test scenario data.

[0271] Optionally, the acquisition module 1400 is specifically used for:

[0272] Acquire test scenario data according to the preset test cycle.

[0273] Optionally, it also includes a generation module 1403, which is specifically used for:

[0274] Obtain operation sequence labels, where each operation sequence label indicates the order of operations among the operation content data contained in the test scenario data; and,

[0275] Obtain basic scenario tags, where each basic scenario tag indicates the execution order of the test scenarios of the basic scenario;

[0276] Test scenario data is generated based on the corresponding operation sequence tags, the basic scenario tags, and the corresponding operation content data.

[0277] Optionally, when executing each of the test scenario data to obtain the corresponding response information, the processing module 1401 is specifically used for:

[0278] Identify the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data;

[0279] Each second test scenario data is executed separately to obtain the corresponding response information. Each second test scenario data is test scenario data that does not correspond to the basic scenario label.

[0280] The third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution is determined, and the third test scenario data is executed.

[0281] Optionally, when executing each second test scenario data and obtaining the corresponding response information, the processing module 1401 is specifically used for:

[0282] For each of the second test scenario data, perform the following operations:

[0283] According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

[0284] Optionally, when obtaining the corresponding response information, the processing module 1401 is specifically used for:

[0285] For the test scenario data, perform the following operations respectively:

[0286] Since the application under test does not depend on other services, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution results;

[0287] In response to the application under test depending on other services, the verification information corresponding to the test scenario data is added to the request information of the dependent service, and the response information returned in response to the request information is obtained.

[0288] Optionally, when adding the verification information corresponding to the test scenario data to the response information containing the scenario execution result, the processing module 1401 is specifically used for:

[0289] For the response information, perform the following operations respectively:

[0290] In response to the obtained response object, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution result;

[0291] In response to the failure to obtain the response object, the verification information is added to the information database. In response to the further acquisition of the response object, the verification information is read from the information database and added to the response information.

[0292] Optionally, when adding the verification information corresponding to the test scenario data to the request information of the service that depends on it, and obtaining the response information returned for the request information, the processing module 1401 is specifically used for:

[0293] For the requested information, perform the following operations respectively:

[0294] In response to obtaining the request object corresponding to the request information of the requested dependent service, the verification information is added to the request information, and the response information returned for the request information is obtained;

[0295] In response to the failure to obtain the requested object, the verification information is added to the information database. In response to the further acquisition of the requested object, the verification information is read from the information database and added to the request information to obtain the response information returned for the request information.

[0296] Based on the above embodiments, see Figure 15 The diagram shown is a structural schematic of an electronic device in an embodiment of this disclosure.

[0297] This disclosure provides an electronic device that may include a processor 1510 (Center Processing Unit, CPU), a memory 1520, an input device 1530, and an output device 1540. The input device 1530 may include a keyboard, a mouse, a touch screen, etc., and the output device 1540 may include a display device, such as a liquid crystal display (LCD) or a cathode ray tube (CRT).

[0298] The memory 1520 may include read-only memory (ROM) and random access memory (RAM), and provides the processor 1510 with program instructions and data stored in the memory 1520. In this embodiment of the disclosure, the memory 1520 may be used to store a program for any application testing method in this embodiment of the disclosure.

[0299] The processor 1510 executes any application testing method in this disclosure embodiment according to the program instructions stored in the memory 1520.

[0300] Based on the above embodiments, this disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the application testing method in any of the above method embodiments.

[0301] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0302] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0303] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0304] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0305] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An application testing method, characterized in that, include: Obtain test scenario data for the application under test. Each test scenario data is generated based on operation content data and operation sequence labels. In response to a request for preset background traffic data, each test scenario data is executed to obtain corresponding response information. Each response includes a scenario execution result and verification information, whereby the verification information is used to verify the correctness of the execution result of the test scenario data. Obtaining the corresponding response information specifically includes: for each test scenario data, performing the following operations: if the application under test does not depend on other services, adding the verification information corresponding to the test scenario data to the response information containing the scenario execution result; if the application under test depends on other services, adding the verification information corresponding to the test scenario data to the request information requesting the dependent service, and obtaining the response information returned for the request information; wherein adding the verification information corresponding to the test scenario data to the request information requesting the dependent service and obtaining the response information returned for the request information... The response information includes the following operations for the request information: When the application under test displays the code location of the verification information during the test and obtains the request object corresponding to the request information of the service that the request depends on, then the JavaAgent is used to add the obtained verification information to the request information and obtain the response information returned for the request information; when the application under test displays the code location of the verification information during the test but does not obtain the request object corresponding to the request information of the service that the request depends on, then the JavaAgent is used to write the obtained verification information into the information database, and when the request object corresponding to the request information is obtained, the verification information is retrieved from the information database, added to the request information, and the response information returned for the request information is obtained. The test results of the application under test are determined based on the request time, scenario execution result, and verification information of the corresponding request information, respectively. The method further includes: Obtain operation sequence tags, wherein each operation sequence tag indicates the order of operations among the operation content data contained in the test scenario data, and the operation sequence tag name is defined by a regular expression; and, Obtain basic scenario tags, where each basic scenario tag indicates the execution order of the test scenarios of the basic scenario; Test scenario data is generated based on the corresponding operation sequence tags, the basic scenario tags, and the corresponding operation content data.

2. The method as described in claim 1, characterized in that, Obtain test scenario data, specifically including: Listen for code modification events affecting the application under test; In response to the detected code modification event, obtain test scenario data.

3. The method as described in claim 1, characterized in that, Obtain test scenario data, specifically including: Acquire test scenario data according to the preset test cycle.

4. The method as described in claim 1, characterized in that, Each test scenario data is executed separately to obtain the corresponding response information, specifically including: Identify the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data; Each second test scenario data is executed separately to obtain the corresponding response information. Each second test scenario data is test scenario data that does not correspond to the basic scenario label. The third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution is determined, and the third test scenario data is executed.

5. The method as described in claim 4, characterized in that, Execute each second test scenario data separately and obtain the corresponding response information, specifically including: For each of the second test scenario data, perform the following operations: According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

6. The method as described in claim 1, characterized in that, Add the verification information corresponding to the test scenario data to the response information containing the scenario execution results, specifically including: For the response information, perform the following operations respectively: In response to the obtained response object, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution result; In response to the failure to obtain the response object, the verification information is added to the information database. In response to the further acquisition of the response object, the verification information is read from the information database and added to the response information.

7. An application testing device, characterized in that, include: The acquisition module is used to acquire test scenario data, where each test scenario data is generated based on operation content data and operation sequence tags; The processing module is used to respond to the request information for the preset background traffic data, execute each of the test scenario data respectively, and obtain the corresponding response information. Each response information includes the scenario execution result and verification information, and the verification information is used to verify the correctness of the execution result of the test scenario data. When the corresponding response information is obtained, the processing module is specifically used to perform the following operations for the test scenario data: in response to the application under test not depending on other services, add the verification information corresponding to the test scenario data to the response information containing the scenario execution result; in response to the application under test depending on other services, add the verification information corresponding to the test scenario data to the request information requesting the dependent service, and obtain the response information returned for the request information. When the verification information corresponding to the test scenario data is added to the request information of the service that depends on it, and the response information returned for the request information is obtained, the processing module is specifically used to perform the following operations for the request information: When the application under test displays the code location of the verification information during the test and obtains the request object corresponding to the request information of the service that depends on it, then the JavaAgent is used to add the obtained verification information to the request information and obtain the response information returned for the request information; when the application under test displays the code location of the verification information during the test but does not obtain the request object corresponding to the request information of the service that depends on it, then the JavaAgent is used to write the obtained verification information into the information database, and when the request object corresponding to the request information is obtained, the verification information is retrieved from the information database, added to the request information, and the response information returned for the request information is obtained. The verification module is used to determine the test result of the application under test based on the request time, scenario execution result and verification information of the corresponding request information. The device further includes a generation module, which is specifically configured to: obtain operation sequence tags, wherein each operation sequence tag indicates the operation order among the operation content data contained in the test scenario data, and the operation sequence tag is defined by a regular expression; and obtain basic scenario tags, wherein each basic scenario tag indicates the execution order of the test scenario of type basic scenario; and generate test scenario data based on the corresponding operation sequence tags, the basic scenario tags, and the corresponding operation content data.

8. The apparatus as claimed in claim 7, characterized in that, The acquisition module is specifically used for: Listen for code modification events affecting the application under test; In response to the detected code modification event, obtain test scenario data.

9. The apparatus as claimed in claim 7, characterized in that, The acquisition module is specifically used for: Acquire test scenario data according to the preset test cycle.

10. The apparatus as claimed in claim 7, characterized in that, When executing each of the test scenario data and obtaining the corresponding response information, the processing module is specifically used for: Identify the first test scenario data corresponding to the basic scenario tag whose tag type is to be executed first, and execute the first test scenario data; Each second test scenario data is executed separately to obtain the corresponding response information. Each second test scenario data is test scenario data that does not correspond to the basic scenario label. The third test scenario data corresponding to the basic scenario tag whose tag type is not prioritized for execution is determined, and the third test scenario data is executed.

11. The apparatus as claimed in claim 10, characterized in that, When executing each second test scenario data and obtaining the corresponding response information, the processing module is specifically used for: For each of the second test scenario data, perform the following operations: According to the operation sequence labels corresponding to the data in the second test scenario, execute the corresponding operation content data and obtain the corresponding response information.

12. The apparatus as claimed in claim 7, characterized in that, When adding the verification information corresponding to the test scenario data to the response information containing the scenario execution result, the processing module is specifically used for: For the response information, perform the following operations respectively: In response to the obtained response object, the verification information corresponding to the test scenario data is added to the response information containing the scenario execution result; In response to the failure to obtain the response object, the verification information is added to the information database. In response to the further acquisition of the response object, the verification information is read from the information database and added to the response information.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1-6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images