Automated testing method and device

Abstract

The present invention discloses an automated testing method and apparatus, relating to the field of computer technology. A specific embodiment of the method includes: a server selecting a test task, the test task comprising one or more test cases; the server establishing a test result set based on the test task; and the server sending a test message, wherein the test message includes an identifier of the test result set. This embodiment can increase the scalability of an automated testing system, reduce the workload of routine maintenance of the automated testing system, and reduce the system complexity caused by changes in requirements. The execution end and the server end can be automatically associated, eliminating the need for binding.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an automated testing method, device, electronic equipment, and computer-readable medium. Background Art

[0002] Automated testing is the process of converting human-driven testing activities into machine-driven execution. Typically, after a test case is designed and reviewed, testers execute the test step by step according to the procedures described in the test case, comparing the actual results with the expected results. In this process, automated testing is introduced to save manpower, time, and hardware resources and improve testing efficiency.

[0003] In the process of implementing the present invention, the inventors discovered that the prior art has at least the following problems:

[0004] The existing storage and organization method for automated test configuration information is to classify it according to the system dimension. There are multiple test plans under one system, multiple test tasks under one test plan, multiple test cases under one test task, multiple test scenarios under one test case, and multiple test data under one test scenario. Test plans, test tasks, test cases, test scenarios, and test data are stored using relational data tables. Horizontal expansion is not easy. For example, a test plan is a data table, and a test task is a data table. Adding multiple levels of test sub-plans between the test plan and the test task requires adding multiple sub-plan data tables and implementing the corresponding code, which is extremely costly to modify. It is also not easy to reuse test tasks. If a test plan wants to directly use the test tasks in another test plan, it needs to copy the test tasks to the test plan for use. If the original test tasks change, they cannot be synchronized to the test tasks in the test plan.

[0005] The existing automated test execution organization method is implemented through execution end binding. When the automated test is executed, the automated test scheduling service (i.e., the server side) randomly selects from the available execution ends based on the test task execution information, or dispatches the test task to the execution end with a fixed network address for execution. The execution end needs to be manually associated with the automated test scheduling service before the subsequent automated test execution can be realized, and automatic association cannot be achieved. It cannot be implemented for some scenarios where the execution machine scheduling does not have a fixed network address. At the same time, the correspondence between different types of test tasks and different types of execution machines is also difficult to handle, and usually only one type of execution machine with a fixed network address can be handled. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides an automated testing method and device, which can increase the scalability of the automated testing system, reduce the workload of daily maintenance of the automated testing system and the system complexity caused by changes in requirements, and the execution end and the server end can be automatically associated without binding.

[0007] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, an automated testing method is provided, comprising: the server side selects a test task, wherein the test task includes one or more test cases; the server side establishes a test result set based on the test task; the server side sends a test message, wherein the test message includes an identifier of the test result set.

[0008] Optionally, the method also includes: the execution end determines the identifier of the test result set from the test message; the execution end queries the test result set based on the identifier of the test result set; the execution end executes the test case in the test task; the execution end saves the execution result of the test case to the test result set.

[0009] Optionally, the method further includes: when the test case includes multiple test scenarios; the execution end establishes a thread according to each test scenario, and the execution end executes the thread, wherein the number of concurrently executed threads does not exceed a preset threshold.

[0010] Optionally, the method further comprises: saving the test task in a tree structure, wherein the test task is saved in a node of the tree structure, and the node has a unique identifier;

[0011] The node has a parent node and / or a child node, wherein the parent node and / or the child node store a test task associated with the test task.

[0012] Optionally, the method further includes: changing the association relationship between the test tasks by changing the parent node and / or child node of the node.

[0013] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, an automated testing device is provided, comprising: a selection module on the server side, used to select a test task, wherein the test task includes one or more test cases; an initial module on the server side, used to establish a test result set based on the test task; and a message module on the server side, used to send a test message, wherein the test message includes an identifier of the test result set.

[0014] Optionally, the device also includes: a receiving module at the execution end, used to determine the identifier of the test result set from the test message; a positioning module at the execution end, used to query the test result set according to the identifier of the test result set; an execution module at the execution end, used to execute the test cases in the test task; and a storage module at the execution end, used to save the execution results of the test cases to the test result set.

[0015] Optionally, the device also includes a thread module on the execution end, which is used to establish a thread according to each test scenario when the test case includes multiple test scenarios; the execution module on the execution end is also used to execute the thread, wherein the number of concurrently executed threads does not exceed a preset threshold.

[0016] Optionally, the apparatus further comprises: a test task organizing module, configured to store the test tasks in a tree structure, wherein the test tasks are stored in nodes of the tree structure, and the nodes have unique identifiers;

[0017] The node has a parent node and / or a child node, wherein the parent node and / or the child node store a test task associated with the test task.

[0018] Optionally, the selection module on the server side is further configured to modify the association relationship between the test tasks by changing the parent node and / or child node of the node.

[0019] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in an automated testing method.

[0020] To achieve the above object, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by one or more processors, any method described in an automated testing method is implemented.

[0021] One embodiment of the above invention has the following advantages or beneficial effects: because the technical means of sending test messages on the server side is adopted, the technical problem of the traditional method requiring binding of the execution side is overcome, thereby achieving the technical effect of automatically associating the execution side with the server side.

[0022] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0024] Figure 1 is a schematic diagram of the main steps on the server side of an automated testing method according to an embodiment of the present invention;

[0025] Figure 2 2. It is a schematic diagram of a tree-structured test plan according to an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of the main steps of the execution end of an automated testing method according to an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of the main parts of a server side of an automated testing device according to an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of the main parts of an execution end of an automated testing method device according to an embodiment of the present invention;

[0029] Figure 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0030] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] Figure 1 FIG. 1 is a schematic diagram of the main steps of an automated testing method according to an embodiment of the present invention. Figure 1 As shown:

[0033] Step S101 represents that the server side selects a test task, and the test task includes one or more test cases. A test task refers to performing a certain operation with a test target, such as executing a program on a computer to test the execution of the program on this computer. The test task may include one or more test subtasks, and one or more test tasks may also form a test plan (a test plan and a test task are essentially equivalent), the test task includes one or more test cases, and the test case may also include the data of one or more test scenarios. The server side is used to manage and organize the test task or the test plan so that the test target completes a specific test task. When the server side selects the test task, it also determines the test case included in the test task, and the data of the test scenario included in the test case.

[0034] The method further includes: storing the test tasks in a tree structure, wherein the test tasks are stored in nodes of the tree structure, each node having a unique identifier; the nodes having parent nodes and / or child nodes, wherein the parent nodes and / or child nodes store test tasks associated with the test tasks. The test tasks stored in the tree structure can be stored in an external database. If a test task includes multiple test subtasks, the test task can be stored in a parent node of the tree structure, and the test subtasks can be stored in child nodes of the parent node, each node having a unique identifier. Obviously, the concepts of parent node and child node are relative. Furthermore, the unique identifier of the corresponding child node is stored in the parent node. A tree structure is a data structure in which data elements have a "one-to-many" tree-like relationship. It is a nonlinear data structure. Generally, in a tree structure, the root node has no parent node, each remaining node has one and only one parent node, leaf nodes have no child nodes, and each remaining node can have one or more child nodes. The tree structure can represent hierarchical relationships, subordinate relationships, parallel relationships, and recursive relationships between data elements. The parent node and / or the child node store test tasks associated with the test task, i.e., the test task and the test subtask are associated with each other, and multiple test tasks or test subtasks form a complete test task plan using the test task execution logic as a recursive relationship in a tree structure. For example, if the test task execution logic is that test task 1 includes test subtask 1.1, then in the tree structure, the node where test task 1 is located is the parent node of test subtask 1.1, and the node where test subtask 1.1 is located is the child node of test task 1.

[0035] By using a tree structure to save test tasks or test subtasks, horizontal lateral expansion of automated test tasks is achieved. Because the tree structure has multiple levels of nodes, the root node of the tree can be a test plan, each node can be a test task, and the leaf node of the tree can be a specific test case, which realizes the horizontal lateral expansion problem of unlimited hierarchical size. Furthermore, a recursive relationship can be established between the nodes of the tree in the form of links. By changing the parent node and / or child node of the node, the association relationship between the test tasks is changed. For example, the child node B (containing test task Y) of node A (containing test task X) is replaced with the child node C (containing test task Z), then the test subtask Y of test task X is replaced with the test subtask Z. Since when recursing the tree structure, it is generally recursive from the root node to the leaf node step by step, so the recursive relationship between the parent node and the child node can be established by saving the unique identifier of the child node (such as node name, node number, etc.) in the parent node. Those skilled in the art should understand that if the child node of the parent node also has child nodes (herein referred to as grandchild nodes for distinction), then the recursive relationship between the parent node and the child node can be established by only saving the unique identifier of the child node in the parent node. If the unique identifier of the grandchild node is saved in the parent node, the recursive relationship established in this way may cause confusion in the relationship between the nodes, that is, it is sufficient to save the unique identifier of the node of the next level corresponding to it in a node. When selecting a test task, it is only necessary to select the node of the required test task or test subtask, and construct a tree-structured test plan with the execution logic of the test task as the recursive relationship of the tree structure, or on the basis of the existing test plan, by modifying the recursive relationship between the test task nodes, construct a tree-structured test plan. This step solves the problem of executing automated tests across root test plans.

[0036] For example, node A is an embedded list data of Map data type (Map is a data structure with key-value pairs), and the Map data is {id:id value, name:name, childs:[child id1, child id2, child id3]}, where the id part stores the unique identifier of the node (such as id value), the name part stores the name of the node, and the childs part stores the unique identifiers of the child nodes of node A, child id1, child id2 and child id3. The child nodes can also have an independent Map data type data corresponding to them (that is, the grandchild node of node A). By recursively traversing the node, you can obtain information about all the child nodes contained in the node.

[0037] For another example, Figure 2The figure shows a tree-structured test plan according to an embodiment of the present invention. As shown in the figure, node 1 is the root node; nodes 3, 5, and 6 are leaf nodes; node 4 is a child node of node 2; and nodes 5 and 6 are child nodes of node 4 and grandchild nodes of node 2. Node 1 stores the unique identifiers of nodes 2 and 3, node 2 stores the unique identifier of node 4, and node 4 stores the unique identifiers of nodes 5 and 6. Based on the above recursive relationship, information about nodes 2, 3, 4, 5, and 6 can be recursively obtained through node 1.

[0038] Furthermore, when the content of a node is changed, the parent node and the root node of the node will not be affected. Furthermore, since different types of automated test frameworks have different attribute values, it is a relatively abstract process for the configuration process of the automated test system and has nothing to do with the specific test task type. For example, the display of the tree structure only requires recursive traversal of the node information with permission (i.e., permission to query), assembling the tree structure information as JSON data, and returning it to the front page. For the node information of the tree, only the unique identifier of the node and the unique identifier of the stored child node, the node name and other information are concerned. For different types of automated test frameworks, additional composite type data can be stored in the test plan root node. For example, the type of test task and the attribute value of the test task, etc., these characteristic values can be placed in the basic configuration information child node under the root node of the test plan. Since the tree structure has the characteristic of no fixed structure, it is convenient to store different types of automated test tasks without creating an additional data structure. For the software development process, the change of software requirements will also bring about the increase and change of system fields. For the tree structure storage, there is no need to create an additional database structure, which saves the process of designing and modifying the database structure. Since the tree structure has no fixed structure, the number of tables in the database system can also be greatly reduced.

[0039] The present invention provides a flexible test configuration information storage scheme, solves the changeable demand of automated test, also provides multiple information organization mode to automated test personnel, reduces the design complexity of automated test system, reduces relevant development workload. Traditional automated test system is difficult to adapt to the configuration and execution demand of changeable automated test because it adopts relational database to store, is also difficult to realize the customization demand of automated test personnel (different automated test frameworks, the difference between different tested projects is very large, usually difficult to be compatible), and the cost of secondary development and modification of system is higher. For carrying out the same test task under different test environments, the method for adopting test plan root node link subnode in this invention solves this problem. For example, when creating link operation, initiated by the relevant personnel of test plan root node, select root node information and destination root node, synchronize all subnodes under this root node, copy under destination root node, do not need to create new test task subnode information again, can reuse test task subnode information. Like this when automated test personnel revise the information under test task subnode, test plan also can be automatically updated, avoids the problem of needing to maintain two sets of test plans simultaneously, because the test task that synchronously produces is still the same task, can also avoid the problem that same task is carried out simultaneously.

[0040] The automated testing system also displays all test plan information. A test plan represents the root node information of a group of test plans, including relevant stakeholder email addresses, basic configuration information, and the network address of the execution machine. All child nodes of the test plan root node, namely task nodes and subtask nodes, share the configuration information of the root node. A test plan root node can have multiple child nodes. A child node can be a test subplan or a test task. The root node stores the child node's unique identifier. Detailed information about the child node can be stored as another independent node. Test subplan nodes can also have child nodes under them, and the node containing the test case can be the final leaf node. Test case nodes store test case data and test scenario data. When editing a test case, specify the required test scenario data name. Multiple scenario data names are allowed. Test case and test scenario data can be stored in a table, as shown in Table 1. When selecting different test case combinations in a test task, the corresponding table data can be displayed in the test scenario data table based on the scenario name.

[0041] Table 1

[0042] Scene Name Use case data item name 1 Use case data item name 2 … Scenario 1 Use case data item value 1 Use case data item value 2 … Scenario 2 Use case data item value 1 Use case data item value 2 … … … … …

[0043] The root node of the test plan may include:

[0044] Root node unique identifier;

[0045] Root node name;

[0046] Test stakeholder email information;

[0047] Basic configuration information (multiple items);

[0048] Message queue name;

[0049] Test plan type;

[0050] Information about the person who created and modified the test plan (including the corresponding organizational information);

[0051] The unique identifier of the child node.

[0052] The test sub-plan node can include:

[0053] Node unique identifier;

[0054] Node name;

[0055] Test sub-plan creation and modification information (including corresponding organizational information);

[0056] The unique identifier of the child node.

[0057] The test task node may include:

[0058] Node unique identifier;

[0059] Node name;

[0060] Test task creator and modifier information (including corresponding organizational information); associated test case sequence:

[0061] Test case node unique identifier 1 and test case name;

[0062] Test case node unique identifier 2 and test case name;

[0063] Test case node unique identifier 1 and test case name;

[0064] Scene data.

[0065] Automation testers can add and maintain the required information in the system based on the tree-structured test plan and select one or more test plans and test tasks to execute from the tree structure. Different nodes can have additional data items to meet the execution requirements of different test frameworks.

[0066] The sample code for recursively traversing the tree structure is as follows:

[0067]

[0068]

[0069] Step S102 indicates that the server creates a test result set based on the test task. The test result set is used to store relevant data during and after the execution of the test task. Furthermore, the test result set can also be stored in the node, and the test result set can be initialized before creating the test result set.

[0070] The test result set node can include:

[0071] Node unique identifier

[0072] Create and modify person information (including corresponding organizational information)

[0073] Associated test case order:

[0074] Test case node unique identifier 1 and test case name

[0075] Test case node unique identifier 2 and test case name

[0076] Test case node unique identifier 1 and test case name

[0077] Scene data.

[0078] The scene data may include a list of checkpoint results.

[0079] Step S103 indicates that the server sends a test message, wherein the test message includes an identifier of the test result set.

[0080] The automated test server and execution side communicate via test messages, which can be stored in a test message queue. A test task node can include the name of the message queue in the root node of the test plan. When executing a test task, the message is distributed to the test side based on the message queue name. The test message includes the identifier of the initialized test result set corresponding to the test task node to be executed.

[0081] If the execution end is not executed when performing automated testing, the test message will be placed in the message queue and the execution end will process the corresponding test message after it is started.

[0082] The sample code for sending a message on the server is as follows:

[0083]

[0084] Figure 3 Schematic diagram showing the main steps of the execution end of an automated testing method according to an embodiment of the present invention;

[0085] Step S301 indicates that the execution end determines the identifier of the test result set from the test message; step S302 indicates that the execution end queries the test result set based on the identifier; step S303 indicates that the execution end executes the test case in the test task; and step S304 indicates that the execution end saves the execution result of the test case to the test result set. By using the execution end to share the computational workload of the server, the pressure on the automation system is reduced when the number of execution ends is large.

[0086] After the test execution end receives the test message sent by the server, it parses the test message and obtains the identifier of the test result set. Based on the identifier of the test result set, it queries the test result set and creates a directory in the test result set. The corresponding test environment is initialized according to the data of the test plan root node, and the test code included in the test task (the test code is the implementation of the test case) is downloaded. The corresponding call configuration information is generated in the corresponding directory of the test code according to the test case and test scenario data. According to the configuration information, the test code is executed. After execution is completed, the required execution results are parsed according to the execution log of the test code, and the results are saved to the test result set. The corresponding values in the scenario data are written into the checkpoint result list under different scenarios. Since the test code execution process changes the value of the test case data item, the value is also updated. The execution of this test task ends. The execution end will continue to execute and wait for the test message sent by the server to execute other test tasks. In traditional automated testing, scheduling the execution end requires an execution end resource pool and binding the network address of the execution end, which cannot dynamically specify a new execution end. The present invention solves this problem by using message communication.

[0087] The method further includes: when the test task includes multiple test scenarios; the execution end establishes a thread according to each of the test scenarios; the execution end executes the threads, wherein the number of concurrently executed threads does not exceed a preset threshold.

[0088] If there are multiple test scenarios in a test task, and the test cases in them are non-user interface operation test cases, then these test cases will be executed in parallel on the execution end.

[0089] When a test task includes multiple scenarios, the execution end creates a subdirectory in the directory created by the initialized test result set using the test scenario identifier (such as the scenario name, scenario number, etc.), and downloads the configured test case code to the subdirectory corresponding to the test scenario, with each scenario corresponding to a subdirectory. When initializing the test scenario data, the data under different test scenarios will be generated into the configuration information under the corresponding test scenario. Therefore, each test scenario can correspond to an execution thread of the test code, and each thread is used to execute the test code under the corresponding test scenario directory.

[0090] You can use a thread pool to launch automated tests for multiple scenarios. Pre-set a threshold for the number of concurrent threads to avoid excessive test scenarios, which could lead to system crashes. After the test code completes, the code execution log is used to analyze the required results. Based on the test scenario identifier, the results are written to the different test scenario result data in the test result set. The thread is then released, and the thread pool continues executing the thread for the next pending test scenario. This process ends when there are no more pending test scenario threads in the thread pool and the current thread completes execution.

[0091] Furthermore, in the absence of an available execution terminal, a virtual machine or container can be started locally (i.e., on the server side) and the configuration of the execution terminal can be deployed within its system, thereby scheduling the execution of local automated test tasks. Typically, the local network can access the corresponding test environment (i.e., the execution terminal) network, while the test environment network typically cannot access the local network. Virtual machines or containers can share the same network permissions as the local network by using a shared network, and the communication method for test messages is to pull messages from consumers to producers. Therefore, test tasks can be executed within local virtual machines or containers. Furthermore, when the existing execution terminal environment does not meet the requirements of the corresponding automated test environment, the above method can also be used to implement automated testing.

[0092] The sample code for the execution end to receive the test message and execute the test case is as follows:

[0093]

[0094]

[0095]

[0096] Figure 4 FIG. 4 is a schematic diagram of the main parts of the server side of an automated testing device 400 according to an embodiment of the present invention. Figure 4 As shown:

[0097] The server-side selection module 401 is used to select a test task, which includes one or more test cases. When the server selects a test task, it also determines the test cases included in the test task and the test scenario data included in the test cases.

[0098] The initialization module 402 of the server is used to establish a test result set according to the test task. Furthermore, the test result set can also be stored in the node, and the test result set can be initialized before establishing the test result set.

[0099] The server-side message module 403 is used to send a test message, wherein the test message includes the identifier of the test result set. The server-side of the automated test communicates with the execution end via test messages, and the test message can be stored in a test message queue. The test task node may include the name of the message queue in the root node of the test plan. When the corresponding test task is executed, it will be distributed to the test end according to the message queue name. The test message includes the identifier of the initialized test result set corresponding to the test task node to be executed.

[0100] The selection module 401 on the server side is further configured to modify the association relationship between the test tasks by changing the parent node and / or child node of the node.

[0101] The device 400 also includes: a test task organization module, which is used to save the test task in a tree structure, and the test task is saved in a node of the tree structure, and the node has a unique identifier; the node has a parent node and / or a child node, wherein the parent node and / or the child node saves the test task associated with the test task. When the selection module 401 on the server side selects a test task, it only needs to select the node of the required test task or test subtask, and use the execution logic of the test task as the recursive relationship of the tree structure to construct a test plan with a tree structure, or on the basis of the existing test plan, by modifying the recursive relationship between the test task nodes, a test plan with a tree structure is constructed, thereby solving the problem of executing automated tests across root test plans. Furthermore, when the content of a node changes, it will not affect the content of the parent node and the root node of the node. Furthermore, since different types of automated testing frameworks have different attribute values, the configuration process of the automated testing system is a relatively abstract process and has nothing to do with the specific test task type. When the automated tester modifies the information under the test task sub-node, the test plan will also be automatically updated, avoiding the need to maintain two sets of test plans at the same time. Since the test tasks generated synchronously are still the same task, the problem of simultaneous execution of the same task can also be avoided.

[0102] Figure 5 FIG. 4 is a schematic diagram of the main parts of the execution end of an automated testing method device 400 according to an embodiment of the present invention. Figure 5 As shown:

[0103] The receiving module 501 of the execution end is configured to determine the identifier of the test result set from the test message.

[0104] The positioning module 502 of the execution end is used to query the test result set according to the identifier of the test result set.

[0105] The execution module 503 of the execution end is used to execute the test cases in the test task.

[0106] The storage module 504 of the execution end is used to save the execution result of the test case to the test result set.

[0107] After the receiving module 501 of the execution end receives the test message sent by the server end, it parses the test message and obtains the identification of the test result set. The positioning module 502 of the execution end queries the test result set according to the identification of the test result set, and can set up a directory in the test result set. According to the data initialization corresponding test environment of the test plan root node, the test code (test code is the realization of the test case) included in the test task is downloaded. According to the test case and the test scenario data, corresponding call configuration information is generated in the corresponding directory of the test code. According to the configuration information, the execution module 503 of the execution end executes the test code. After execution is completed, according to the execution log of the test code, the required execution result is parsed. The storage module 504 of the execution end saves the result to the test result set, and the corresponding value in the scenario data is written into the checkpoint result list under different scenarios. Since the test code execution process can change the value of the test case data item, the value is also updated. End the execution of this test task. The execution end will continue to execute and wait for the test message sent by the receiving server end to perform other test tasks. Traditional automated testing requires a resource pool for executing terminals, which binds the terminal's network address and prevents the dynamic assignment of new terminals. This invention addresses this problem by utilizing message communication. By using the executing terminals to share the server's computational workload, the pressure on the automation system is reduced when the number of executing terminals is large.

[0108] The device 400 also includes a thread module at the execution end, which is used to establish a thread according to each test scenario when the test case includes multiple test scenarios. The execution module 503 at the execution end is also used to execute the threads, wherein the number of concurrently executed threads does not exceed a preset threshold. Automated testing of multiple scenarios can be started in the form of a thread pool, and a threshold is set in advance for the concurrent number of threads to avoid too many test scenarios and too many threads being opened, causing the system to freeze. After the test code is executed, the required execution results are parsed according to the code execution log, and the results are written into the different test scenario result data in the test result set according to the identification of the test scenario. The thread is released, and the thread pool continues to execute the thread of the next test scenario to be executed. The current test task is terminated until there are no test scenario threads to be executed in the thread pool and the current thread execution is completed.

[0109] Figure 6 An exemplary system architecture 600 is shown to which an automated testing method or an automated testing device according to an embodiment of the present invention may be applied.

[0110] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0111] Users can use terminal devices 601, 602, 603 to interact with server 605 via network 604 to receive or send messages, etc. Various communication client applications, such as database applications and automated testing applications, can be installed on terminal devices 601, 602, 603.

[0112] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0113] Server 605 may be a server that provides various services, such as a background management server that provides support for users using terminal devices 601, 602, and 603. The background management server may analyze and process received data such as product information query requests, and feed back processing results (such as test task node information, etc.) to the terminal device.

[0114] It should be noted that the automated testing method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, an automated testing device is generally provided in the server 605 .

[0115] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0116] Figure 7 FIG. 7 is a schematic diagram showing the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0117] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the system 700 are also stored in the RAM 703. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0118] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0119] In particular, according to the embodiments disclosed in the present invention, the process described in the above step diagram can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the step diagram. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are performed.

[0120] It should be noted that the computer-readable medium described in the present invention includes computer-readable signal media or computer-readable storage media, or any combination thereof. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. Computer-readable storage media specifically include, but are not limited to, electrical connections having one or more conductors, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In the present invention, computer-readable storage media includes any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device; computer-readable signal media includes a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF (radio frequency), or any combination of the above.

[0121] The step diagrams or block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the step diagram or block diagram can represent a module, program segment, or part of the code, and the above module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed in parallel, or sometimes in the opposite order, and the order of execution depends on the functions involved. It should also be noted that each box in the block diagram or step diagram and its combination can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The modules or units involved in the embodiments of the present invention may be implemented in software or hardware. The modules or units described may also be provided in a processor. For example, they may be described as follows: a processor including a selection module on the server side, an initial module on the server side, and a message module on the server side. The names of these modules or units do not, in certain cases, constitute limitations on the modules or units themselves. For example, the initial module on the server side may also be described as a "module for establishing a test result set according to the test task."

[0123] In another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment, or may exist independently and not incorporated into the device. The computer-readable medium carries one or more programs. When executed by the device, the device may: select a test task on a server side, the test task including one or more test cases; create a test result set based on the test task; and send a test message, wherein the test message includes an identifier of the test result set.

[0124] According to the technical solution of the embodiment of the present invention, the scalability of the automated testing system can be increased, the workload of daily maintenance of the automated testing system and the system complexity caused by demand changes can be reduced, and the execution end and the server end can be automatically associated without binding.

[0125] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An automated testing method, characterized in that: include: The server side selects a test task, wherein the test task includes one or more test cases; The server side establishes a test result set according to the test task; The server sends a test message, wherein the test message includes an identifier of the test result set; After receiving the test message sent by the server, the test execution end determines the identifier of the test result set from the test message, queries the test result set according to the identifier of the test result set, establishes a directory in the test result set, initializes the corresponding test environment according to the data of the test plan root node, downloads the test code included in the test task, generates corresponding configuration information in the corresponding directory of the test code according to the test case and test scenario data, and executes the test code according to the configuration information; After the execution is completed, the required execution results are parsed according to the execution log of the test code, the execution results are saved to the test result set, and the corresponding values in the scenario data are written into the checkpoint result list under different scenarios; The method further includes: storing the test task in a tree structure, wherein the test task is stored in a node of the tree structure, and the node has a unique identifier; The node has a parent node and / or a child node, wherein the parent node and / or the child node stores the test task associated with the test task; and the parent node stores a unique identifier of its corresponding child node.

2. The method according to claim 1, characterized in that The method further comprises: In the case where the test case includes multiple test scenarios; The execution end establishes a thread according to each of the test scenarios; The execution end executes the threads, wherein the number of concurrently executed threads does not exceed a preset threshold.

3. The method according to claim 1, characterized in that The method further comprises: By changing the parent node and / or child node of the node, the association relationship between the test tasks is changed.

4. An automated testing device, characterized in that: include: A selection module on the server side is used to select a test task, wherein the test task includes one or more test cases; An initial module on the server side, used to create a test result set according to the test task; A message module on the server side is used to send a test message, wherein the test message includes an identifier of the test result set; A receiving module of the execution end, configured to determine an identifier of the test result set from the test message; A positioning module of the execution end, configured to query the test result set according to the identifier of the test result set; An execution module on the execution end is used to establish a directory in the test result set, initialize the corresponding test environment according to the data of the test plan root node, download the test code included in the test task, generate corresponding configuration information in the corresponding directory of the test code according to the test case and test scenario data, and execute the test code according to the configuration information; The storage module on the execution side is used to parse the required execution results according to the execution log of the test code after execution is completed, save the execution results to the test result set, and write the corresponding values in the scenario data into the checkpoint result list under different scenarios; The device also includes: a test task organization module, which is used to save the test task in a tree structure, and the test task is saved in a node of the tree structure, and the node has a unique identifier; the node has a parent node and / or a child node, and the parent node and / or the child node saves the test task associated with the test task; the unique identifier of its corresponding child node is saved in the parent node.

5. The device according to claim 4, characterized in that The apparatus further comprises a thread module at an execution end, configured to establish a thread according to each test scenario when the test case includes multiple test scenarios; The execution module of the execution end is further configured to execute the threads, wherein the number of concurrently executed threads does not exceed a preset threshold.

6. The device according to claim 4, characterized in that The selection module on the server side is further configured to modify the association relationship between the test tasks by changing the parent node and / or child node of the node.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

8. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by one or more processors, the method according to any one of claims 1 to 3 is implemented.

Images