A method, apparatus, device, and storage medium for automated testing

Abstract

The present disclosure provides a method, device, equipment and storage medium for automated testing, relating to the technical field of computer data processing, and particularly to the technical field of software testing. The specific implementation scheme is as follows: obtaining task information of a first test task that has been executed, the task information including task identification, task execution parameters and / or task execution results; obtaining case information of a test case executed by the first test task, the case information including program code and execution results of the test case; and displaying the task information and the case information. The method provides more basis for analyzing the failure cause of the test case by obtaining the task information and the case information of the executed test task, including the program code of the test case, and displaying the same, and is more conducive to quickly locating the failure cause of the test case.

Description

Technical Field

[0001] This disclosure relates to the field of computer data processing technology, and more particularly to the field of software testing technology. Background Technology

[0002] Currently, test reports typically only show success (pass) or failure (fail) for each test case, along with limited log information. Most failed cases require syncing the test code locally, running it multiple times, and analyzing the results to find the root cause of the problem. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for automated testing.

[0004] According to one aspect of this disclosure, an automated testing method is provided, comprising: obtaining task information of an executed first test task, the task information including a task identifier, task execution parameters and / or task execution results; obtaining test case information of test cases executed by the first test task, the test case information including the program code and execution results of the test cases; and displaying the task information and test case information.

[0005] According to another aspect of this disclosure, an automated testing apparatus is provided, comprising: a task information acquisition module for acquiring task information of an executed first test task, the task information including a task identifier, task execution parameters, and / or task execution results; a test case information acquisition module for acquiring test case information of test cases executed by the first test task, the test case information including the program code and execution results of the test cases; and a test information display module for displaying the task information and test case information.

[0006] According to another aspect of this disclosure, an automated testing integration system is provided, comprising: a test task execution subsystem for executing a first test task or a second test task in the above-described automated testing method; a test task report display subsystem for displaying task information and test case information in the above-described automated testing method; and a test case source code management subsystem for obtaining program code from the test case information of the above-described automated testing method.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described automated testing methods.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to perform any of the above-described automated tests.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method for any of the above-described automated tests.

[0010] This disclosure provides an automated testing method, apparatus, device, and storage medium. The method acquires and displays task information and test case information of executed test tasks, including the program code of the test cases, thereby providing more evidence for analyzing the reasons for test case failures and making it easier to quickly locate the reasons for test case failures.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is a flowchart illustrating the method for implementing automated testing according to the first embodiment of this disclosure;

[0014] Figure 2 This is a schematic diagram of the process of executing a test task according to the second embodiment of this disclosure;

[0015] Figure 3 This is a schematic diagram of a graphical interface for displaying reports and analyzing failed test cases according to the third embodiment of this disclosure;

[0016] Figure 4 This is a schematic diagram of the page hierarchy structure for displaying classification statistics information according to the fourth embodiment of this disclosure;

[0017] Figure 5 This is a schematic diagram of the test framework used in the fifth embodiment of this disclosure;

[0018] Figure 6 This is a schematic diagram of the structure of the automated testing apparatus according to an embodiment of this disclosure;

[0019] Figure 7 This is a schematic diagram of the structure of the automated testing integration system according to an embodiment of the present disclosure;

[0020] Figure 8 This is a block diagram of an electronic device used to implement the automated testing method of the embodiments of this disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] Figure 1 The main flow of an embodiment of the present disclosure for implementing automated testing is shown, including: providing an automated testing method, including:

[0023] Operate S110 to obtain the task information of the first test task that has been executed. The task information includes the task identifier, task execution parameters and / or task execution result.

[0024] Among them, a test task refers to a computer task used to execute a set of test cases, including obtaining test cases to be executed, setting up the test environment for executing the test, executing the test cases, collecting test execution results and log information and environment information during the execution process, and a series of other operations.

[0025] Typically, each test task is assigned a unique character or number to identify its execution; this is known as the task identifier. This identifier allows real-time access to the task's status and the execution parameters used when submitting it. If the test task has completed, the identifier can also be used to retrieve the execution result.

[0026] Task execution parameters mainly refer to the parameters specified when executing a task, such as the scope of test cases to be executed, the test platform used to execute the test cases, and the mode (parallel, serial) used to execute the test cases, etc.

[0027] The testing platform used refers to a specific framework that drives the test execution, such as Ant, Gradle, Postman, etc.

[0028] The task execution results mainly include: the completion status of the test task, including completion, interruption, or exit; the execution time of the test task, including start time and end time; relevant information collected during the execution of test cases, such as screen output information, response information received by the test terminal, log file information, and test environment information (including the machine name, operating system, and runtime engine of the test); and the execution results of the test cases.

[0029] Operate S120 to obtain the test case information of the test cases executed by the first test task. The test case information includes the program code and execution result of the test cases.

[0030] The execution result of a test case refers to whether the test case passes or fails. The execution result of a test case can be obtained from the execution result of the test task.

[0031] Test cases are usually written in programming languages ​​or scripting languages. Before executing the test cases, the source program containing these programming languages ​​or scripting languages ​​is compiled into an executable binary target program. Then, the binary target program is executed to obtain the execution result of the test cases.

[0032] Test case code refers to the program code in the source program, especially the program code used to control test steps and determine test results.

[0033] The program code for test cases can typically be obtained through the following methods:

[0034] 1) Obtain from the code repository storing the source program;

[0035] 2) If the source code of the test cases is packaged in the task execution result of the test task, it can also be obtained from the task execution result.

[0036] Operate S130 to display task information and use case information.

[0037] Since the same set of test cases may produce different results when executed on different test platforms, operating systems, or machines, obtaining and displaying the task information of executing test cases helps to compare the task execution results and task execution parameters, and to obtain the correlation between the task execution results and task execution parameters based on the comparison results, thereby helping to further locate the root cause of test failure.

[0038] For example, if a set of tests consistently fails on Windows but succeeds on Linux, it indicates that the test failures are operating system-related, thus confirming that the test results are OS-dependent. Developers can then focus their analysis on the code segments relevant to the operating system.

[0039] For example, if all execution parameters of a test task are the same, but it keeps failing from a certain point in time, it may be caused by a change made at a point before that point. In this case, the developers can focus on analyzing the code segments that were modified or modified in the period before that point.

[0040] In this way, even without repeating the test tasks, the reasons for the test case failures can be simply analyzed, which can greatly reduce the number of times the test tasks are repeated.

[0041] Obtaining and displaying the program code of test cases helps to understand the test logic and test intent, as well as the context information of test execution; if the test case fails due to an error in the test program code, the error can also be directly observed by obtaining the program code of the test case.

[0042] In this way, when users see the execution results of the test cases, they can also see the program code of the test cases without having to download the corresponding code to their local machine, thus making it easier to analyze the reasons for the failure of the test cases.

[0043] It should be noted that Figure 1 The illustrated embodiment is merely a basic example of the automated testing method disclosed herein. Implementers may further modify the implementation according to their needs and conditions. Figure 1 The methods used in the illustrated embodiments can be further refined and expanded.

[0044] Figure 2 Another embodiment of this disclosure is shown, illustrating the main flow of receiving specified task execution parameters and executing a first test task. (See also...) Figure 2 As shown, it includes:

[0045] Operate S2010 to receive test task execution parameters;

[0046] The task execution parameters include:

[0047] The module name for automated test cases;

[0048] Task sources: for example, tinker, sandbox, tower (checker), agile, online, monitor;

[0049] Target address (target): The address of the task target;

[0050] Agile pipeline ID;

[0051] Task triggerer;

[0052] Task submission time, etc.

[0053] Operation S2020: Determine if the required parameters are missing. If so, continue with operation S2030; otherwise, continue with operation S2040.

[0054] Implementers can specify some task execution parameters as required, requiring a value to be specified; while other non-required parameters can be specified or not, and if not specified, the default value will be used.

[0055] When operating S2030, a message appears indicating that a required parameter is missing.

[0056] In step S2040, check if a basic task is required for the test. If not, continue to step S2050; if so, continue to step S2060.

[0057] When determining whether a base task is required for a test, you can specify it using dedicated task execution parameters, such as execution parameters indicating whether a base task needs to be executed. Alternatively, you can determine it by the test source. For example, test tasks originating from the sandbox generally do not require a base task, while test tasks originating from the tinker require a base task for comparison.

[0058] Operation S2050: Add one test task data to the task table.

[0059] Operation S2060: Add one basic task data and one test task data to be compared to the task table.

[0060] The test task can then be executed immediately, or it can be started by another thread at a specified time. The process of executing the test task mainly includes:

[0061] Operate S2070 to obtain task execution parameters;

[0062] Operate S2080 to execute the specified test cases on the test platform corresponding to the source of the task, based on the task execution parameters;

[0063] In this embodiment of the disclosure, a mapping table between different task sources and test platforms is also established. When a user only knows the test source but not the corresponding test platform, the mapping table can be used to determine the test platform to be run.

[0064] In other embodiments, if the user clearly knows the test platform to be used, the test platform can be specified directly in the execution parameters. In this way, the user does not need to know the specific execution instructions or scripts for each test platform; they only need to specify the test source or test platform, greatly simplifying the operation of executing test tasks.

[0065] Operate S2090 to collect logs generated during test case execution;

[0066] In step S2100, check if the test cases in the log are in the test case table. If not, continue to step S2120; if yes, continue to step S2130.

[0067] Here, the test case table is used to collect the test cases executed in this test task. Based on the test cases executed in this test task, it is convenient to collect detailed information related to the test cases, count the test cases, and associate them with the test case source code in the code repository.

[0068] Operation S2110: Add test cases to the test case table in the database.

[0069] Operate S2120 to collect detailed information about test cases and add them to the test case information table.

[0070] In this embodiment of the disclosure, the detailed information of the test cases includes:

[0071] Test case path;

[0072] Test case name;

[0073] Test verification points: the expected and actual values ​​output by the test code;

[0074] Test execution status: Log information related to the test;

[0075] Operate S2130 to update task information.

[0076] Record the execution results of the test tasks based on their execution results.

[0077] If the test task is associated with a base task and is to be compared, then it is also necessary to compare the test cases of the test task and the base task, find the parts in the execution result of the test task to be compared that are different from the test case information of the same test cases in the execution result of the base task, that is, the test case difference information, and mark them.

[0078] Therefore, when displaying use case information, more eye-catching formats such as "bold" or "red" can be used to highlight the differences.

[0079] In this embodiment of the disclosure, test tasks can be executed in parallel. For example, a process can be created for each user executing a test task, allowing test tasks executed by multiple users to run in parallel without affecting each other.

[0080] In this embodiment of the disclosure, test cases in the test task can also be specified by parameters to run in parallel. For example, when the parameter for determining the running mode indicates parallel execution, a new thread is started to run the specified test cases.

[0081] This embodiment of the disclosure can receive different test task execution parameters, and can determine the platform used to execute the test, the test cases to be executed, and the test execution mode through the task execution parameters. In this way, it can meet various user needs and encapsulate the differences in executing test cases on various test platforms, thereby greatly simplifying the operation of executing test cases on different test platforms. Regardless of the test source or the test platform used, execution can be performed from a single entry point, making it more convenient to use.

[0082] Figure 3 Another embodiment of this disclosure is shown, in Figure 3 In the illustrated embodiment, a graphical user interface (GUI) is used to display task and use case information as controls or pages. Figure 3 The content shown in the box is the control (e.g., button, text box, list or frame) or page on the graphical interface, which implementers can flexibly choose according to implementation needs and conditions.

[0083] The report display 301 shows the task information 3011 in the task table, the test report, and detailed test case information 3012 for each test. The detailed test case information includes test case information 30121, log information, and server response information 30122 from the test case table.

[0084] If the response information returned by the server is serialized, this embodiment of the disclosure will further deserialize the serialized information before displaying it. This ensures that the server response information 30122 is always readable and meaningful.

[0085] Specifically, for ordinary test tasks, the test report and detailed test case information 3012 for each test will display the failed test cases and the number of failed test cases; if there are no failed test cases, the task execution result will be displayed as "success (pass)".

[0086] For test tasks that are related to a base task and are to be compared, the test report and detailed test case information for each test will display: the number of test cases that failed in the base task; the number of test cases that failed in the test task; the number of test cases (diff cases) that differ in results between the test task and the base task; if the number of test cases with different results is greater than 1, a button will be displayed. Clicking the button will display detailed information about the test cases with different results.

[0087] In this embodiment of the disclosure, the test case program code is displayed through the graphical interface viewing test case code 302. Subsequently, if the user needs to modify the program code, they can also edit the user code 309 to display the integrated development environment (WebIDE), allowing the user to quickly modify and submit the test case code without having to set up an additional integrated development environment, download, and submit the test code.

[0088] Specifically, in this embodiment of the disclosure, after clicking "Edit test case code 309", the code of the test case will be automatically pulled from the code library, the dependent library (lib) will be installed, and the test case can be added, deleted, searched, modified, run and submitted through the page integrated development environment.

[0089] In addition, the graphical interface also provides a test case retry function. Users can confirm the task execution parameters of the retry test task based on the task information 3011 in the task table and the test report and detailed test case information 3012, and specify the task execution parameters to execute the test task 308 to re-execute the failed test cases.

[0090] Furthermore, in this embodiment of the disclosure, multiple retry methods are provided, including: original instance retry 304, custom retry 305, and online retry 306.

[0091] The original instance retry 304 option is used to re-execute the test cases on the original instance with the original task execution parameters. After the execution is completed, the results, verification point information, log button (for viewing log information), and response information button (for viewing server response information) are displayed through the execution result pop-up window.

[0092] Custom 305 retry allows you to customize the input of any network address and port (ip:port), network domain name (DNS), configuration keywords, and other task execution parameters to execute test cases. After completion, the results will be displayed in a pop-up window, along with verification point information, a log button (for viewing log information), and a response information button (for viewing server response information).

[0093] The 306 online retry option is used to re-execute test cases on an online instance with the original task execution parameters. After execution, a pop-up window displays the results, checkpoint information, a log button (for viewing log information), and a response information button (for viewing server response information). Online instances are typically either already in operation or instances previously used exclusively for testing.

[0094] In this embodiment of the disclosure, upon receiving the execution command for the retry test case, the retry test case task is executed in response to the execution command for the retry test case (second test task). The execution parameters of the retry test case task are determined based on the task information and test case information of the already executed test task.

[0095] Here, the retry task can specify the same task execution parameters as the test task, so it will not be described again.

[0096] In this way, test cases can be easily retried and the latest test case execution results can be obtained on the test report display page to further determine the reason for the test case failure, without having to set up an additional test environment to retry failed test cases.

[0097] If the reason for a test case failure can be determined by the verification point information, server response information, and log information in the test case information, then the failed test case can be marked with a 307 annotation to record its failure reason.

[0098] In this embodiment, after clicking on label 307, users can select a failure reason category from a drop-down list, such as "Feature offline," "Data change," "Feature change," "Environmental issues," "Case code bug," or "Other." Users can then manually fill in the specific failure reason and more detailed information. In this way, once a tester analyzes the failure reason of a test case, it can be recorded and shared with other testers or developers, providing a historical record for tracing the source of test case failures.

[0099] In this embodiment of the disclosure, in addition to the program code and execution results of the test cases, more detailed test case information is provided, such as log information, checkpoint information, and server response information. This provides more evidence for analyzing the reasons for test case failures.

[0100] The above information can be obtained by acquiring the log files generated by executing test cases; extracting test verification point information from the log files; and deserializing the serialized response information to obtain the deserialized response information.

[0101] If the information recorded during test case execution is insufficient to determine the cause of failure, the failed test cases can be retried in real time, or even retried by immediately modifying the test code, to further verify the analysis results of the cause of failure.

[0102] Once the cause of failure is identified, the test case can be immediately annotated. In this way, a single interface can integrate all the basic functions and steps of test case failure analysis, including report viewing, test case retrying, and failure cause annotation. The interface is simple to use, greatly facilitating the analysis of test case failures by testers or developers.

[0103] In this way, even non-testers can try to analyze and retry test cases, which is more conducive to quickly achieving product quality standards and shortening the launch cycle.

[0104] Figure 4 Another embodiment of this disclosure is shown.

[0105] In this embodiment of the disclosure, the following method is also performed: obtaining task information of at least one executed test task and test case information of test cases executed by each test task; and performing classification statistics based on the task information of at least one executed test task and test case information of test cases executed by each test task to obtain classification statistics information.

[0106] In this disclosure embodiment, the following are also provided: Figure 4 The statistics page 40 shown includes: statistics page 401 for test source 1, statistics page 402 for test source 2, etc., which are divided according to the test source.

[0107] The statistics page for each test source is further divided and grouped according to business, task, and test case dimensions.

[0108] Taking Test Source 1 Statistics Page 401 as an example, the Test Source 1 Statistics Page will display Business Dimension Statistics Page 4011, Task Dimension Statistics Page 40111, and Test Case Dimension Statistics Page 40112.

[0109] On the Test Source 1 statistics page (401), the first row displays the total number of tasks run for that test source, the number of tasks passed, and the number of tasks that failed, accompanied by a pie chart and other visual representations. Next to the pie chart, a list of the three business modules with the most failed test cases is displayed, sorted by failure percentage. For example, as shown in Table 1:

[0110]

[0111] Table 1

[0112] It also displays a "All Failed Tasks" link, which redirects to the failed tasks page, allowing users to flip through the pages to view all failed tasks.

[0113] In addition, under each business module, statistics can be further performed according to the task dimension, displaying the three test tasks with the most failed test cases, and providing a link to the details page of each test task to view detailed information about each failed test task.

[0114] Similarly, within each business module, statistics can be further analyzed by test case dimension, displaying the 5 test cases with the most failures and providing links to the details pages of each failed test case to view detailed information. In the test case details, the "Historical Marking Records" button can be clicked to display the historical reasons for the marked failures of that test case.

[0115] When displaying categorized statistical information, implementers can write their own front-end code and use visualization tools to create diverse displays such as pie charts and trend charts.

[0116] In this embodiment of the disclosure, by classifying and statistically analyzing test tasks and test case information, product quality can be observed more intuitively, weaknesses of the product can be identified, and the most urgent problems to be fixed or solved can be found, providing a basis for decision-making to further improve product quality efficiently and quickly.

[0117] To better implement the automated testing method of this disclosure, in another embodiment of this disclosure, it is also achieved by using, as Figure 5 The test framework shown has optimized the organization structure and calling framework of test cases.

[0118] like Figure 5 As shown, the test framework includes:

[0119] The 501 public protocol encapsulation is used to encapsulate protocol code calls;

[0120] For example, libraries related to common protocols such as http, nshead, mcpack, pb, and rpc can be placed in a public lib library to provide a unified calling interface for test cases.

[0121] The common tool wrapper 502 is used to encapsulate the code calls of the tool;

[0122] For example, common tools such as BNS resolution tools, case execution tools, assertion tools, logging tools, and report generation tools are encapsulated, and the relevant implementation libraries are placed in a public lib library, providing a unified calling interface for test cases to use;

[0123] The log generation encapsulation 503 is used to encapsulate the code calls for log generation;

[0124] The task execution encapsulation 504 is used to encapsulate the code call for test task execution, use task execution parameters to replace the test environment variables specified in the configuration file, and support the individual and parallel execution of test cases;

[0125] The test environment variables include the host address of the test host, the test source, and the test platform.

[0126] Furthermore, in another embodiment of this disclosure, task execution encapsulation can also support: executing a test case independently; using keyword configuration instead of ip:port; batch running of cases supports passing task_id; logging by timestamp by default if task_id is not passed; adding verification point display to the test report (for platform service calls); concurrent execution of test cases with customizable QPS to avoid service degradation due to excessive concurrency.

[0127] In this way, by encapsulating task execution, the implementation of test code for each business module can be greatly simplified, such as business module 1 and business module 2. Each business module only needs to focus on the business logic and test logic of its respective business, without having to consider the code related to the execution of test tasks.

[0128] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] According to embodiments of this disclosure, this disclosure also provides an apparatus for automated testing, such as... Figure 6 As shown, the device 60 includes: a task information acquisition module 601, used to acquire task information of the executed first test task, the task information including task identifier, task execution parameters and / or task execution result; a test case information acquisition module 602, used to acquire test case information of the test cases executed by the first test task, the test case information including the program code and execution result of the test cases; and a test information display module 603, used to display task information and test case information.

[0130] According to an embodiment of this disclosure, the first test task includes a basic task and a test task to be compared. The device 60 further includes: a test case information comparison module, used to compare the test case information of the basic task and the test task to be compared to obtain the difference information of the test cases; and a difference information comparison module, used to display the difference information.

[0131] According to one embodiment of this disclosure, the device 60 further includes: a test task execution module, configured to execute a first test task according to specified task execution parameters, wherein the task execution parameters include a test platform and execution parameters of the test platform.

[0132] According to one embodiment of this disclosure, the device 60 further includes: a plurality of task information acquisition modules, used to acquire task information of at least one executed test task and test case information of test cases executed by each test task; and a classification and statistics module, used to perform classification and statistics based on the task information of at least one executed test task and the test case information of test cases executed by each test task to obtain classification and statistics information.

[0133] According to one embodiment of this disclosure, the device 60 further includes: a second test task execution module, configured to execute the second test task in response to an execution command for the second test task, wherein the execution parameters of the second test task are determined based on task information and test case information.

[0134] According to one embodiment of this disclosure, the device 60 further includes: receiving modifications to the program code; and submitting the modifications.

[0135] According to one embodiment of this disclosure, the device 60 further includes: receiving a failure type input by a user for a test case, and adding the failure type to the test case information obtained by executing the test case in the first test task.

[0136] According to one embodiment of this disclosure, the test case information further includes test verification point information, and the device 60 further includes: acquiring a log file generated by executing test cases; and extracting test verification point information from the log file.

[0137] According to one embodiment of this disclosure, the use case information also includes serialized response information returned by the service, and the device 60 further includes: deserializing the serialized response information to obtain deserialized response information.

[0138] According to embodiments of this disclosure, this disclosure also provides an automated test integration system, such as... Figure 7 As shown, the automated testing integration system includes: a test task execution subsystem 701, used to execute the aforementioned test tasks (first test task or second test task); a test task report display subsystem 702, used to display the task information and test case information obtained after the test task is completed; and a test case source code management subsystem 703, used to read and write the program code in the test case information.

[0139] like Figure 7 As shown, in one embodiment of this disclosure, the test task execution subsystem 701 stores task information in a task database, such as MySQL, when executing a test task. It also stores logs generated during the execution of the test task in a log database, such as Redis.

[0140] After the test task is completed, the test task report display subsystem 702 retrieves task information and test case information from the task library and log library, and retrieves the program code of the test case from the source code management subsystem 703 according to the test case name and test case path, as well as the program code for editing and submitting the test case.

[0141] In another embodiment of this disclosure, the automated integration system further includes: a test task statistics display subsystem 704, used to acquire task information of at least one executed test task and test case information of test cases executed by each test task; to perform classification statistics based on the task information of at least one executed test task and test case information of test cases executed by each test task to obtain classification statistics information, and to display the classification statistics information.

[0142] In this way, the code management of test cases, the execution and analysis of test tasks can be integrated into one, enabling faster and more convenient execution and analysis of test cases, and providing a more intuitive understanding of the quality of the tested product.

[0143] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0144] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0145] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0146] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the automated testing methods of this disclosure. For example, in some embodiments, the automated testing methods of this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the automated testing methods of this disclosure described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the automated testing methods of this disclosure by any other suitable means (e.g., by means of firmware).

[0148] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0152] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0153] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0154] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0155] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for automated testing, comprising: Obtain task information for the first executed test task, including task identifier, task execution parameters, and / or task execution result; Obtain test case information for the test cases executed by the first test task. The test case information includes the program code and execution result of the test cases. The program code is source code obtained from the code library storing the source code or extracted from the task execution result of the first test task after the first test task is executed, for the purpose of analyzing the execution result of the test cases. The program code is used to control the test steps and determine the test result. The program code does not need to be downloaded to the local machine so that the analysts can directly observe the reasons for the failure of the test cases. The task information and the test case information are displayed together on the same interface. The integrated display includes associating the task identifier, task execution parameters, task execution results with the program code and execution results of the test cases to form a complete test context for analyzing the execution results of the test cases. The reasons for test case failure are determined by the test verification point information, server response information, and log information in the test case information; wherein, the test verification point information is extracted from the log file generated by executing the test case; the server response information includes the serialized response information returned by the server, and the response information obtained by deserializing the serialized response information; The program receives modifications to the program code on the same interface and submits the modified program code to the code repository; in response to the retry command for the test case, the program determines the execution parameters of the second test task based on the task information and the test case information, and executes the second test task, wherein the retry command includes original instance retry, custom retry, or online retry.

2. The method according to claim 1, wherein the first test task includes a basic task and a test task to be compared, and the method further includes: By comparing the test case information of the basic task and the test task to be compared, the difference information of the test cases is obtained; Display the difference information.

3. The method according to claim 1, further comprising: The first test task is executed according to the specified task execution parameters, wherein the task execution parameters include the test platform and the execution parameters of the test platform.

4. The method according to claim 3, further comprising: Obtain task information for at least one executed test task, and test case information for each test task. Based on the task information of the at least one executed test task and the test case information of the test cases executed by each test task, classification statistics are obtained by performing classification statistics.

5. The method according to claim 1, further comprising: In response to the execution command of the second test task, the second test task is executed, wherein the execution parameters of the second test task are determined based on the task information and the test case information.

6. The method according to claim 1, further comprising: Receive modifications to the program code; Submit the aforementioned modifications.

7. The method according to claim 1, further comprising: Receive the failure type input by the user for the test case. The failure type is added to the test case information obtained when the test case is executed in the first test task.

8. The method according to claim 1, wherein the test cases run by the first test task are based on a test framework, the test framework comprising: Public protocol encapsulation is used to encapsulate code calls to the protocol; Common tool encapsulation, used to encapsulate the code calls to tools; Log generation encapsulation is used to encapsulate the code calls for log generation; Task execution encapsulation is used to encapsulate the code calls for executing test tasks.

9. An automated testing apparatus, comprising: The task information acquisition module is used to acquire the task information of the first test task that has been executed. The task information includes the task identifier, task execution parameters and / or task execution result. The test case information acquisition module is used to acquire test case information of the test cases executed by the first test task. The test case information includes the program code and execution result of the test case. The program code is the source code obtained from the code library storing the source code or extracted from the task execution result of the first test task after the first test task is completed, for the purpose of analyzing the test case execution result. The program code is used to control the test steps and determine the test result. The program code does not need to be downloaded to the local machine so that the analysts can directly observe the reasons for the failure of the test cases. The test information display module is used to integrate and display the task information and the test case information in the same interface. The integrated display includes associating and displaying the task identifier, task execution parameters, task execution results and the program code and execution results of the test cases to form a complete test context for analyzing the execution results of the test cases. The reasons for test case failure are determined by the test verification point information, server response information, and log information in the test case information; wherein, the test verification point information is extracted from the log file generated by executing the test case; the server response information includes the serialized response information returned by the server, and the response information obtained by deserializing the serialized response information; The program receives modifications to the program code on the same interface and submits the modified program code to the code repository; in response to the retry command for the test case, the program determines the execution parameters of the second test task based on the task information and the test case information, and executes the second test task, wherein the retry command includes original instance retry, custom retry, or online retry.

10. The apparatus according to claim 9, wherein the first test task includes a basic task and a test task to be compared, and the apparatus further includes: The test case information comparison module is used to compare the test case information of the basic task and the test task to be compared, and obtain the difference information of the test cases. The difference information comparison module is used to display the difference information.

11. The apparatus according to claim 9, further comprising: The test task execution module is used to execute the first test task according to the specified task execution parameters, wherein the task execution parameters include the test platform and the execution parameters of the test platform.

12. The apparatus of claim 11, further comprising: Multiple task information acquisition modules are used to acquire task information of at least one executed test task, as well as test case information of the test cases executed by each test task; The classification and statistics module is used to perform classification and statistics based on the task information of the at least one executed test task and the test case information of the test cases executed by each test task to obtain classification and statistics information.

13. The apparatus according to claim 9, further comprising: The second test task execution module is used to execute the second test task in response to the execution command of the second test task, wherein the execution parameters of the second test task are determined based on the task information and the test case information.

14. An automated test integration system, comprising: A test task execution subsystem is used to execute the first test task or the second test task as described in any one of claims 1-8; The test task report display subsystem is used to display the task information and test case information as described in any one of claims 1-8; A source code management subsystem for test cases is used to obtain program code from the test case information as described in any one of claims 1-8.

15. The test integration system according to claim 14, further comprising: The test task statistics display subsystem is used to display the classification statistics information as described in claim 4.

16. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

17. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

18. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images