Method and device for implementing automated testing in cloud network system and computer-readable storage medium

Abstract

The present invention discloses a method, device, and computer-readable storage medium for implementing automated testing in a cloud network system. The method for implementing automated testing in a cloud network system includes: obtaining a test case and its data from an orchestration engine; analyzing the obtained test case to obtain characteristic information of the test case; determining a specific test tool that matches the test case from multiple test tools based on the obtained characteristic information; converting the test case data into test input data that conforms to the data specification of the specific test tool; and calling the specific test tool to execute the test case using the test input data.

Description

Technical Field

[0001] The present invention generally relates to the field of software testing, and more specifically, to a method, device, and computer-readable storage medium for implementing automated testing in a cloud network system. Background Art

[0002] As network cloudification continues to advance, testing requirements for cloud-network systems in various scenarios are becoming increasingly frequent, complex, and varied. A wide variety of orchestration engine frameworks and testing tools are available for automated testing platforms. Typically, a single test task involves multiple test objects and contains numerous test cases, each executed by a different test tool. Within the same test task, test cases from different test tools often require test data exchange. However, significant differences exist between the data specifications of the test tools and those of the orchestration engines, creating practical barriers to data communication.

[0003] Therefore, there is a need in the prior art for adaptively connecting test tools and transferring test parameters across test tools. Summary of the Invention

[0004] The inventors have proposed a method for realizing adaptive docking of an orchestration engine with a test tool, which enables an automated testing platform based on an orchestration engine to face the challenges brought by the integration of multiple use cases, multiple objects, and multiple tools, and has the ability to adaptively dock test tools on demand and transfer test parameters across test tools.

[0005] According to one aspect of the present invention, the corresponding test tool can be connected to the orchestration engine according to the characteristic information of the test case, and the test case can be executed using the adapted test tool, thereby solving the problem that different test cases require different test tools to test;

[0006] According to another aspect of the present invention, a mechanism for converting data specifications of multiple testing tools into data specifications of an orchestration engine is provided to solve the data interaction problem between different testing tools in the same testing task.

[0007] In order to achieve one purpose of the present invention, according to one aspect of the present invention, a method for implementing automated testing in a cloud network system is provided, including: obtaining test cases and their data from an orchestration engine; analyzing the obtained test cases to obtain characteristic information of the test cases; based on the obtained characteristic information, determining a specific test tool that matches the test case from multiple test tools; converting the test case data into test input data that conforms to the data specification of the specific test tool; and calling the specific test tool to execute the test case using the test input data.

[0008] In order to achieve another object of the present invention, according to another aspect of the present invention, a system for implementing automated testing in a cloud network system is provided, including: a parsing module, used to obtain test cases and their data from an orchestration engine, and analyze the obtained test cases to obtain characteristic information of the test cases; a conversion module, used to convert the data of the test cases into test input data that conforms to the data specification of the specific test tool; and a scheduling module, used to determine a specific test tool that matches the test case from multiple test tools based on the characteristic information obtained from the parsing module, and call the specific test tool to execute the test case using the test input data received from the conversion module.

[0009] In order to achieve the above-mentioned object, according to another aspect of the present invention, a computing device is provided, comprising: a processor; and a memory, which is coupled to the processor and has computer-readable instructions stored thereon, and when executed by the processor, the computer-readable instructions enable the processor to perform the steps of the above-mentioned method.

[0010] In order to achieve the above-mentioned object, according to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned automated network defense method are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that, in this specification and the drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted.

[0012] Figure 1 is a schematic diagram illustrating a system for implementing automated testing in a cloud network system according to an embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram illustrating a process of implementing automated testing in a cloud network system according to an embodiment of the present disclosure; and

[0014] Figure 3 3 is a schematic diagram showing a computing device for implementing automated testing in a cloud network system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] The following detailed description of exemplary embodiments refers to the accompanying drawings. The same reference numerals in different figures identify the same or similar elements. In addition, the drawings are not necessarily drawn to scale. Moreover, the following detailed description does not limit the present invention. Instead, the scope of the present invention is defined by the appended claims.

[0016] Reference throughout this specification to "one embodiment," "an embodiment," or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter. Thus, the appearances of the phrases "in one embodiment," "in an embodiment," or "in some embodiments" in various places throughout this specification are not necessarily referring to the same embodiment(s). Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0017] An orchestration engine is a workflow engine used to orchestrate test tasks. It monitors workflow execution, including the order and conditions of workflow execution, and outputs workflow logs and feedback. However, it does not execute tasks within the workflow. The orchestration engine and testing tools have a one-to-many relationship, as a workflow generated by the orchestration engine may contain many different subtasks, and different subtasks may require integration with different testing tools.

[0018] The task processing integration module (for example, Job worker) is a concept in an open source orchestration engine. The task processing integration module is a module specifically used to process subtasks in the workflow. However, the input and output specifications of each test tool are different. The two subtasks before and after a workflow may need to interact with data, which requires the task processing integration module to be able to convert the direct data specifications of many test tools with the data specifications of a few test tools (for example, zeebe). However, the existing technology cannot achieve this conversion. The inventors recognized the problems in the existing technology and proposed the present invention. The present invention is proposed to solve the demand for adaptive docking of test tools and the transfer of test parameters across test tools. The present invention can be integrated into the task processing integration module.

[0019] This invention proposes a technical solution for implementing automated testing in a cloud network system. Specifically, this technical solution includes: obtaining test cases and their data from an orchestration engine; analyzing the obtained test cases to obtain test case feature information; determining a specific test tool from multiple test tools that matches the test case based on the obtained feature information; converting the test case data into test input data that conforms to the data specifications of the specific test tool; and invoking the specific test tool to execute the test case using the test input data.

[0020] Figure 1 This is a schematic diagram illustrating a system for implementing automated testing in a cloud network system according to an embodiment of the present disclosure. The system according to this embodiment includes an orchestration engine, an automated testing device, and a test framework set. Orchestration engine maintenance and regulation:

[0021] 1) Test case task queue: The test case task queue of the orchestration engine is regulated and maintained by the orchestration engine;

[0022] 2) Test case data set: The test case data set of the orchestration engine. The data complies with the data specifications of the orchestration engine.

[0023] The test framework collection includes a collection of test tools that have been connected to the system and can be expanded as needed in the future.

[0024] The automated testing device according to the present invention is integrated into the task processing integrated module and the internal logic is executed therein. The task processing integrated module is a general term for the test case processing system. For example, the present invention interfaces with the test tool in the form of a task processing integrated module.

[0025] The automated testing apparatus includes a parsing module, a scheduling module, and a conversion module. The parsing module is configured to obtain test cases and their data from an orchestration engine and analyze the obtained test cases to obtain characteristic information of the test cases. The conversion module is configured to convert the test case data into test input data that conforms to the data specifications of a specific test tool. The scheduling module is configured to determine a specific test tool that matches the test case from multiple test tools based on the characteristic information obtained from the parsing module, and to invoke the specific test tool to execute the test case using the test input data received from the conversion module.

[0026] As shown in the figure, in operation ①, a test case is retrieved from the orchestration engine's test case task queue. In operation ②, relevant data for the test case is retrieved from the orchestration engine's test case dataset. The parsing module analyzes the test case-related information, and in operation ③, the analysis results are sent to the scheduling module. In operation ④, the corresponding test task for the test case is executed based on the analysis results, and in operation ⑤, the test results are fed back to the orchestration engine.

[0027] As shown in the figure, the parsing module obtains test case data from the test case dataset of the orchestration engine, extracts relevant information about the test case, classifies the test case based on the information, performs one-to-one matching between the test case and the test tool, and then analyzes the test environment required by the test case in its adapted test tool, and passes the parsing results to the scheduling module.

[0028] The scheduling module establishes a session between the test case and its adapted test tool based on the parsing results:

[0029] 1. The test case data is converted by the conversion module as the test input data of the test case;

[0030] 2. Initialize the required test environment for the test case in the adapted test tool;

[0031] 3. Adapt to the rules of the test tool and start the test work corresponding to the test case;

[0032] 4. Obtain the test results from the test tool, transform them through the transformation module, and then feed them back to the test case dataset of the orchestration engine;

[0033] The conversion module provides a conversion mechanism for test case data:

[0034] 1. Convert the test case data obtained by the orchestration engine into data that conforms to the test tool specifications, and then use it as the test input data of the test case in the adapted test tool;

[0035] 2. Convert the test result data obtained by the test tool into data that conforms to the specifications of the orchestration engine. Then, the orchestration engine can integrate the test results of the previous test case into the test input data of the next test case, realizing data interaction between test cases of different test tools.

[0036] For example, a carrier built an automated testing platform for cloud network system testing. This testing platform is built on the Zeebe orchestration engine, with the Zeebe broker responsible for managing each test task and deploying each test case within the task. However, Zeebe does not provide technical support for interoperability with testing tools. Cloud network system testing requirements are complex, often requiring test case A to interface with Yardstick, test case B to interface with Functest, and the test results of Test Case A to serve as input data for Test Case B within the same test task. Yardstick and Functest differ in their methods and processes for invoking test services, and the data specifications of Zeebe, Yardstick, and Functest also differ significantly.

[0037] To address the above issues, a mechanism according to the present invention is integrated into the Job Worker responsible for processing each test case, which allows for adaptive docking between Yardstick and Functest, as well as a mechanism according to the present invention that allows for the data of Zeebe, Yardstick, and Functest to be freely converted according to specifications. This ensures that the Job Worker can dock with the corresponding test tool (Yardstick or Functest) based on the characteristics of the test case. It also enables the conversion of test result data of Test Case A that conforms to the Yardstick data specification -> data that conforms to the Zeebe data specification -> test input data of Test Case B that conforms to the Functest data specification, ensuring data interaction between Test Case A and Test Case B.

[0038] Figure 2A schematic diagram of a process 200 for implementing automated testing in a cloud network system according to an embodiment of the present disclosure is shown. At block 201 , the process 200 for implementing automated testing in a cloud network system according to some embodiments of the present disclosure begins.

[0039] At block 203 , the test case and its data are obtained from the orchestration engine.

[0040] At block 205 , the acquired test cases are analyzed to obtain feature information of the test cases.

[0041] At block 207 , a specific test tool matching the test case is determined from a plurality of test tools based on the acquired feature information.

[0042] At block 209 , the test case data is converted into test input data that complies with the data specification of the specific test tool.

[0043] At block 211 , the specific test tool is called to execute the test case using the test input data.

[0044] At block 213 , the process 200 of implementing automated testing in a cloud network system ends.

[0045] Figure 3 is a block diagram illustrating a system 300 for implementing automated testing in a cloud network system. In various embodiments, the system 300 performs Figure 2 In some embodiments, the system 300 provides a processing unit for Figure 2 The system 300 may further include instructions for one or more of the methods described in the method, so that the processing unit performs the method or a part of the method based on the instructions provided by the system 300.

[0046] System 300 includes memory 325 , storage 330 , interconnect (eg, BUS) 320 , one or more CPUs 305 (also referred to herein as processors 305 ), I / O device interface 310 , I / O device 312 , and network interface 315 .

[0047] Each CPU 305 retrieves and executes programming instructions stored in memory 325 or storage 330. Interconnect 320 is used to move data, such as programming instructions, between CPU 305, I / O device interface 310, storage 330, network interface 315, and memory 325. Interconnect 320 can be implemented using one or more buses. In various embodiments, CPU 305 can be a single CPU, multiple CPUs, or a single CPU with multiple processing cores. Memory 325 is typically included to represent random access memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), or flash memory). Storage 330 is typically included to represent non-volatile memory, such as a hard drive, solid-state device (SSD), removable memory card, optical storage device, or flash memory device.

[0048] In some embodiments, memory 325 stores instructions 360, and storage 330 stores database 332 and data source 336. However, in various implementations, instructions 360, database 332, and data source 336 are stored partially in memory 325 and partially in storage 330, or they are all stored in memory 325 or all stored in storage 330, or they are accessed over network 350 via network interface 315.

[0049] Data sources 336 may include various data from a network or other sources.

[0050] Instructions 360 are processor executable instructions, including fetch instructions 362, analyze instructions 364, determine instructions 366, convert instructions 368, and call instructions 370. These instructions 362-370 may be configured to execute instructions previously executed in Figure 2 The method discussed in .

[0051] In various embodiments, I / O device 312 includes an interface capable of presenting information and receiving input. For example, I / O device 312 can present information (e.g., redundant and abnormal samples) to a user interacting with system 300 and receive input (e.g., marking actions) from the user.

[0052] System 300 is connected to network 350 via network interface 315 .

[0053] Figure 3 is intended to represent the major components of the example system 300 according to an embodiment of the present disclosure. However, in some embodiments, the various components may have more Figure 3 greater or lesser complexity than shown, and may exist in Figure 3 Components shown or components in addition to those shown. In addition, in some embodiments, Figure 3The various components shown in the Figure 3 More, fewer, or different features than those shown in .

[0054] Compared to the prior art, the present invention has significant advantages and effects. The present invention uses a Jobworker to obtain test cases and test case data from an orchestration engine, parse test case-related information, adaptively connect the test case to a test tool, convert the test case data to conform to the test tool's data specifications, use it as test input data, initialize the test environment required by the test case in the test tool, call the test tool's test service, obtain test results, convert them into data that conforms to the orchestration engine's data specifications, and wait for the Jobworker to obtain the next test case.

[0055] Compared with a test platform system that only uses a specific test tool to execute test cases, the present invention has the following advantages and effects:

[0056] The patented method is not limited to a specific orchestration engine framework or specific test tool. It has the ability to adaptively connect test cases to test tools in complex application scenarios with multiple use cases, multiple test tools, and multiple frameworks, and has excellent flexibility and compatibility.

[0057] - Provides a mechanism for converting orchestration engine data specifications and test tool data specifications to ensure data interaction between test cases of different test tools and provide good data communication capabilities;

[0058] - In the future, the system's compatibility will be enhanced to enable one test case to connect to multiple test tools.

[0059] As will be appreciated based on the foregoing description, the above-described embodiments of the present disclosure can be implemented using computer programming or engineering techniques, which include computer software, firmware, hardware, or any combination or subset thereof, wherein the technical effect is to establish and operate an application network based on a file system. Any such obtained program with computer readable code means can be implemented or provided in one or more computer readable media, thereby making a computer program product (i.e., product) according to the embodiments discussed in the present disclosure. The computer readable medium can be, for example, but not limited to, a fixed (hard) drive, a floppy disk, an optical disk, a magnetic tape, a semiconductor memory such as a read-only memory (ROM), and / or any transmission / reception media (such as the Internet or other communication networks or links). The product containing computer code can be manufactured and / or used by executing the code directly from a medium, by copying the code from one medium to another medium, or by sending the code over a network.

[0060] These computer programs (also referred to as programs, software, software applications, "applications" or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., a disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. However, "machine-readable medium" and "computer-readable medium" do not include transient signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0061] Although the present disclosure has been described in conjunction with specific exemplary embodiments, it should be understood that various changes, substitutions and alterations apparent to those skilled in the art may be made to the disclosed embodiments without departing from the spirit and scope of the disclosure as set forth in the appended claims.

Claims

1. A method for implementing automated testing in a cloud network system, comprising: Get test cases and their data from the orchestration engine; Analyze the acquired test cases to obtain characteristic information of the test cases; Based on the acquired feature information, determine a specific test tool that matches the test case from multiple test tools; Analyze the test environment required by the test case in the specific test tool; Converting the test case data into test input data that conforms to the data specifications of the specific test tool; Initializing a required test environment for the test case in the specific test tool; as well as Invoking the specific test tool to execute the test case using the test input data; The test result data obtained by the specific test tool is converted into data that complies with the specifications of the orchestration engine, and the test result of the previous test case is integrated into the test input data of the next test case by the orchestration engine.

2. The method of claim 1, wherein the method is implemented in a task processing integration module, and the task processing integration module obtains the test cases and their data from an orchestration engine.

3. The method as claimed in claim 2, wherein the task processing integration module parses the characteristic information of the test task, and adaptively connects different test tools for tasks with different test requirements according to the characteristic information to be compatible with the docking protocols of multiple test tools.

4. The method of claim 2, wherein the task processing integration module implements data and protocol conversion between different testing tools, and implements data specification conversion between multiple testing tools and orchestration engines.

5. A system for implementing automated testing in a cloud network system, comprising: A parsing module is used to obtain test cases and their data from the orchestration engine, analyze the obtained test cases to obtain characteristic information of the test cases, and analyze the test environment required by the test cases in a specific test tool; A conversion module, configured to convert the test case data into test input data that conforms to the data specification of the specific test tool; as well as a scheduling module, configured to determine, from a plurality of test tools, a specific test tool that matches the test case based on the feature information obtained from the parsing module, initialize a required test environment for the test case in the specific test tool, and call the specific test tool to execute the test case using the test input data received from the conversion module; The test result data obtained by the specific test tool is converted into data that complies with the specifications of the orchestration engine, and the test result of the previous test case is integrated into the test input data of the next test case by the orchestration engine. 6 . The system of claim 5 , wherein the system is implemented in a task processing integration module, and the task processing integration module obtains the test cases and their data from an orchestration engine.

7. The system as claimed in claim 6, wherein the task processing integration module parses the characteristic information of the test task, and adaptively connects different test tools for tasks with different test requirements according to the characteristic information to be compatible with the docking protocols of multiple test tools.

8. The system of claim 6, wherein the task processing integration module implements data and protocol conversion between different testing tools, and implements data specification conversion between multiple testing tools and orchestration engines.

9. A computing device comprising: processor; and A memory coupled to the processor and having computer-readable instructions stored thereon, which, when executed by the processor, cause the processor to perform the method steps of any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method steps according to any one of claims 1 to 4 when executed by a processor.

Images