Test method, device and electronic equipment

Abstract

The present application provides a test method, device and electronic device, which are applicable to the field of computer technology. The method includes: a test execution device obtains test requirements, and obtains an automated test case set according to the test requirements; the automated test case set includes several automated test cases, and the business fixture method is called in the automated test case; wherein the test execution device is deployed with a target development tool, and the target development tool is used to develop automated test cases; the test execution device executes the automated test cases in the automated test case set according to a preset order until all the automated test cases in the automated test case set are executed to obtain the test results. In an embodiment of the present application, a target development tool is provided to develop the automated test cases required for the test, and the target development tool can make the development of automated test cases more convenient, thereby improving the overall efficiency of the test.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular to testing methods, devices and electronic equipment. Background Art

[0002] With the development of modern network information technology, various servers are widely used, and the requirements for server security, stability, and efficiency are getting higher and higher. Server firmware, such as the server's basic input and output system (BIOS) or baseboard management controller (BMC), as an important part of the server, plays a vital role in the normal operation of the server. In order to ensure the performance of the server, testing the server's BIOS or BMC firmware is an essential task.

[0003] In the past, the way to test the firmware such as BIOS or BMC of the server was mostly manual testing. Now, more and more test technicians tend to use automated testing. In the automated testing method, it is usually necessary to complete the development of automated test cases, and then complete the test by executing the automated test cases. In the development of automated test cases, it is necessary to convert manual test cases into a script language executed by the machine. However, the current development process of automated test cases is complicated and time-consuming, which leads to low efficiency of automated testing methods for software. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a testing method, a device and an electronic device to solve the problem.

[0005] The first aspect of an embodiment of the present application provides a testing method, including: a test execution device obtains test requirements, and obtains an automated test case set according to the test requirements; the automated test case set includes a number of automated test cases, and the business fixture method is called in the automated test case; wherein the test execution device is deployed with a target development tool, and the target development tool is used to develop automated test cases.

[0006] The test execution device executes the automated test cases in the automated test case set according to a preset sequence until all the automated test cases in the automated test case set are executed to obtain a test result.

[0007] In a first possible implementation manner of the first aspect, executing an automated test case in an automated test case set includes:

[0008] The test execution device executes the setup phase of the business fixture method called by the automated test case and establishes a connection with the device under test.

[0009] The test execution device executes the main part of the automated test case, which includes: the pre-operation of the automated test case, the main test operation and the post-operation.

[0010] The test execution equipment executes the teardown phase of the business fixture method called by the automated test case.

[0011] In a second possible implementation manner of the first aspect, the target development tool includes a connection layer, a business logic layer, and a use case layer.

[0012] The connection layer is used to establish a communication connection channel between the test execution device and the device under test. The implementation of the connection layer's functions depends on the connection library.

[0013] The business logic layer is used to implement specific business functions, and the implementation of the business logic layer functions depends on the business method library.

[0014] The use case layer is used to combine multiple independent business functions for testing to determine whether these business functions can run together. The implementation of the use case layer functions depends on the use case library.

[0015] In a third possible implementation manner of the first aspect, the business method library includes business fixture methods corresponding to IPMI, SNMP, SSH, Redfish, serial port, socket, HTTP protocol and file operations under the operating system, respectively.

[0016] Based on the third possible implementation of the first aspect, in a fourth possible implementation of the first aspect, the connection library includes a connection abstract base class and a connection entity class; the connection abstract base class is a declaration for defining methods related to communication connections, and the connection entity class is a class for specifically implementing the methods defined by the connection abstract base class.

[0017] The business method library contains business proxy classes and business entity classes. The business proxy class is a class used to provide a unified external interface for each specific business entity class, and the business entity class is a class that specifically implements various business functions.

[0018] In a fifth possible implementation manner of the first aspect, calling a business fixture method in an automated test case includes:

[0019] The test execution device obtains the test requirements, determines the business fixture methods that need to be called by the automated test cases according to the test requirements, and adds the business fixture methods that need to be called by the automated test cases to the business fixture method collection.

[0020] Determine whether the business fixture method set covers all test requirements.

[0021] When the business fixture method set covers all test requirements, the business fixture methods included in the business fixture method set that has covered all test requirements are passed into the automated test case.

[0022] Based on the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, the method further includes:

[0023] When the business fixture method set does not cover all test requirements, the test execution device generates business fixture methods corresponding to the uncovered test requirements, and adds the business fixture methods corresponding to the uncovered test requirements to the business fixture method set to obtain an updated business fixture method set; the execution is repeated until the updated business fixture method set covers all test requirements.

[0024] Through the embodiment of the present application, when developing automated test cases for test requirements, the automated test cases can call the required business fixture methods. Since the business fixture methods are called in the automated test cases, and the same business fixture method can be called in multiple test cases, it is avoided to repeatedly write similar codes, thus reducing the time for automated test case development, thereby improving the efficiency of testing.

[0025] A second aspect of an embodiment of the present application provides a testing device, including:

[0026] The first processing module is used to obtain test requirements and obtain an automated test case set based on the test requirements; the automated test case set includes several automated test cases, and the business fixture method is called in the automated test case; wherein the test device is deployed with a target development tool, and the target development tool is used to develop automated test cases.

[0027] The second processing module is used to execute the automated test cases in the automated test case set according to a preset order until all the automated test cases in the automated test case set are executed to obtain a test result.

[0028] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the electronic device implements the steps of the test method described in any one of the first aspects above.

[0029] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, comprising: storing a computer program, wherein when the computer program is executed by a processor, the electronic device implements the steps of the test method described in any one of the first aspects above.

[0030] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 It is a schematic diagram of an application scenario of a test method provided in an embodiment of the present application;

[0033] Figure 2 It is a flowchart of a testing method provided in an embodiment of the present application;

[0034] Figure 3 is a schematic diagram of the structure of a target development tool provided by an embodiment of the present application;

[0035] Figure 4 It is a flowchart of determining a business fixture method that needs to be called for an automated test case provided by an embodiment of the present application;

[0036] Figure 5 is a schematic diagram of the structure of the test device provided in the embodiment of the present application;

[0037] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] With the rapid development of Internet technology and the further deepening of computer applications, the application scope of various servers is becoming wider and wider, and it is becoming more and more important to ensure the security, stability and efficiency of servers. In order to ensure the performance of the server, it has become an indispensable task to test the firmware of the server, such as BIOS or BMC, which is an important part of the server. In the past, the way to test the BMC or BIOS of the server was usually manual testing, but now more automated testing is used. In the automated testing method, it is necessary to develop automated test cases according to the test requirements, and then complete the test by executing the automated test cases. Automated test cases refer to specific test cases referenced in the automated test execution. Automated test case development requires converting manual test cases into scripting languages ​​that can be executed by machines.

[0040] Among them, the server's BMC is the basic core functional subsystem of the server, which can collect various information on the server and report these messages to the upper-level network management software in different ways. For example, BMC provides intelligent platform management interface (IPMI) for upper-level network management software to query, or BMC reports the collected information to the upper-level network management software in a timely manner through a variety of industry-wide specifications such as simple network management protocol (SNMP), simple mail transfer protocol (SMTP), and Redfish protocol, so that operation and maintenance personnel can handle it in a timely manner. BIOS is a firmware that runs when the computer is turned on. Its main function is to control the basic programs after the computer is started, including soft / hard disk drives, keyboard settings, memory, and related devices. In some scenarios, such as on x86 servers, BIOS needs to collaborate with various hardware and chips, including BMC, and BIOS can use IPMI to communicate with BMC.

[0041] It can be seen that when BMC or BIOS is running, it is necessary to contact with various communication protocols or interfaces. Therefore, when testing the BMC or BIOS of the server, there is a test requirement to call different communication protocols or different interfaces at the same time. If the automated test method is used, when developing automated test cases, it is necessary to make each developed automated test case able to cover these test requirements. The development of automated test cases is mainly completed by test technicians, but the current development process of automated test cases is relatively complicated and time-consuming, which leads to low efficiency when performing automated tests on firmware such as BMC or BIOS of the server. This is because in the prior art, when developing automated test cases, test technicians usually use various existing tools for testing. However, on the one hand, these tools are often used for specific communication protocols. For example, pyipmi is a Python-based IPMI tool library for communicating and managing servers that support the IPMI protocol; and paramiko is a Python-based SSH library for implementing the functions of the SSH protocol. On the other hand, the initialization and use methods of these tools are not unified, which will cause users to spend more time initializing or configuring these tools. Therefore, the operation of the test technicians in developing automated test cases will be more complicated and time-consuming, thereby reducing the efficiency of automated testing of firmware such as the server's BMC or BIOS.

[0042] Taking the above situation into consideration, an embodiment of the present application provides a testing method, including: a test execution device obtains test requirements, and obtains an automated test case set according to the test requirements; the automated test case set includes a number of automated test cases, and the automated test cases call business fixture methods; wherein the test execution device is deployed with a target development tool, and the target development tool is used to develop automated test cases; the test execution device executes the automated test cases in the automated test case set according to a preset order until all the automated test cases in the automated test case set are executed, and the test results are obtained.

[0043] In the embodiment of the present application, a target development tool is provided to develop the automated test cases required for testing, and the target development tool can help test technicians to realize the hierarchical development of test case codes, business function codes, and communication connection codes. And the use of different methods is standardized in the target development tool. Therefore, based on the embodiment of the present application, the development of automated test cases can be made more convenient, thereby improving the overall efficiency of testing firmware such as the BMC or BIOS of the server.

[0044] It should be noted that the test method provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, personal computers, and industrial computers. In this case, the electronic device is the execution subject of the test method provided in the embodiment of the present application, which can be referred to as a test execution device. The embodiment of the present application does not impose any restrictions on the specific type of electronic device. In addition, in addition to being applicable to the test scenario of firmware such as BMC or BIOS of the server, the method provided in the embodiment of the present application can also be applied to other test scenarios involving the need to use different communication protocols or different interfaces and the need to develop automated test cases, for example, automated testing of the server's operating system (OS) or application software running on the operating system.

[0045] The following describes some concepts involved in the embodiments of the present application:

[0046] Test execution device: a device that executes the test method provided in the embodiments of the present application.

[0047] Device under test: A device that has firmware or software to be tested. The number of devices under test can be one or more.

[0048] Class: In object-oriented programming, a "class" is an abstract data type used to describe a collection of objects with the same properties and behaviors. An "object" is an instance of a "class" and has the properties and behaviors defined by the "class".

[0049] Fixture function: A fixture function is a function provided by the pytest testing framework that can be used to perform specific operations during testing, such as providing pre- and post-test settings. The fixture function has the following features: the fixture function can be passed as a parameter to a test case; multiple fixture functions can be passed to the same test case; fixture functions can call each other; fixture functions can implement setup and teardown functions.

[0050] Business fixture method: also known as business fixture function, refers to a fixture function that encapsulates code or data that implements specific business logic. Business fixture methods include setup phase and teardown phase. Business fixture methods can be called by automated test cases.

[0051] Automated test cases: Automated test cases are also test cases required during testing, but automated test cases are executed in an automated manner.

[0052] Target development tools: Tools used to develop automated test cases.

[0053] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0054] Please refer to Figure 1 , Figure 1 A schematic diagram of an application scenario of the test method of the present application is shown, and the scenario includes a test execution device 101 and a device under test 102. In the embodiment of the present application, the device under test may be a server deployed with a BMC or BIOS to be tested, and the number of the devices under test may be one or more. Figure 1 Only one device under test is shown.

[0055] As an optional embodiment of this application, please refer to Figure 2 , Figure 2 The following is a flow chart of the test method provided in Example 1 of the present application, which is described in detail as follows:

[0056] S200: The test execution device executes a test preparation operation.

[0057] Before conducting a test, the test execution device needs to perform a test preparation operation. The test preparation operation refers to a setup operation that needs to be performed before the test in order to complete the test. The test preparation operation may include an initialization operation and a parameter configuration operation. Among them, the initialization operation refers to initializing the target development tool in the test execution device. The parameter configuration operation refers to configuring parameters related to the test environment, the test execution device, the device under test, etc. Exemplarily, the parameter configuration operation may include configuring the target identifier. The target identifier can be used to indicate which devices under test are used for testing. For example, the target identifier can be an identifier such as the IP address, MAC address or serial number of the device that can distinguish the device.

[0058] S201, the test execution device obtains the test requirements, and obtains an automated test case set according to the test requirements; the automated test case set includes several automated test cases, and the business fixture method is called in the automated test case; wherein, the test execution device is deployed with a target development tool, and the target development tool is used to develop automated test cases.

[0059] The test execution device first obtains the test requirements, and then obtains the automated test case set based on the test requirements. The automated test case set needs to cover these test requirements. The automated test case set includes several automated test cases. Among them, the test execution device is deployed with a target development tool, which is used to develop automated test cases. Therefore, the test execution device completes the development of automated test cases based on the target development tool. In addition, the business fixture method will be called in the automated test case.

[0060] It should be noted that in the embodiments of the present application, a business fixture method refers to a fixture function that encapsulates code or data that implements a specific business function / business logic. A business fixture method can be called by an automated test case, and an automated test case can call multiple business fixture methods, thereby using the business function / business logic of the business fixture method it calls.

[0061] In an optional embodiment, the automated test case can be generated based on a target development tool provided in an embodiment of the present application. Figure 3 , Figure 3 The structure diagram of the target development tool provided by an embodiment of the present application is shown. The target development tool includes a connection layer 301, a business logic layer 302 and a use case layer 303. Correspondingly, the function implementation of the connection layer depends on the connection library, the function implementation of the business logic layer depends on the business logic library, and the function implementation of the use case layer depends on the use case library. The various layers of the target development tool are described in detail as follows:

[0062] The connection layer 301 is mainly responsible for establishing a communication connection channel between the test execution device and the device under test. The implementation of this function of the connection layer depends on the connection library. The connection library provides various connection abstract base classes and connection entity classes, through which different management methods for communication connections can be implemented.

[0063] The connection abstract base class is used to define the declaration of methods related to communication connections. It only provides a unified specification or interface, but the connection abstract base class does not provide specific implementation code. Among them, the declaration of methods related to communication connections means that when writing the code of methods related to communication connections, the way of declaring the method is clearly standardized. The connection abstract base class can provide a unified interface for each specific connection entity class. Since the abstract base class does not provide specific implementation code, when applying the connection abstract base class, the subclass that inherits it needs to implement the methods it defines. Among them, the connection abstract base class includes but is not limited to the following methods: connect() method, disconnect() method, send() method and recv() method. Among them, the connect() method is used to package the connection establishment method of the connection entity class (some connection objects do not need to be executed and can be left blank); the disconnect() method is used to package the connection destruction method of the connection entity class (some connection objects do not need to be executed and can be left blank); the send() method is used to send messages or instructions; the recv() method is used to receive messages or instructions. It can be seen that based on the connection abstract base class, the use of the four basic methods related to communication connection, namely, establishing connection, destroying connection, receiving message or instruction, and sending message or instruction, is standardized and unified. Therefore, if the four basic methods related to communication connection of the connection layer need to be used in the business logic layer later, these four basic methods related to communication connection can be called by calling the connection abstract base class, which reduces the difficulty of code development in the business logic layer.

[0064] The connection entity class refers to a class that specifically implements the methods defined by the connection abstract base class and provides specific implementation code. It can be seen that the connection entity class can be a subclass of the connection abstract base class, inheriting the connection abstract base class and implementing the methods defined in the connection abstract base class it inherits. The connection entity class usually implements the corresponding specific connection logic according to the specific communication connection type. For example, for the communication connection type of the IPMI protocol, a connection entity class of the IPMI protocol can be designed to be responsible for the communication connection based on the IPMI protocol. For the communication connection type of the secure shell protocol (SSH), a connection entity class of the SSH protocol can be designed to be responsible for the communication connection based on the SSH protocol.

[0065] The business logic layer 302 is mainly responsible for implementing specific business functions. The implementation of this function of the business logic layer depends on the business method library. The business method library contains a variety of independent business methods, through which a variety of specific business functions can be implemented. For example, opening a browser is a specific business method; entering an account and password on the web login page of the browser and clicking Login is also a specific business method. Among them, the business method library provides business proxy classes and business entity classes.

[0066] The business proxy class is used to provide a unified external interface for each specific business entity class. By calling the unified interface provided by the business proxy class, the service functions of various business entity classes can be realized. The business proxy class also only provides a unified interface, not specific implementation code, and can be inherited by the business entity class. In addition, the business proxy class has a connection attribute.

[0067] A business entity class refers to a class that specifically implements various business functions. The business entity class provides specific implementation codes corresponding to various business functions. The business entity class can be a subclass of a business proxy class, inheriting the business proxy class. The correspondence between a business proxy class and a business entity class can be a one-to-many relationship, and one business proxy class can correspond to multiple types of business entity classes. Business entity classes of the same type use the same communication connection protocol. In addition, the business entity class also has a connection attribute. In the method provided in an embodiment of the present application, the design of the business proxy class utilizes the principle of dependency inversion, and the connection attribute is a key part of implementing dependency inversion.

[0068] Among them, the test technicians can use the target development tool to customize different business fixture methods and store them in the business method library. Specifically, the test technicians can customize different business fixture methods based on the business logic layer and connection layer in the target development tool. The business method library of the target development tool can include business fixture methods corresponding to IPMI, SNMP, SSH, Redfish, serial port (Serial), socket (Socket), HTTP protocol and file operations under the operating system. It should be noted that when customizing these business fixture methods, the scope of these business fixture methods needs to be set to the session level so that these business fixture methods can be globally effective at the use case layer.

[0069] The logical structure of the business fixture method consists of three parts: (1) Instantiate the business proxy class to obtain an instance object of the business proxy class (which can be called a business object). Because the business proxy class and the business entity class share the connection attribute, the test technician can establish a connection channel between the test execution device and the device under test by calling the connect() method when developing the business fixture method. (2) Return the business object through the yield keyword for use by the use case layer. (3) Recycle the business object: Call the disconnect() method to destroy the connection between the test execution device and the device under test, and restore the environmental status of the test execution device and the device under test that have changed due to the establishment of the connection channel.

[0070] The use case layer 303 is mainly responsible for combining multiple independent business functions for testing to determine whether these business functions can work together. The implementation of this function of the use case layer depends on the use case library. The use case library contains multiple executable test cases. When generating automated test cases, test cases that meet the test requirements can be selected from the use case library for use in automated test cases.

[0071] Among them, the test technician can develop test cases using the target development tool and store them in the test case library. Specifically, the test technician can develop test cases based on the use case layer and the business logic layer in the target development tool.

[0072] In another possible embodiment, the target development tool provides a basic library in addition to the connection library, business logic library and use case library corresponding to the connection layer, business logic layer and use case layer respectively. The basic library is used to provide basic data or resources required for generating automated test cases.

[0073] Therefore, the test execution device can use the target development tools deployed by itself to develop various automated test cases, and then store these automated test cases in the case library. Among them, the automated test case can be composed of test cases, and each automated test case can call multiple business fixture methods.

[0074] Specifically, after obtaining the test requirements, the test execution device can select the business fixture methods that need to be called for the automated test cases for the test requirements from the business method library, and select the test cases that meet the test requirements from the case library for use in the automated test cases. Among them, if the case library cannot fully cover the test requirements, that is, if there are no test cases corresponding to certain test requirements in the case library, the test technician can also regenerate the test cases for these test requirements based on the target development tool, so as to obtain executable automated test cases, thereby ensuring that the test execution device can obtain an automated test case set that can cover all test requirements.

[0075] S202, the test execution device executes the automated test cases in the automated test case set according to a preset sequence until all the automated test cases in the automated test case set are executed to obtain a test result.

[0076] After the test execution device obtains the automated test case set according to the test requirements, the test execution device executes each automated test case in the automated test case set according to a preset sequence to obtain a test result.

[0077] In an embodiment of the present application, a target development tool is provided to develop automated test cases, and the target development tool is composed of a connection layer, a business logic layer, and a use case layer. The target development tool can help test technicians to realize the hierarchical development of test case codes, business function codes, and communication connection codes. And in the target development tool, the use of different methods, such as the use of methods related to communication connections, is standardized. Therefore, based on the embodiment of the present application, the development of automated test cases can be made more convenient, thereby improving the overall efficiency of testing firmware such as the BMC or BIOS of the server.

[0078] In a possible embodiment, the implementation process of “executing the automated test cases in the automated test case set” in the above step S202 may include steps S2021-2022, which are described in detail as follows:

[0079] S2021, the test execution device executes the setup phase of the business fixture method called by the automated test case and establishes a connection with the device under test.

[0080] Before executing the main part of each automated test case, the test execution device executes the setup phase of the business fixture method called by the automated test case and establishes a connection between itself and the device under test. Among them, the setup phase of the business fixture method is the part of the business fixture method used to prepare the test environment and data. The main purpose of executing the setup phase of the business fixture method is to prepare the necessary environment and data for the test. For example, some variables or objects that are necessary in the test may be initialized in the setup phase. Therefore, before executing the main part of each automated test case, the test execution device needs to execute the setup phase of the business fixture method called by the automated test case.

[0081] S2022, the test execution device executes the main part of the automated test case, where the main part of the automated test case includes: pre-operation, main test operation and post-operation of the automated test case.

[0082] After the test execution device completes the setup phase of the business fixture method called by the automated test case and establishes a connection with the device under test, it executes the main part of the automated test case. The main part of the automated test case includes: the pre-operation of the automated test case, the main test operation and the post-operation.

[0083] Among them, the pre-operation of the automated test case refers to a series of operations or work that need to be performed before the main test operation of the automated test case is executed. Exemplarily, the pre-operation of the automated test case may include setting up the test environment to ensure that the main test operation of the automated test case can be executed in the expected environment and obtain the correct test results. The main test operation of the automated test case refers to the test operation actually performed on the tested function or business logic during the automated test process. For example, the main test operation of the automated test case may include operations such as inputting test data and simulating user operations to verify whether the implementation of the tested function or business meets expectations. Among them, the main test operation of the automated test case is determined according to the test requirements, and the specific content of the main test operation of the automated test case is not limited here. The post-operation of the automated test case refers to a series of operations or work that need to be performed after the main test operation of the automated test case is executed. For example, the post-operation of the automated test case may include cleaning up the data used in the test process.

[0084] S2023, the test execution device executes the teardown phase of the business fixture method called by the automated test case.

[0085] After executing the main part of each automated test case, the test execution device executes the teardown phase of the business fixture method called by the automated test case.

[0086] Among them, the teardown phase of the business fixture method is the part of the business fixture method used to clean up the environment after the test and restore it to the state before the test. The main purpose of executing the teardown phase of the business fixture method is to clean up the environment and data after the test is completed, so as to restore the environment and data to the state before the test. For example, in the teardown phase, you can perform operations such as closing database connections, releasing resources, and deleting temporary files, so as to avoid the environment and data changed due to this test from affecting the test results of the next test and reducing the accuracy of the test. Therefore, after executing the main part of each automated test case, the test execution device also needs to execute the teardown phase of the business fixture method called by the automated test case.

[0087] In one possible embodiment, please refer to Figure 4 , Figure 4 A flow chart of determining a business fixture method to be called for an automated test case provided by an embodiment of the present application is shown, and is described in detail as follows:

[0088] S401, the test execution device obtains the test requirements, determines the business fixture methods that need to be called by the automated test cases according to the test requirements, and adds the business fixture methods that need to be called by the automated test cases to the business fixture method set.

[0089] The test execution device obtains the test requirements, determines all the business fixture methods that need to be called by the automated test cases according to the test requirements, and adds the business fixture methods that need to be called by the automated test cases to the business fixture method set. Among them, the business fixture methods can be pre-customized and stored in the business method library of the target development tool. The business method library can include business fixture methods corresponding to IPMI, SNMP, SSH, Redfish, serial port (Serial), socket (Socket), HTTP protocol and file operations in the operating system. The business method library can also include business fixture methods corresponding to other protocols or operations that need to be executed during testing.

[0090] After the test execution device obtains the test requirements, the technician can operate on the test execution device: select the business fixture method to be used from the business method library of the target development tool, and the test execution device obtains the test technician's operation information on the device to determine the business fixture method to be called by the automated test case. After the test execution device obtains the test requirements, the test execution device can also determine the business fixture method to be called by the automated test case from the business method library of the target development tool based on the test requirements.

[0091] S402, determine whether the business fixture method set covers all test requirements.

[0092] After the test execution device obtains the business fixture method set, it needs to determine whether the determined business fixture methods (ie, the business fixture method set) cover all test requirements.

[0093] Specifically, by determining whether the business fixture methods that the automated test case needs to call for the test requirements all exist in the business fitxure method set, it is determined whether the business fixture method set covers all test requirements. If the business fixture methods that the automated test case needs to call for the test requirements all exist in the business fitxure method set, it indicates that the business fixture method set covers all test requirements; otherwise, it indicates that the business fixture method set does not cover all test requirements. For example, when the test requirements involve testing of IPMI, SNMP, and SSH, the automated test case needs to call the business fixture methods corresponding to IPMI, SNMP, and SSH respectively for these test requirements. At this time, by determining whether the business fixture methods corresponding to IPMI, SNMP, and SSH respectively exist in the business fitxure method set, it is determined whether the business fixture method set covers all test requirements.

[0094] S403, when the business fixture method set covers all test requirements, each business fixture method in the business fixture method set that has covered all test requirements is passed into the automated test case.

[0095] When the business fixture method set covers all test requirements, the business fixture method set at this time can be called a business fixture method set that has covered all test requirements. At this time, the test execution device passes each business fixture method in the business fixture method set that has covered all test requirements into the automated test case, and the automated test case can call the business functions of these business fixture methods.

[0096] S404, when the business fixture method set does not cover all test requirements, the test execution device generates a business fixture method corresponding to the uncovered test requirements, and adds it to the business fixture method set to obtain an updated business fixture method set; repeat the execution until the updated business fixture method set covers all test requirements.

[0097] When the business fixture method set does not cover all test requirements, the business fixture method set at this time can be called a business fixture method set that does not cover all test requirements. At this time, the test execution device generates corresponding business fixture methods for the uncovered test requirements using the target development tool, and adds the business fixture methods corresponding to the uncovered test requirements to the business fixture method set to obtain an updated business fixture method set. Repeat the above steps until the updated business fixture method set covers all test requirements.

[0098] In an optional embodiment, different ways can be used to pass each business fixture method into an automated test case. Optionally, the first way: the function name of the business fixture method can be used as a parameter for the automated test case to call. The second way: the business fixture method can be passed into the automated test case through the decorator pattern: use decorators in the automated test case, such as the @pytest.mark.parametrize decorator to call the business fixture method; or, use the @pytest.fixture decorator to declare the business fixture method, so that the business fixture method can be identified and called into the automated test case for use. The third way: use the first and second ways at the same time to pass each business fixture method into the automated test case. When applying, the specific way to pass the business fixture method into the automated test case can be determined according to the actual situation, which is not limited here.

[0099] The following are two sample codes that use the second method, that is, passing the business fixture method into the automated test case through the decorator pattern.

[0100] @pytest.mark.usefixtures()?

[0101] Decorator mode parameter passing:

[0102] @pytest.mark.parametrize("fixture_service_1_2",argvalues=[{"dut_id":1},{"dut_id":2},...],indirect=True)

[0103] or:

[0104] #--Business fixture--

[0105] @pytest.fixture(params=["ssh1","ssh2",…])

[0106] def multi_ssh(request):

[0107] "'Parameterization of multiple ssh connections'"

[0108] return request.getfixturevalue(request.param)

[0109] #--Test Case--

[0110] def test_get_os_version(multi_ssh):

[0111] "'Test multiple test machine operating system host name'"

[0112] assert multi_ssh.get_hostname()==hostname

[0113] Through the embodiment of the present application, when developing automated test cases for test requirements, the automated test cases can call the required business fixture methods. Since the business fixture methods are called in the automated test cases, and the same business fixture method can be called in multiple test cases, it is avoided to repeatedly write similar codes, thus reducing the time for developing automated test cases and improving the efficiency of testing.

[0114] Corresponding to the test method described in the above embodiment, Figure 5 A schematic diagram of the structure of a testing device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0115] Reference Figure 5 , the testing device 500 comprises:

[0116] The first processing module 501 obtains the test requirements and obtains an automated test case set according to the test requirements; the automated test case set includes a number of automated test cases, and the business fixture method is called in the automated test case; wherein the test device is deployed with a target development tool, and the target development tool is used to develop the automated test case.

[0117] The second processing module 502 is used to execute the automated test cases in the automated test case set according to a preset order until all the automated test cases in the automated test case set are executed to obtain a test result.

[0118] The process of each module in the test device provided in the embodiment of the present application realizing its own function can be specifically referred to the aforementioned Figure 1 The description of the illustrated embodiment and other related method embodiments will not be repeated here.

[0119] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0120] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0122] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0123] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0124] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used to describe various elements in some embodiments of the present application in the text, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first table can be named as the second table, and similarly, the second table can be named as the first table without departing from the scope of the various described embodiments. The first table and the second table are both tables, but they are not the same table.

[0125] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0126] The testing method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, laptop computers, industrial computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0127] Figure 6 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 6 As shown, the electronic device 600 of this embodiment includes: at least one processor 602 ( Figure 6 ), a memory 601, in which a computer program 603 that can be run on a processor 602 is stored. When the processor 602 executes the computer program 603, the steps in the above-mentioned test method embodiments are implemented, for example Figure 1 Alternatively, when the processor 602 executes the computer program 603, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 5 Functionality of the modules shown.

[0128] The electronic device 600 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a memory 601 and a processor 602. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 600 and does not constitute a limitation of the electronic device 600. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include an input sending device, a network access device, a bus, etc.

[0129] The processor 602 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0130] In some embodiments, the processor 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. The processor 602 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 600. Further, the processor 602 may also include both an internal storage unit of the electronic device 600 and an external storage device. The processor 602 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program, etc. The processor 602 may also be used to temporarily store data that has been sent or is to be sent.

[0131] In addition, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0132] An embodiment of the present application also provides an electronic device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the electronic device implements the steps of any of the above-mentioned method embodiments.

[0133] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0134] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0135] An embodiment of the present application also provides a chip system, which includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps in the above-mentioned method embodiments.

[0136] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0137] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A testing method, characterized in that: include: The test execution device obtains the test requirements and obtains an automated test case set according to the test requirements; The automated test case set includes a plurality of automated test cases, in which a business fixture method is called; wherein the test execution device is deployed with a target development tool, and the target development tool is used to develop the automated test case; The test execution device executes the automated test cases in the automated test case set according to a preset sequence until all the automated test cases in the automated test case set are executed to obtain a test result.

2. The testing method according to claim 1, characterized in that: The executing of the automated test case in the automated test case set comprises: The test execution device executes the setup phase of the business fixture method called by the automated test case and establishes a connection with the device under test; The test execution device executes the main part of the automated test case, and the main part of the automated test case includes: the pre-operation, the main test operation and the post-operation of the automated test case; The test execution device executes the teardown phase of the business fixture method called by the automated test case.

3. The testing method according to claim 1, characterized in that: The target development tool includes a connection layer, a business logic layer and a use case layer; The connection layer is used to establish a communication connection channel between the test execution device and the device under test, and the implementation of the function of the connection layer depends on the connection library; The business logic layer is used to implement specific business functions, and the implementation of the business logic layer functions depends on the business method library; The use case layer is used to combine multiple independent business functions together for testing to determine whether these business functions can operate in coordination. The implementation of the functions of the use case layer depends on the use case library.

4. The testing method according to claim 3, characterized in that: The business method library includes business fixture methods corresponding to IPMI, SNMP, SSH, Redfish, serial port, socket, HTTP protocol and file operations under the operating system.

5. The testing method according to claim 3, characterized in that: The connection library includes a connection abstract base class and a connection entity class; the connection abstract base class is used to define the declaration of methods related to the communication connection, and the connection entity class is a class that specifically implements the methods defined by the connection abstract base class; The business method library includes business proxy classes and business entity classes. The business proxy class is a class used to provide a unified external interface for each specific business entity class, and the business entity class is a class that specifically implements various business functions.

6. The testing method according to any one of claims 1 to 5, characterized in that: The business fixture method is called in the automated test case, including: The test execution device obtains the test requirements, determines the business fixture method that needs to be called by the automated test case according to the test requirements, and adds the business fixture method that needs to be called by the automated test case to the business fixture method set; Determine whether the business fixture method set covers all the test requirements; When the business fixture method set covers all the test requirements, the business fixture methods included in the business fixture method set that has covered all the test requirements are passed into the automated test case.

7. The testing method according to claim 6, characterized in that: The method further comprises: When the business fixture method set does not cover all the test requirements, the test execution device generates a business fixture method corresponding to the uncovered test requirements, and adds the business fixture method corresponding to the uncovered test requirements to the business fixture method set to obtain an updated business fixture method set; and repeats the execution until the updated business fixture method set covers all the test requirements.

8. A testing device, characterized in that: include: The first processing module is used to obtain test requirements and obtain an automated test case set according to the test requirements; The automated test case set includes a plurality of automated test cases, in which a business fixture method is called; wherein the test device is deployed with a target development tool, and the target development tool is used to develop the automated test case; The second processing module is used to execute the automated test cases in the automated test case set according to a preset order until all the automated test cases in the automated test case set are executed to obtain a test result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images