Server test method, test device, equipment and storage medium

Abstract

The invention provides a server testing method and device, equipment and a storage medium, and the device comprises an equipment information module which stores the equipment information of a plurality of pieces of equipment to be tested, and selects target equipment to be tested according to a test task; the test case module is used for integrating test case sets of a plurality of test types and selecting a test case of at least one target test type according to the test task; the test environment detection and deployment module is used for detecting whether a test tool required by a test case of a target test type is installed in the target to-be-tested equipment or not; the test module is used for sending a related test instruction to the target to-be-tested equipment according to the test case of the target test type under the condition that the test tool is installed; and the test report module is used for collecting a test result obtained by the target to-be-tested equipment running the related test instruction based on the test tool. The invention provides an automatic and systematized testing device and method.

Description

Technical Field

[0001] The present disclosure relates to the field of testing technology, and in particular to a testing method, a testing device, equipment and a storage medium for a server. Background Art

[0002] The server has high-speed CPU computing power, long-term reliable operation, powerful I / O external data throughput and better scalability. Before leaving the factory, the server needs to be tested and verified to ensure its functionality and stability.

[0003] At present, some automated testing technologies for servers are mostly applicable to servers based on X86 and ARM architectures. However, for hardware testing and software testing of servers based on the fifth-generation Reduced Instruction Set Computer-five (RISC-V) architecture, different test parameters are usually input manually for test cases to implement relevant tests on the server. In addition, it is necessary to manually collect test logs and verify the logs to determine the accuracy of the test results. The above test methods are mainly completed manually, which is costly, time-consuming, and inefficient. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides an automated and systematic server testing method, testing device, equipment and storage medium.

[0005] In a first aspect, a test device for a RISC-V architecture server is provided, comprising:

[0006] The device information module stores the device information of multiple devices under test and selects the target device under test according to the test task;

[0007] A test case module integrates test case sets of multiple test types and selects at least one test case of a target test type according to the test task;

[0008] A test environment detection and deployment module detects whether the target device to be tested has installed the test tools required for the test case of the target test type;

[0009] A test module, which sends relevant test instructions to the target device under test according to the test case of the target test type when the test tool has been installed; and

[0010] The test report module collects the test results obtained by the target device under test based on the test tool running the relevant test instructions.

[0011] Optionally, the test tool is adapted to the RISC-V architecture.

[0012] Optionally, the test environment detection and deployment module also remotely controls the target device under test to install the test tool if it is detected that the test tool is not installed; and detects again whether the test tool required for the test case of the target test type is installed in the target device under test.

[0013] Optionally, the test environment detection and deployment module controls the target device under test to install a toolkit of the test tool that has been integrated into itself based on relevant installation instructions, or controls the target device under test to automatically select a compilation method that is compatible with the RISC-V architecture, receive or download an installation package of the test tool, compile based on the selected compilation method, and install the compiled installation package of the test tool.

[0014] Optionally, the test case set includes at least a plurality of test cases, expected results, and the collected test results, and the test report module further generates a test report by comparing the test results with the expected results.

[0015] In a second aspect, a test method for a RISC-V architecture server is provided, including:

[0016] Selecting a target device to be tested according to the test task and device information of multiple devices to be tested that are stored;

[0017] Selecting at least one test case of a target test type from a test case set of multiple test types according to the test task;

[0018] Detecting whether the target device to be tested has installed a test tool required for a test case of the target test type;

[0019] sending relevant test instructions to the target device under test according to the test case of the target test type when the test tool has been installed; and

[0020] Collect test results obtained by the target device under test based on the test tool running the relevant test instructions.

[0021] Optionally, the test tool is adapted to the RISC-V architecture.

[0022] Optionally, the testing method further includes:

[0023] When it is detected that the test tool is not installed, the target device under test is remotely controlled to install the test tool, and the test tool required for the test case of the target test type is again detected to be installed in the target device under test.

[0024] Optionally, when it is detected that the test tool is not installed, remotely controlling the target device to be tested to install the test tool includes:

[0025] Based on relevant installation instructions, the target device under test is controlled to install the tool kit of the test tool that has been integrated into itself, or the target device under test is controlled to automatically select a compilation method adapted to the RISC-V architecture, receive or download the installation package of the test tool, compile based on the selected compilation method, and install the compiled installation package of the test tool.

[0026] Optionally, the test case set includes at least a plurality of test cases, expected results, and the collected test results, and the testing method further includes:

[0027] A test report is generated by comparing the test result with the expected result.

[0028] In a third aspect, an electronic device is provided, including:

[0029] Memory, used to store a set of programs, instructions or codes;

[0030] At least one processor is used to execute the program, instruction or code stored in the memory to implement the steps of the testing method as described above.

[0031] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the test method described above is implemented.

[0032] The server testing method, testing device, equipment and storage medium provided by the present disclosure can automatically select the device to be tested and the test case set of the corresponding test type according to the test task, and send the test instructions to the device to be tested after detecting the test environment of the device to be tested, and collect the test results to generate a test report. Therefore, the server can be tested in a more automated and systematic manner.

[0033] It should be noted that the above general description and the following detailed description are merely exemplary and explanatory and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram showing the structure of a test system for a RISC-V architecture server provided according to an embodiment of the present application is shown;

[0035] Figure 2 A schematic diagram showing the structure of a test device for a RISC-V architecture server provided according to an embodiment of the present application is shown;

[0036] Figure 3 A flow chart showing a method for testing a RISC-V architecture server provided according to an embodiment of the present application is shown;

[0037] Figure 4 A flow chart showing another method for testing a RISC-V architecture server provided according to an embodiment of the present application is shown;

[0038] Figure 5 A schematic structural diagram of an electronic device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] In order to facilitate the understanding of the present application, the present application will be described more comprehensively below with reference to the relevant drawings. The preferred embodiments of the present application are given in the drawings. However, the present application can be implemented in different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive.

[0040] The following will take the RISC-V architecture server as an example to illustrate the test system and test method of the server of the present application.

[0041] Figure 1 A structural schematic diagram of a RISC-V architecture server test system provided according to an embodiment of the present application is shown. Figure 2 A structural schematic diagram of a RISC-V architecture server test device provided according to an embodiment of the present application is shown.

[0042] See also Figure 1 The test system includes a test machine 100, a switch 200, a load balancer 300, and a device under test 400.

[0043] The test machine 100 is equipped with a test device 110 to complete the hardware test and software test of the device under test 400. In this embodiment, the test device 110 installed on the test machine 100 is, for example, an automated test platform. In other embodiments, the test device can also be a terminal device, a server, or software that can be installed on a hardware device. The test device 110 integrates multiple test case sets for implementing the test of different types of modules in the device under test 400.

[0044] The device under test 400 may be, for example, a RISC-V architecture server or other RISC-V related mobile terminals. Among them, the server is a device that provides computing services and has the ability to respond to service requests, provide services, and ensure services.

[0045] It should be noted that the test system includes more than one device under test 400. Correspondingly, the test device 110 can test multiple devices under test 400 in series or in parallel, and perform multiple tests on the same device under test 400 in series or in parallel.

[0046] The switch 200 establishes communication between the test machine 100 and the device under test 400 to perform related test tasks.

[0047] The load balancer 300 is configured between the switch 200 and the device under test 400 to effectively distribute network traffic among multiple servers to ensure that the test can proceed smoothly.

[0048] The above-mentioned test system can perform hardware, interface, function, CPU, memory, storage, network and other aspects of the device under test (taking the RISC-V architecture server as an example), and finally form a test report by collecting the test results to be presented on the test device 110 of the test machine 100.

[0049] See also Figure 2 The testing device 110 includes a device information module 111 , a test case module 112 , a test environment detection and deployment module 113 , a test module 114 , and a test report module 115 .

[0050] The device information module 111 is used to store the device information of multiple devices under test and select the target device under test according to the test task. Exemplarily, the device information module 111 includes the device information of multiple devices under test 400 registered in the test device 110, and the device information at least includes the device IP. The test task at least includes the device information and test content of the device under test 400. The device information module 111 selects the target device under test that matches the received test task.

[0051] The test case module 112 is used to integrate test case sets of multiple test types, and select at least one test case set of a target test type according to the test task. Exemplarily, the multiple test types include at least operating system testing, Web application testing, database testing, cloud platform testing, and containerized application (Kubernetes, K8S) testing on multiple hosts in the management cloud platform. Further, it can also include test types such as big data testing, high-performance computing testing, and virtualization testing. The test case set includes at least multiple test cases, expected results, and collected test results. Taking the operating system test case set as an example, the corresponding test cases include at least functional test cases, performance test cases, stability test cases, interface test cases, stress test cases, network test cases, and test cases generated by combining the values ​​of different test items. The expected results are, for example, the verification specifications of the corresponding test items. The collected test results refer to the test results of the corresponding test items, which can be, for example, score values, read and write rate values, json values ​​of interface parameters, etc. The test case module 112 selects a test case of at least one target test type corresponding to the test content in the received test task. Exemplarily, taking the test content in the test task as testing the network performance of the operating system as an example, the test case module 112 selects a network test case in the operating system test case set.

[0052] In other embodiments, the test types can be further divided into operating system performance testing, operating system function testing, operating system stability testing, operating system interface testing, operating system stress testing, operating system network testing, Web application performance testing..., database performance testing..., cloud platform performance testing..., management of containerized applications (Kubernetes, K8S) on multiple hosts in the cloud platform...

[0053] The test environment detection and deployment module 113 is used to detect whether the target device under test has installed the test tools required for the test cases of the target test type. Exemplarily, the required performance test tools include, for example, speccpu, fio, iozone, stress-ng, iperf, Unixbench, Ltpstress, etc. Further, the test environment detection and deployment module 113 determines whether the device under test 400 has the test tools required for the test cases corresponding to the test tasks by sending instructions to the device under test 400 to check whether there is an installation package of the corresponding test tool in the device under test 400. And when the corresponding installation package is detected, the result of the installed test tool is returned to the test module 114.

[0054] Furthermore, the above-mentioned required testing tools are adapted to the RISC-V architecture.

[0055] Further, the test environment detection and deployment module 113 also remotely controls the device under test 400 to install the corresponding test tool when it is detected that the test tool is not installed, and detects again whether the test tool required for the test case of the target test type is installed in the device under test 400. Exemplarily, the step of remotely controlling the device under test 400 to install the corresponding test tool by the test environment detection and deployment module 113 includes: the test environment detection and deployment module 113 controls the device under test 400 to install the toolkit of the test tool integrated with itself based on the relevant installation instructions stored in advance, or controls the device under test 400 to automatically select the compilation method adapted to the RISC-V architecture, receive or download the toolkit of the test tool, compile based on the selected compilation method, and install the installation package of the compiled test tool. The above compilation is to apply the test tool that is not suitable for the RISC-V architecture to the present embodiment. Further, when the required test tool is a test tool suitable for the RISC-V architecture integrated on the device under test 400, such as the name of the yum install toolkit, the relevant installation instructions instruct the device under test 400 to install directly. When the required test tool is a test tool that does not support the RISC-V architecture, the relevant installation instructions include instructing the device under test to automatically select a compilation method that is adapted to the RISC-V architecture, the test environment detection and deployment module 113 sends an scp transmission instruction to transfer the required tool package to the device under test 400 or sends a download instruction (such as a wget tool package download link) to enable the device under test 400 to download the required tool package, instructs the device under test 400 to compile the required tool package, and instructs the device under test 400 to install the tool under test (such as dpkg -i tool package name). Exemplarily, the type of the required test tool (one of the above two categories) can be determined during the detection process.

[0056] Exemplarily, taking the test content in the test task as testing the network performance of the operating system as an example, the test environment detection and deployment module 113 sends an instruction to the device under test 400 to check whether there is an installation package of the test tool iperf for executing the network performance test case of the operating system in the device under test 400. When the test environment detection and deployment module 113 detects the installation package of the test tool iperf, it returns the result of installation to the test module 114, otherwise the test environment detection and deployment module 113 controls the device under test 400 to install the toolkit that has been integrated and adapted to the RISC-V architecture based on the relevant installation instructions, or automatically selects the compilation method adapted to the RISC-V architecture, downloads or receives the installation package of the test tool iperf transmitted by the test device 110, compiles based on the selected compilation method, and installs the compiled test tool iperf. And again detect whether the test tool required for the test case of the target test type is installed in the device under test 400.

[0057] The test module 114 sends relevant test instructions to the target device under test according to the test case of the target test type when the test tool has been installed. Further, upon receiving the installed detection result sent by the test environment detection and deployment module 113, the test module 114 is triggered to send relevant test instructions to the device under test 400 one by one according to the test steps set in the target test case set.

[0058] The test report module 115 is used to collect the test results obtained by the target device under test based on the relevant test instructions in the test case of the target test type run by the test tool. Further, the device under test 400 executes the received test instructions based on the corresponding test tool and generates test results. The test report module 115 collects the test results and returns the test results to the corresponding test case set. The test report module 115 also compares the test results with the expected results, for example, comparing the collected score values ​​with the score values ​​or score ranges in the expected results, comparing the read and write rate values, calibrating the json values ​​of the interface parameters, etc. The test report can be, for example, an excel version or an html version, and is displayed in a chart. Further, the test report at least includes the total number of test cases, the test success rate, and the test results.

[0059] Furthermore, the test device 110 also includes a log collection module 116, which is used to obtain relevant test logs when the test results do not match the expected results. Furthermore, the above test logs can also be presented in the test report, for example. Among them, the collection of test logs is to facilitate the test personnel to analyze and study the error logs, reduce the trouble of manually capturing error logs, and avoid missing relevant error logs, thereby further improving the test efficiency.

[0060] Figure 3 A flow chart of a testing method for a RISC-V architecture server provided according to an embodiment of the present application is shown. Figure 4 A flow chart showing another method for testing a RISC-V architecture server provided according to an embodiment of the present application is shown.

[0061] See also Figure 3 , provides a testing method suitable for RISC-V architecture servers, comprising the following steps:

[0062] Step S510: Submit the test task. For example, a tester submits the test task to the test device 110. The test task also includes at least device information of the device under test and test content.

[0063] Step S520: Selecting a target device under test according to the test task and the device information of the stored multiple devices under test. The device information module 111 of the test apparatus 110 selects a target device under test that matches the device information contained in the received test task from the stored registration information.

[0064] Step S530: Select at least one test case of a target test type from a set of test cases of multiple test types according to the test task. The test case module 112 of the test device 110 selects at least one test case of a target test type corresponding to the test content in the received test task.

[0065] Step S540: Detect whether the target device to be tested has installed therein a test tool required by a test case of the target test type.

[0066] Furthermore, the test tools required above are adapted to the RISC-V architecture. Figure 4 , the steps include:

[0067] Step S541: Detect whether the target device under test has installed the test tools required for the test cases of the target test type. The test environment detection and deployment module 113 of the test device 110 determines whether the device under test 400 has the test tools required for the test cases corresponding to the test task when at least one test case of the target test type corresponding to the test task is selected.

[0068] If it is detected that it is not installed, execute:

[0069] Step S542: Remotely control the device under test to install the corresponding test tool. Exemplarily, the test environment detection and deployment module 113 controls the device under test 400 to install the toolkit of the test tool that has been integrated with itself based on the relevant installation instructions stored in advance, or controls the device under test 400 to automatically select the compilation method adapted to the RISC-V architecture, receive or download the toolkit of the test tool, compile based on the selected compilation method, and install the installation package of the compiled test tool. Furthermore, when the required test tool is a test tool for the RISC-V architecture integrated on the device under test 400, such as the name of the yum install toolkit, the relevant installation instructions instruct the device under test 400 to install directly. When the required test tool is a test tool that does not support the RISC-V architecture, the relevant installation instructions include instructing the device under test to automatically select a compilation method that is adapted to the RISC-V architecture, the test environment detection and deployment module 113 sends an scp transmission instruction to transfer the required tool package to the device under test 400 or sends a download instruction (such as a wget tool package download link) to enable the device under test 400 to download the required tool package, instructs the device under test 400 to compile the required tool package, and instructs the device under test 400 to install the tool under test (such as dpkg -i tool package name). Exemplarily, the type of the required test tool (one of the above two categories) can be determined during the detection process.

[0070] Return to step S541 to detect again whether the test tool required by the test case of the target test type is installed in the device to be tested.

[0071] After detecting that it has been installed, continue to execute step S550: send relevant test instructions to the target device under test according to the test case of the target test type. After receiving the installed detection result sent by the test environment detection and deployment module 113, the trigger test module 114 sends relevant test instructions to the device under test 400 one by one according to the test steps set in the target test case set.

[0072] Step S560: Collect the test results obtained by the target device under test based on the test tool running the relevant test instructions. The device under test 400 executes the received test instructions based on the corresponding test tool and generates test results. The test report module 115 collects the test results and returns the test results to the corresponding test case set.

[0073] In other embodiments, the server testing method further includes:

[0074] Step S570: Compare the test results with the expected results and generate a test report. The test report module 115 also compares the test results with the expected results and generates a test report.

[0075] In other embodiments, the server testing method further includes:

[0076] Step S580: Obtain relevant test logs when the test results do not match the expected results. The log collection module 116 of the test device 110 obtains relevant test logs when the test results do not match the expected results, so as to facilitate the tester to analyze and study the error logs.

[0077] The embodiment of the present disclosure also provides an electronic device.

[0078] Figure 5 A schematic diagram of an electronic device provided according to an embodiment of the present disclosure is shown.

[0079] See also Figure 5 The electronic device 600 includes one or more of a processing component 610, a memory 620, a power component 630, a multimedia component 640, an audio component 650, an input / output (I / O) interface 660, a sensor component 670, and a communication component 680. The processing component 610 is used to control the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 610, for example, includes one or more processors (not shown in the figure) to execute instructions to at least perform the steps of the test method of the server.

[0080] In addition, the processing component 610 includes, for example, one or more modules to facilitate interaction between the processing component 610 and other components. The memory 620 is configured to store various types of data to support operations in the electronic device 600. The power supply component 630 provides power to various components of the electronic device 600. The multimedia component 640 includes a screen that provides an output interface between the electronic device 600 and the user.

[0081] The audio component 650 is configured to output and / or input audio channels.

[0082] The input / output interface 660 provides an interface between the processing component 610 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc.

[0083] The sensor component 670 includes one or more sensors for providing status assessment of various aspects for the electronic device 600 .

[0084] The communication component 680 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices.

[0085] The electronic device 600 may be, for example, a computer.

[0086] Exemplarily, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital channel processors (DSPs), digital channel processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned server testing method.

[0087] Exemplarily, a non-volatile computer-readable storage medium is also provided, such as a memory 620 including computer program instructions, which can be executed by a processor in a processing component 610 in an electronic device 600 to complete a server testing method for executing the above instructions.

[0088] It should be noted that the numerical values ​​in this article are only used for exemplary description. In other embodiments of the present application, other numerical values ​​can also be sampled to implement this solution. The specific settings should be reasonable according to the actual situation, and this application does not limit this.

[0089] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly explaining the present application, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from this are still within the scope of protection of the present application.

[0090] It should also be understood that the terms and expressions used herein are for descriptive purposes only, and one or more embodiments of this specification should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or parts thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, changes and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

Claims

1. A server testing device, wherein: include: The device information module stores the device information of multiple devices under test and selects the target device under test according to the test task; A test case module, integrating test case sets of multiple test types, and selecting at least one test case of a target test type according to the test task; A test environment detection and deployment module detects whether the target device to be tested has installed the test tools required for the test case of the target test type; A test module, which sends relevant test instructions to the target device under test according to the test case of the target test type when the test tool has been installed; as well as The test report module collects the test results obtained by the target device under test based on the test tool running the relevant test instructions.

2. The server testing device according to claim 1, wherein: The test tool is adapted to the RISC-V architecture.

3. The server testing device according to claim 2, wherein: The test environment detection and deployment module also remotely controls the target device under test to install the test tool if it is detected that the test tool is not installed, and detects again whether the test tool required for the test case of the target test type is installed in the target device under test.

4. The server testing device according to claim 3, wherein: The test environment detection and deployment module controls the target device under test to install the tool kit of the test tool that has been integrated into itself based on relevant installation instructions, or controls the target device under test to automatically select a compilation method adapted to the RISC-V architecture, receive or download the installation package of the test tool, compile based on the selected compilation method, and install the compiled installation package of the test tool.

5. The server testing device according to any one of claims 1 to 4, wherein: The test case set includes at least a plurality of test cases, expected results, and the collected test results. The test report module also generates a test report by comparing the test results with the expected results.

6. A method for testing a server, wherein: include: Selecting a target device to be tested according to the test task and device information of multiple devices to be tested that are stored; Selecting at least one test case of a target test type from a test case set of multiple test types according to the test task; Detecting whether the target device to be tested has installed a test tool required for a test case of the target test type; Sending relevant test instructions to the target device under test according to the test case of the target test type when the test tool has been installed; as well as Collect test results obtained by the target device under test based on the test tool running the relevant test instructions.

7. The server testing method according to claim 6, wherein: The test tool is adapted to the RISC-V architecture.

8. The server testing method according to claim 7, wherein: Also includes: When it is detected that the test tool is not installed, the target device under test is remotely controlled to install the test tool, and the test tool required for the test case of the target test type is again detected to be installed in the target device under test.

9. The server testing method according to claim 8, wherein: Remotely controlling the target device to be tested to install the test tool when detecting that the test tool is not installed includes: Based on relevant installation instructions, the target device under test is controlled to install the tool kit of the test tool that has been integrated into itself, or the target device under test is controlled to automatically select a compilation method adapted to the RISC-V architecture, receive or download the installation package of the test tool, compile based on the selected compilation method, and install the compiled installation package of the test tool.

10. The server testing method according to any one of claims 6 to 9, wherein: The test case set includes at least a plurality of test cases, expected results, and the collected test results, and the testing method further includes: A test report is generated by comparing the test result with the expected result.

11. An electronic device, wherein: include: Memory, used to store a set of programs, instructions or codes; At least one processor is used to execute the program, instruction or code stored in the memory to implement the steps of the server testing method as described in any one of claims 6-10.

12. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program or instruction. When the computer program or instruction is executed by a processor, the server testing method according to any one of claims 6 to 10 is implemented.

Images