A cluster topology testing method and device based on IB network

Abstract

The present invention provides a cluster topology testing method and device based on an IB network, the method comprising: determining a deployment server from a cluster to be tested, deploying a test software system platform on the deployment server, and issuing a test file to the deployment server; triggering a cluster scan from the deployment server, recording the initial topology information in the cluster through the test software system platform and performing persistent storage; executing the test file based on the initial topology information, establishing a container as a master node in the deployment server, selecting a cluster node to be tested as a slave node, sending the image of the container to the slave node, uploading the test data results of all slave nodes to the master node, and generating a test report based on the test data results. The technical solution of the present invention realizes real-time synchronous update of test tasks in large-scale clusters.

Description

Technical Field

[0001] The invention belongs to the field of network testing, and in particular relates to a cluster topology testing method and device based on an IB network. Background Art

[0002] InfiniBand (IB) is a high-speed network communication protocol suitable for data centers (DC) and high-performance computing (HPC). The high bandwidth and low latency of the IB network provide a high-speed and reliable communication foundation for large-scale cluster data transmission and high-performance computing applications.

[0003] System network testing involves testing and evaluating the server's network functions, performance, availability, and security, and is specifically manifested in bandwidth testing, latency testing, packet loss rate testing, throughput testing, network function testing, and non-functional testing. These network tests are usually implemented through professional network testing tools, hardware equipment, or software. However, with the continuous development of network technology, some emerging network testing methods and tools have gradually been introduced to adapt to more complex, higher performance requirements, and clustered network environments.

[0004] The cluster testing method provided by related technologies usually uses known cluster information files. Once the cluster topology changes, it is impossible to actively discover the changed device information in the cluster, and it is impossible to meet the irregular changes in the customer's actual scenarios under the agile development model. It is impossible to dynamically capture the changes, additions, and removals of devices in the cluster. It is necessary to manually execute the deployment-related test plan files and basic environment. If large-scale node replacement is involved, this configuration method is very inefficient and requires a lot of human resources. Summary of the invention

[0005] The purpose of the present invention is to provide a cluster topology testing method and device based on an IB network, aiming to solve the problems of large-scale nodes in the cluster, difficult maintenance, and difficult acquisition of device information updates.

[0006] According to a first aspect of the present invention, a cluster topology testing method based on an IB network is provided, comprising:

[0007] Determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server;

[0008] Triggering a cluster scan from the deployment server, recording the initial topology information in the cluster through the test software system platform and performing persistent storage;

[0009] The test file is executed based on the initial topology information, a container is established in the deployment server as a master node, a cluster node to be tested is selected as a slave node, an image of the container is sent to the slave node, test data results of all slave nodes are uploaded to the master node, and a test report is generated according to the test data results.

[0010] Preferably, the deploying the test software system platform on the deployment server further comprises: transferring the test software system platform deployment package to the root directory in the deployment server by transmission or copying, and setting the directory permission to be readable and writable.

[0011] Preferably, the deployment server deploys the test software system platform, further comprising:

[0012] Execute the test software system platform code deployment package;

[0013] Update system software package management sources, uninstall old system software and install adapted software versions, update and upgrade existing software versions to adapt to the platform; install open source software based on the IB protocol;

[0014] Deploy the Jenkins platform for visual editing and triggering test file execution;

[0015] deploying the test software system platform,

[0016] Deploy private repository images for container image management.

[0017] Preferably, the recording of initial topology information within the cluster further includes:

[0018] Record the information of all servers, switches, and routers in the cluster.

[0019] Preferably, after uploading the test data results of all slave nodes to the master node, the method further comprises:

[0020] The master node actively destroys the slave nodes generated in this round of testing, and executes the self-destruction code to destroy the master node itself.

[0021] According to a second aspect of the present invention, there is provided a cluster topology testing device based on an IB network, comprising:

[0022] A deployment unit, used to determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server;

[0023] A scanning unit, used to trigger a cluster scan from the deployment server, record the initial topology information in the cluster through the test software system platform and store it persistently;

[0024] The test unit is used to execute the test file based on the initial topology information, establish a container as a master node in the deployment server, select a cluster node to be tested as a slave node, send the image of the container to the slave node, upload the test data results of all slave nodes to the master node, and generate a test report according to the test data results.

[0025] Compared with the related art, the technical solution of the present invention has the following advantages:

[0026] The present invention establishes a mature monitoring mechanism, including self-destruction of master nodes, persistent storage of test data and test results of each test node, and dynamic monitoring of the test node status in the cluster, which solves the problem that nodes or devices in large-scale clusters cannot be synchronized in real time before the test task is executed after being updated. Dynamically identify operations such as adding, removing, and replacing nodes in the cluster. According to the test task requirements, based on the latest cluster environment information, dynamically plan the screening of test nodes without manually specifying test nodes. Based on the one-click deployment and destruction test environment of docker technology, the test node is unaware of the changes in software and hardware before and after the test, realizing non-destructive testing of the test node, and avoiding the test node failure or dirty data caused by the test task affecting the operation of the cluster.

[0027] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures and processes indicated in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. It is obvious that the drawings described below are certain embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 It is a topological structure diagram of a data center cluster according to the present invention.

[0030] Figure 2 It is a schematic diagram of a complex topology test process based on an IB network according to the present invention.

[0031] Figure 3It is an overall flow chart of the cluster topology testing method based on the IB network according to the present invention.

[0032] Figure 4 It is a schematic diagram of the software system code platform deployment process according to the present invention.

[0033] Figure 5 It is a schematic diagram of the test file execution process according to the present invention.

[0034] Figure 6 It is a schematic diagram of the test process executed by the software system code platform according to the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Based on the above analysis, the present invention proposes a cluster topology testing method and device based on an IB network. By deploying the software system code platform provided by the present invention, and creating a docker container as a master node (master node), the master node obtains information such as test cluster environment nodes and devices as cluster initialization information, performs cluster topology discovery to obtain the latest cluster environment device information, compiles relevant code and system parameters, and generates a docker image with a unique identifier. Upload the docker image to a private warehouse image. According to the latest cluster environment information, the docker image is deployed to all cluster nodes using a unique identifier. After running docker on the cluster environment node as a slave node (slave node), the slave node uses the socket protocol to establish communication with the master node. The master obtains the slave set of all nodes, selects the qualified nodes in the cluster according to the predetermined test case requirements, and performs state synchronization, test task issuance and other operations through master-slave communication; according to the issued test tasks, the test is performed inside the slave node, including but not limited to 1-1, 1-N, N-1 and other IB protocol-based tests. After the test is completed, the slave node uploads the test data and test results to the master node. The master node waits for the test tasks of all slave nodes to be completed according to the slave set issued, and stores the collected test data and test results persistently and saves them to the log analysis service node. Finally, the background generates a test report and sends it to the designated person.

[0037] Common data center cluster topologies include: Figure 1 Generally, there are multiple clusters in a data center. Routers are used to build communication rules between different clusters, and multiple switches are used to build communication rules between servers in the same cluster. Figure 1 In the figure, 101-105 represent server devices in the cluster, 110-117 represent switch devices in the cluster, 120-121 represent router devices in the cluster, and 130 represents other clusters connected through switches.

[0038] Figure 1 In the figure, 101 indicates that 6 server devices are connected to switch 110, and the communication between these 6 servers needs to go through switch 110; similarly, if it is necessary to communicate with the 3 servers under server 102, it needs to be forwarded through switch 111, and the communication forwarding path is 101-110-111-112-102, where devices 110-112 play a forwarding role; if the server under server 101 needs to communicate with the server under server 105, the communication forwarding path is 101-110-111-120-117-116-105, where 110-111, 116, and 117 play a forwarding role, and router 120 plays a routing role. Device 121 is connected to another cluster environment. The access cluster is isolated from the existing cluster by network and cannot communicate directly. The function described in the present invention is to find a server from the cluster, deploy the software system code platform provided by the present invention, and discover all nodes, devices and other resources in the cluster through the software system code on the server. For the specific deployment process, please refer to Figure 2 .

[0039] See also Figure 3 The cluster topology testing method based on the IB network provided by the present invention includes:

[0040] Step S101: Determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server.

[0041] Figure 2 The software system code platform deployment process provided by the present invention is described in the following. One of the servers 104 is selected to deploy the software system code provided by the present invention. In order to implement the complex topology testing method based on the IB network switch of the present invention, a series of deployment operations need to be performed. The specific steps are as follows:

[0042] In step 201, after selecting a server in the cluster, the software system code platform deployment package of the present invention is transferred or copied to the root folder directory of the selected server through the network or other storage media, and the directory permission is set to read and write. The deployment package involves the installation of multiple subsystems during execution. For the specific process, refer to Figure 4 .

[0043] Step S102: triggering a cluster scan from the deployment server, recording the initial topology information in the cluster through the test software system platform and performing persistent storage.

[0044] In step 202, when the software code platform deployment package is installed, use the SSH protocol to remotely log in to the deployment server, execute the open source system command iblinkinfo / ibnetdiscover based on the IB protocol, and perform a network scan operation within the cluster. Exit the command window after the network scan is successful. If an incorrect result is returned or an error exits, analyze and troubleshoot based on the returned error information and specific execution logs.

[0045] In step 203, after the network scanning operation is correctly executed and returned, the topology information in the cluster is recorded to persistent storage (local storage file or database). This information is used as cluster initialization content for subsequent node and device operation records in the cluster. In a specific embodiment, the recorded information includes but is not limited to the following:

[0046] a) Information about all servers in the cluster, including manufacturer, model, and system type.

[0047] b) Information about all switches in the cluster, including manufacturer, model, firmware version, and number of ports.

[0048] c) Information about all routers in the cluster, including manufacturer, model, firmware version, and number of ports.

[0049] d) The information requirements of IB network devices include the manufacturer, firmware version, LID (a unique identifier for each port in an InfiniBand network), SMLID (a unique identifier for each subnet manager in an InfiniBand network), GUID (a globally unique identifier for an InfiniBand network device), and other required attributes.

[0050] Step S103: execute the test file based on the initial topology information, establish a container as a master node in the deployment server, select a cluster node to be tested as a slave node, send the image of the container to the slave node, upload the test data results of all slave nodes to the master node, and generate a test report based on the test data results.

[0051] In step 204, after the cluster initialization information is successfully obtained, the preset test file is executed based on the obtained cluster information; the specific execution process is referred to Figure 5 .

[0052] In step 205, when the test file task is executed, the software system code platform will push the test data and test results of this round to the background log server. The background code will summarize and analyze the test of this round, and then generate a test report and send it to the designated person.

[0053] As a further preferred embodiment, Figure 4 The following describes some subsystems and key modules involved in the deployment of the software system code platform provided by the present invention. The specific contents are as follows:

[0054] In step 301, the software system code deployment package is executed and the deployment is waited for completion.

[0055] In step 302, the system version is adaptively upgraded during the deployment process, including but not limited to the following: updating the system software package management source; uninstalling the old system software and installing the adapted software version; updating and upgrading the existing software version to adapt to the platform; installing applicable open source software based on the IB protocol. If necessary, the system kernel version can be upgraded.

[0056] In step 303, a Jenkins platform is deployed for visual editing and triggering test file execution (wherein the Jenkins platform can be deployed as an option).

[0057] In step 304, the software system platform of the present invention is deployed, which includes but is not limited to the following functions: distribution of test data and test files, test node screening, management of node and device status within the cluster, management of communication between master-slave nodes, management of docker image building and push, persistent storage of test content, etc.

[0058] In step 305, a docker private warehouse image is deployed for the management of docker images (the private warehouse image can be deployed as an option).

[0059] As a further preferred embodiment, Figure 5 It describes that after the deployment node executes the test file caseA, the software system code platform dynamically selects three servers in the cluster for testing based on the test method of the present invention. Figure 6 The testing process:

[0060] In step 401, the user logs in to the Jenkins platform, selects the test file to be executed and fills in the relevant necessary parameters to trigger the test task.

[0061] In step 402, after the test task is triggered, the software system code platform uses the docker software to create a container on the server according to the test file and input parameters selected by the user, and uses the docker software as the master node to manage the life cycle of this test task. The master node provides functions including but not limited to: screening test nodes, updating the status of nodes and devices in the cluster, sending test data and files, creating and destroying slave nodes to be tested, collecting test results and data, generating docker images, etc.

[0062] In step 403, after the master node is successfully created, the background code is used to automatically analyze the test file and input parameters selected by the user, and perform real-time cluster scanning operations, so that the user can obtain the latest node and device information in the cluster. This operation can avoid the failure of test task execution due to expired cluster data. After being processed by the background algorithm, the test nodes that meet the current round of test tasks are screened out; according to the characteristic values ​​of the input parameters, a docker image with a unique identifier is generated; after the docker image is created, the background code pushes the image to the deployed private warehouse image for the subsequent creation of slave containers.

[0063] In step 404, the master node establishes communication with the node to be tested through the SSH communication protocol. After the communication is established, the node to be tested uses system-related commands to pull the current round of docker images; based on the current round of docker images, use system-related commands to create a slave container on the node to be tested; create a socket channel in the slave container for master node communication.

[0064] In step 405, the master waits for all slave containers of the nodes to be tested to be created, and then attempts to establish communication with the slave using the socket protocol. After the communication is established, the test data and test files of this round are sent to the slave of the node to be tested. If the communication fails to be established, the test of this round ends and the error is reported to the Jenkins platform.

[0065] In step 406, after the master sends all the slave test data and test results, it waits for all the test nodes to complete execution. After the test is completed, all the slave nodes actively push the test data and test results to the master through the socket protocol; after the timeout expires, if the master detects that the test is not completed, it actively obtains the test data and test results of all the test nodes, ends this round of testing and reports the error to the Jenkins platform.

[0066] In step 407, after collecting the test data and test results of all nodes, the master pushes the current round of data to the log analysis platform, which filters the data and generates a test report.

[0067] In step 408, the master actively destroys the slave nodes generated by this round of testing and executes the self-destruction code.

[0068] In step 409, the Jenkins platform receives the test results and ends this round of testing.

[0069] It can be seen that the cluster topology testing method based on the IB network proposed in the present invention has the following advantages compared with the related technologies:

[0070] The present invention establishes a mature monitoring mechanism, including self-destruction of master nodes, persistent storage of test data and test results of each test node, and dynamic monitoring of the test node status in the cluster, which solves the problem that nodes or devices in large-scale clusters cannot be synchronized in real time before the test task is executed after being updated. Dynamically identify operations such as adding, removing, and replacing nodes in the cluster. According to the test task requirements, based on the latest cluster environment information, dynamically plan the screening of test nodes without manually specifying test nodes. Based on the one-click deployment and destruction test environment of docker technology, the test node is unaware of the changes in software and hardware before and after the test, realizing non-destructive testing of the test node, and avoiding the test node failure or dirty data caused by the test task affecting the operation of the cluster.

[0071] Accordingly, the present invention provides a cluster topology testing device based on an IB network in a second aspect, comprising:

[0072] A deployment unit, used to determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server;

[0073] A scanning unit, used to trigger a cluster scan from the deployment server, record the initial topology information in the cluster through the test software system platform and store it persistently;

[0074] The test unit is used to execute the test file based on the initial topology information, establish a container as a master node in the deployment server, select a cluster node to be tested as a slave node, send the image of the container to the slave node, upload the test data results of all slave nodes to the master node, and generate a test report according to the test data results.

[0075] The above-mentioned device can be implemented by the cluster topology testing method based on the IB network provided by the embodiment of the first aspect above. The specific implementation method can refer to the description in the embodiment of the first aspect, which will not be repeated here.

[0076] It is understood that the architectures, names and parameters described in the above embodiments are only examples. Those skilled in the art can also easily combine and adjust the structural features of the above embodiments according to the use requirements, and should not limit the concept of the present invention to the specific details of the above examples.

[0077] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cluster topology testing method based on an IB network, characterized in that: include: Determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server; Triggering a cluster scan from the deployment server, recording the initial topology information in the cluster through the test software system platform and performing persistent storage; Execute the test file based on the initial topology information, establish a container as a master node in the deployment server, select a cluster node to be tested as a slave node, send the image of the container to the slave node, upload the test data results of all slave nodes to the master node, and generate a test report according to the test data results; The step of deploying the test software system platform on the deployment server further includes: Execute the test software system platform code deployment package; Update system software package management sources, uninstall old system software and install adapted software versions, update and upgrade existing software versions to adapt to the platform; install open source software based on the IB protocol; Deploy the Jenkins platform for visual editing and triggering test file execution; Deploy the test software system platform; Deploy private warehouse images for container image management; Uploading the test data results of all slave nodes to the master node, and generating a test report according to the test data results, further comprising: After the master node is successfully created, the background code is used to automatically analyze the test files and input parameters selected by the user, and perform real-time cluster scanning operations. The user obtains the latest node and device information in the cluster. After being processed by the background algorithm, the test nodes that meet the current round of test tasks are screened out; according to the characteristic values ​​of the input parameters, a docker image with a unique identifier is generated; after the docker image is created, the background code pushes the image to the deployed private warehouse image for subsequent creation of slave node containers; After the master node establishes communication with the node to be tested through the SSH communication protocol, the system-related commands are used on the node to be tested to pull the current round of Docker images; based on the current round of Docker images, the system commands are used to create a slave node container on the node to be tested; a socket channel is created in the slave node container for master node communication; After the master node waits for all the nodes to be tested to be created from the node containers, it tries to establish communication with the slave nodes using the socket protocol. After the communication is established, it sends the test data and test files of this round to the nodes to be tested. If the communication fails to be established, the test ends and the error is reported to the Jenkins platform. After the master node sends all the test data and test results of the slave nodes, it waits for all the test nodes to complete execution. After the test is completed, all the slave nodes actively push the test data and test results to the master node through the socket protocol. After the timeout expires, if the master node detects that the test is not completed, it actively obtains the test data and test results of all the test nodes, ends this round of testing, and reports the error to the Jenkins platform. After collecting all node test data and test results, the master node pushes the current round of data to the log analysis platform, which filters the data and generates a test report. The master node actively destroys the slave nodes generated by this round of testing and executes the self-destruction code.

2. The cluster topology testing method based on the IB network according to claim 1, characterized in that: Deploying the Jenkins platform on the deployment server further includes: Transfer or copy the Jenkins platform deployment package to the root directory of the deployment server, and set the directory permissions to read and write.

3. The cluster topology testing method based on the IB network according to claim 1, characterized in that: The recording of the initial topology information within the cluster further includes: Record the information of all servers, switches, and routers in the cluster.

4. The cluster topology testing method based on the IB network according to claim 1, characterized in that: After uploading the test data results of all slave nodes to the master node, the method further includes: The master node actively destroys the slave nodes generated in this round of testing, and executes the self-destruction code to destroy the master node itself.

5. A cluster topology testing device based on an IB network, characterized in that: include: A deployment unit, used to determine a deployment server from the cluster to be tested, deploy a test software system platform on the deployment server, and send a test file to the deployment server; A scanning unit, used to trigger a cluster scan from the deployment server, record the initial topology information in the cluster through the test software system platform and store it persistently; A test unit, configured to execute the test file based on the initial topology information, establish a container as a master node in the deployment server, select a cluster node to be tested as a slave node, send the image of the container to the slave node, upload the test data results of all slave nodes to the master node, and generate a test report according to the test data results; The deployment unit is further used to: Execute the test software system platform code deployment package; Update system software package management sources, uninstall old system software and install adapted software versions, update and upgrade existing software versions to adapt to the platform; install open source software based on the IB protocol; Deploy the Jenkins platform for visual editing and triggering test file execution; deploying the test software system platform, Deploy private warehouse images for container image management; The testing unit is further used for: After the master node is successfully created, the background code is used to automatically analyze the test files and input parameters selected by the user, and perform real-time cluster scanning operations. The user obtains the latest node and device information in the cluster. After background algorithm processing, the test nodes that meet the current round of test tasks are screened out; based on the characteristic values ​​of the input parameters, a Docker image with a unique identifier is generated; After the Docker image is created, the background code pushes the image to the deployed private warehouse image for subsequent creation of slave node containers; After the master node establishes communication with the node to be tested through the SSH communication protocol, the system-related commands are used on the node to be tested to pull the current round of Docker images; based on the current round of Docker images, the system commands are used to create a slave node container on the node to be tested; a socket channel is created in the slave node container for master node communication; After the master node waits for all the nodes to be tested to be created from the node containers, it tries to establish communication with the slave nodes using the socket protocol. After the communication is established, it sends the test data and test files of this round to the nodes to be tested. If the communication fails to be established, the test ends and the error is reported to the Jenkins platform. After the master node sends all the test data and test results of the slave nodes, it waits for all the test nodes to complete execution. After the test is completed, all the slave nodes actively push the test data and test results to the master node through the socket protocol. After the timeout expires, if the master node detects that the test is not completed, it actively obtains the test data and test results of all the test nodes, ends this round of testing, and reports the error to the Jenkins platform. After the master node collects the test data and test results of all nodes, it pushes the current round of data to the log analysis platform, which filters the data and generates a test report; The master node actively destroys the slave nodes generated in this round of testing and executes the self-destruction code.

6. The cluster topology testing device based on IB network according to claim 5, characterized in that: The deployment unit is further used to: The test software system platform deployment package is transferred or copied to the root directory of the deployment server, and the directory permissions are set to be readable and writable.

7. The cluster topology testing device based on IB network according to claim 5, characterized in that: The scanning unit is further used for: Record the information of all servers, switches, and routers in the cluster.

8. The cluster topology testing device based on IB network according to claim 5, characterized in that: The testing unit is further used for: The master node actively destroys the slave nodes generated in this round of testing, and executes the self-destruction code to destroy the master node itself.

Images