Method, device and storage medium for automatically testing k8s cluster network quality
By defining custom network testing resources and deploying listening modules in the k8s cluster, Iperf task containers are generated, and automated testing of the network quality of the k8s cluster is achieved, which solves the problem of inability to automate testing in the existing technology, and improves testing efficiency and accuracy sex.
Patent Information
- Application Number
- CN202110182339.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-10
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-02-10
AI Technical Summary
The existing technology cannot realize automated testing of the network quality of k8s clusters, resulting in the need to manually write configuration files and repetitive redundant work, which cannot be applied to large-scale clusters.
Define custom network test resources by calling k8s' CustomResourceDefinition mechanism, deploy network test resource monitoring module, and generate Iperf task containers to automatically plan and execute network test tasks and collect test results.
It realizes automated testing of the network quality of k8s clusters, reduces manual maintenance work, and can automatically plan test tasks, deploy test tools and output results, and detect link failures and network performance, such as throughput, packet loss rate and latency.
Smart Images

Figure CN114915571B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to computer networks and cloud computing technologies, and more specifically to a method and device for automatically testing the quality of a kubernetes (abbreviated as "k8s") cluster network. Background Art
[0002] With the rapid development of cloud computing, container technology has become the mainstream and trend in the field of cloud computing. As the most mainstream container orchestration platform, k8s is also a hot topic of research.
[0003] As a container orchestration platform, the network is the cornerstone. The network quality of a k8s cluster has a direct impact on the overall performance of the cluster. Although k8s provides excellent container scheduling capabilities, if you want to directly test the network between computing nodes in a k8s cluster, you need to manually write a large number of configuration files, manually submit deployment files to control the order of link tests, and finally manually collect test data. This will consume a lot of manpower to do repetitive and redundant work, and it cannot be used for network quality testing of large-scale k8s clusters. Therefore, it is not possible to automatically test the network quality of a k8s cluster using existing technologies. Summary of the invention
[0004] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.
[0005] Therefore, in view of the problems existing in the prior art, one purpose of the present disclosure is to automatically test the network quality of the k8s cluster. Combined with the testing capabilities provided by the existing network testing tool iperf, it automatically plans test tasks, automatically deploys test components, automatically collects test results, and outputs results in a specified direction, so that the test process is fully automated and manual maintenance work is reduced.
[0006] According to one aspect of the present disclosure, a method for automatically testing the network quality of a k8s cluster is provided, comprising: customizing network test resources for a network test task by calling a resource customization module provided by k8s, wherein the customized network test resources have parameters for configuring test parameters, test execution methods, and test result output directions of an iperf tool; customizing a network test resource monitoring module and deploying it to a k8s cluster; and in response to monitoring the submission of the customized network test resource, generating an iperf task container by the customized network test resource monitoring module to execute the network test task using the iperf tool in the iperf task container.
[0007] According to another aspect of the present disclosure, a device for automatically testing the network quality of a k8s cluster is provided, comprising: a memory on which instructions are stored; and a processor configured to execute the instructions stored on the memory to execute the method described above.
[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, comprising computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the above-described method for automatically testing the quality of a k8s cluster network. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0010] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:
[0011] Figure 1 A flowchart of a method for automatically testing the network quality of a k8s cluster according to an embodiment of the present invention is shown;
[0012] Figure 2 A flowchart showing a process of using iperf to plan and execute an entire test task according to an embodiment of the present invention is shown;
[0013] Figure 3 A schematic diagram showing a process of automatically testing the quality of a k8s cluster network using iperf in a k8s cluster network having four nodes according to an embodiment of the present invention;
[0014] Figure 4 An exemplary configuration of a computing device that can implement an embodiment according to the present disclosure is shown. DETAILED DESCRIPTION
[0015] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and not to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.
[0016] The inventors of the present invention recognize the technical problems existing in the prior art. In addition, the inventors also found that Iperf is a network performance testing tool. Iperf can test TCP and UDP bandwidth quality. Iperf can measure the maximum TCP bandwidth with multiple parameters and UDP characteristics. Iperf can report bandwidth, delay jitter and packet loss. Using this feature of Iperf, it can be used to test the performance of some network devices such as routers, firewalls, switches, etc. The present invention contemplates that Iperf can be used to automatically perform k8s cluster network quality testing. However, the Iperf tool cannot be directly deployed to the k8s cluster, nor can it be directly called by the master node in k8s. To this end, the inventors of the present invention proposed this technical solution to enable the iperf tool to be automatically called to automatically perform k8s cluster network quality testing. More specifically, this technical solution can:
[0017] 1. Automatically test the k8s cluster network, automatically plan the network test task sequence, automatically deploy test tools, and automatically collect test results and output them in the specified direction; and
[0018] 2. Check the failure of the k8s cluster network link, and also test the network performance of each link of the k8s cluster network, such as throughput, packet loss rate, delay, etc.
[0019] The technical solution according to the present invention includes the following three elements:
[0020] (1) Customized network test resources: Based on the CustomResourceDefinition mechanism provided by k8s, define customized network test resources. The parameters of this resource are used to configure the test parameters of the iperf tool, the test execution method, and the test result output direction.
[0021] (2) Custom network test resource listener Operator: Operator is a program that listens to the submission of custom network test resources. It can be deployed to k8s in the form of Deployment, and can be scheduled by k8s at will or by the user-specified scheduling node. After listening to the submission of new custom network test resources, Operator will generate an IperfTask pod and schedule it to a k8s node, which will plan and execute this test task.
[0022] (3) IperfTask pod: It is used to automatically plan the test plan according to the parameters of the specified custom network test resources, automatically schedule and deploy the iperf client and server pods, complete the network test tasks between k8s nodes, collect all test data, and output the results to the specified output direction.
[0023] Figure 1 A flowchart of a method 100 for automatically testing the network quality of a k8s cluster according to an embodiment of the present invention is shown.
[0024] At step 101, a resource customization module provided by k8s is called to customize network test resources for a network test task, wherein the customized network test resources have parameters for configuring test parameters, test execution methods, and test result output directions of an iperf tool.
[0025] In this step, according to the CustomResourceDefinition mechanism provided by k8s, the custom network test resource definition is submitted to the k8s cluster so that k8s can recognize the custom network test resource object.
[0026] Just as an example, here is a custom network test resource definition:
[0027] apiVersion:apiextensions.k8s.io / v1beta1
[0028] kind:CustomResourceDefinition
[0029] metadata:
[0030] name:iperftasks.iperf.test.svc
[0031] spec:
[0032] group:iperf.test.svc
[0033] version:alpha1
[0034] names:
[0035] kind:IperfTask
[0036] listKind:IperfTaskList
[0037] plural:iperftasks
[0038] singular:iperftask
[0039] scope:Namespaced
[0040] Just as an example, here is an example of a custom network test resource:
[0041] apiVersion:iperf.test.svc / alpha1
[0042] kind:IperfTask
[0043] metadata:
[0044] name:my-tcp-iperf
[0045] namespace:iperf-operator
[0046] spec:
[0047] iperfImage:networkstatic / iperf3
[0048] toEmail:xxxxx@mail.com
[0049] serverSpec:
[0050] port:9000
[0051] clientSpec:
[0052] mode:"fast"
[0053] interval:10
[0054] duration:60
[0055] At step 102, customize the network test resource monitoring module and deploy it to the k8s cluster.
[0056] In this step, deploy the network test resource listening Operator program to the k8s cluster. The Operator program will keep running to listen for submissions of network test resources.
[0057] An example of an Operator program according to the present invention is given below.
[0058] 1. Deploy IperfTask CRD resources. IperfTask is used to describe a test task:
[0059] kubectl apply-f. / deploy / crd / ipefTask.yaml
[0060] 2. Create a namespace and mail server information secret:
[0061] Note that the k8s secret string requires base64 encoding
[0062]
[0063] 3. Deploy IperfOperator, which is responsible for monitoring IperfTask and executing tasks: kubectlapply-f. / deploy / iperf_operator.yaml
[0064] 4. Deploy an IperfTask test task (by default, TCP test bandwidth is executed):
[0065]
[0066]
[0067] Among them, interval is equal to the -i parameter of Iperf3, which is the test interval; duration is equal to the test duration of iperf3; and mode is divided into fast and low modes. The low mode is a complete point-to-point test of the node, and the fast mode is a fast point-to-point test of the node.
[0068] 5. To deploy the udp test delay and packet loss rate task execution:
[0069]
[0070] Among them, bwLimit is the parameter for limiting udp bandwidth, which is equal to the -b parameter of iperf3.
[0071] At step 103, in response to monitoring the submission of the custom network test resource, the custom network test resource monitoring module generates an iperf task container to execute the network test task using the iperf tool in the iperf task container.
[0072] In this step, when you need to start the test, submit a configured custom network test resource to the k8s cluster. After the operator listens to the new custom network test resource object, it generates and schedules an IperfTask pod to plan and execute the entire test task.
[0073] Figure 2 A flowchart of a process 200 of using iperf to plan and execute an entire test task according to an embodiment of the present invention is shown.
[0074] At S0 , process 200 begins.
[0075] At S1, IperTask will parse the custom network resource object passed to it, and parse out the iperf test parameters, such as tcp / udp, throughput, bandwidth, test duration, output interval and format, and other parameters that control the execution of the test, such as the execution mode.
[0076] At S2, IperfTask will request information about all computing nodes from the current k8s cluster, such as IP, node name, node label, etc.
[0077] At S3, IperfTask deploys the iperf server program to each computing node, obtains the IP of each iperf server pod after successful deployment, and monitors the changes of IP at all times to replace the original IP.
[0078] At S4, IperfTask plans the test execution plan based on the custom test resources and computing node information, and puts all execution tasks into the execution plan queue.
[0079] At S5, IperfTask takes out the execution tasks of the current batch from the execution plan queue.
[0080] At S6, if a batch of execution tasks can be taken out, all test tasks in the batch are executed in parallel. The test links of the network test tasks in the same batch should not affect each other. iperfTask schedules the iperf client program in the execution task to the target node, configures the iperf server IP and iperf test parameters. After the iperf client completes the test process, it reports the test results of the current test link to iperfTask. After completing all the test tasks in this batch, request the next batch of test tasks from the task queue.
[0081] At S7, if there are no remaining test tasks in the task queue, all inter-node link tests are completed. IperfTask calculates the statistics of all collected test result data and outputs them to the specified output direction, such as sending emails, storing results in a database, etc.
[0082] At S8, after IperfTask completes all work, it cleans up the resources used in this test and then exits.
[0083] At S9 , the process 200 ends.
[0084] Figure 3 A schematic diagram of a process of automatically testing the quality of a k8s cluster network using iperf in a k8s cluster network having four nodes according to an embodiment of the present invention is shown. The following takes a k8s cluster including four nodes as an example to illustrate the automatic testing of the quality of a k8s cluster network.
[0085] 1: According to the CustomResourceDefinition mechanism provided by k8s, define a custom network test resource with kind as CustomResourceDefinition, as shown above. Submit the custom network test resource definition to the k8s cluster by calling the kubernetes API Server interface through the kubectl tool or HTTP request, so that k8s can recognize the custom network test resource object.
[0086] 2: Deploy the Operator program in Deployment mode, grant the Operator pod query and scheduling permissions for k8s resources, and the pod of the Operator program can be freely scheduled by k8s, or the user can specify the scheduling node by configuring NodeSelector, Affinity or Anti-Affinity information during deployment. Submit the deployment Operator information to the k8s cluster through the kubectl tool or directly request the kubernetes API Server interface through HTTP. After the Operator program is scheduled to run, it will call the k8s API Server WATCH API interface to listen to the custom network test resource objects submitted by the user to k8s.
[0087] 3: When the test needs to start, the user writes a custom network test resource, sets the iperf test parameters, task execution mode, result output direction and other information, as shown in Example 2. Submit the custom network test resource information to the k8s cluster through the kubectl tool or directly call the k8sAPI Server interface in HTTP mode. After the Operator listens to the custom network test resource object through the k8s API ServerWATCH API interface, it generates the IperfTask Deployment deployment information, calls the k8s API Server interface to deploy IperfTask, and sets the namespace and name information of the custom network test resource to be executed by IperfTask. The IperfTask pod is responsible for planning and executing the entire k8s cluster network test task.
[0088] S1: IperTask calls the k8s API Server interface to obtain the custom network test resource object according to the namespace and name of the custom network test resource. And the parser will parse the custom network resource object passed to it. Parse the iperf test parameters, such as tcp / udp, throughput, bandwidth, test duration, output interval and format, etc., and some other parameters that control the test execution, such as execution mode, result output direction, etc.
[0089] S2: IperfTask calls the k8s API Server interface to request information about all computing nodes in the current k8s cluster, such as IP, node name, node label, etc.
[0090] S3: IperfTask deploys the iperf server program to each computing node. IperfTask can deploy the iperf server using the k8s DaemonSet method or directly using the pod method of the specified node. IperfTask uses the k8s API Server interface to monitor the deployment process, obtain the IP of each successfully deployed iperf server pod, and monitor the changes of the IP at all times to replace the original IP.
[0091] S4: IperfTask plans the test execution plan based on the custom test resources and computing node information, and puts all execution tasks into the execution plan queue. An execution task includes the target node information to be deployed by the iperf client, the target iperf server information, iperf test parameters and other information. A batch test task consists of one or more test tasks. If a batch test task has only one test task, it is a serial execution mode. If there are multiple link test tasks that do not affect each other, it is a parallel mode. If there are no duplications between the nodes of multiple links, they can be considered to be independent of each other and can be executed in parallel. For example, for the four nodes node1, node2, node3, and node4, the two link tests from node1 to node2 and from node3 to node4 can be considered to be independent of each other and can be executed in parallel, and the same is true for direction tests.
[0092] S5: IperfTask takes out the execution tasks of the current batch from the execution plan queue.
[0093] S6: If a batch of execution tasks can be taken out, execute the test tasks in the batch in serial or parallel. iperfTask schedules the iperf client program in the execution task to the target node. iperfTask deploys an iperf client program using the k8s Job type and specifies the target node, and configures the target iperf server IP and iperf test parameters. After the iperf client completes the test process, it reports the test results of the current test link to iperfTask. After completing all the test tasks in this batch, iperfTask requests the next batch of test tasks from the task queue.
[0094] S7: If there are no remaining test tasks in the task queue, all inter-node link tests are completed. IperfTask calculates the statistics of all collected test result data and outputs them to the specified output direction, such as sending emails, storing results in a database, etc.
[0095] S8: After IperfTask completes all work, it cleans up the resources used in this test and exits. All k8s resources generated by IperfTask will set the referrence bit of IperfTask's id in metadata, indicating that it is an IperfTask sub-resource. As long as IperfTask exits, the k8s cluster will cascade delete these sub-resources to achieve the purpose of clearing. Compared with the prior art, the present invention has one or more of the following advantages and effects:
[0096] 1. Automatically test the k8s cluster network, automatically plan the execution plan, automatically deploy the test tools, and automatically collect the test results and output them in the specified direction.
[0097] 2. Using the existing network testing tools, you can test whether the network between k8s cluster nodes is connected or has faults, and you can also test the network's throughput, packet loss rate, latency and other performance data.
[0098] Figure 4 An exemplary configuration of a computing device 1200 capable of implementing embodiments according to the present disclosure is shown.
[0099] The computing device 1200 is an example of a hardware device to which the above aspects of the present disclosure can be applied. The computing device 1200 can be any machine configured to perform processing and / or computing. The computing device 1200 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smart phone, a vehicle-mounted computer, or a combination thereof.
[0100] like Figure 4 As shown, computing device 1200 may include one or more elements that can be connected or communicated with bus 1202 via one or more interfaces. Bus 1202 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, etc. Computing device 1200 may include, for example, one or more processors 1204, one or more input devices 1206, and one or more output devices 1208. One or more processors 1204 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). Processor 1204 may be configured to implement an automatic test method according to the present invention, for example. Input device 1206 may be any type of input device capable of inputting information to a computing device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. Output device 1208 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer.
[0101] The computing device 1200 may also include or be connected to a non-transitory storage device 1214, which may be any non-transitory storage device that can implement data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a tape, or any other magnetic medium, a compressed disk or any other optical medium, a cache memory, and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions, and / or code. The computing device 1200 may also include a random access memory (RAM) 1210 and a read-only memory (ROM) 1212. The ROM 1212 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 1210 may provide volatile data storage and store instructions related to the operation of the computing device 1200. The computing device 1200 may also include a network / bus interface 1216 coupled to a data link 1218. The network / bus interface 1216 may be any type of device or system capable of enabling communication with an external device and / or network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth TM equipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.).
[0102] The present disclosure may be implemented as any combination of an apparatus, a system, an integrated circuit, and a computer program on a non-transitory computer-readable medium. One or more processors may be implemented as an integrated circuit (IC), an application specific integrated circuit (ASIC), or a large scale integrated circuit (LSI), a system LSI, a super LSI, or a super LSI component that performs some or all of the functions described in the present disclosure.
[0103] The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, applications, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to enable a computer such as one or more processors to perform the above steps and the steps described in the accompanying drawings. For example, one or more memories store the software or algorithm in executable instructions, and one or more processors may associate a set of instructions to execute the software or algorithm to provide various functions according to the embodiments described in the present disclosure.
[0104] Software and computer programs (which may also be referred to as programs, software applications, applications, components or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or an assembly language or a machine language. The term "computer-readable medium" refers to any computer program product, apparatus or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.
[0105] For example, computer readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer readable program code in the form of instructions or data structures and can be accessed by a general or special purpose computer or a general or special purpose processor. As used herein, disk or disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically by laser. Combinations of the above are also included in the scope of computer readable media.
[0106] The subject matter of the present disclosure is provided as an example of devices, systems, methods and programs for performing the features described in the present disclosure. However, in addition to the above-mentioned features, other features or variations may also be expected. It is expected that the implementation of the components and functions of the present disclosure may be completed with any emerging technology that may replace any of the above-mentioned implementation technologies.
Claims
1. A method for automatically testing the network quality of a k8s cluster. include: Customize network test resources for network test tasks by calling the resource customization module provided by k8s, wherein the customized network test resources have parameters for configuring the test parameters, test execution method, and test result output direction of the iperf tool; Customize the network test resource monitoring module and deploy it to the k8s cluster; as well as In response to monitoring the submission of a custom network test resource, the custom network test resource monitoring module generates an iperf task container to use the iperf tool in the iperf task container to execute the network test task, wherein the iperf task container deploys an iperf server program to each computing node, and schedules the iperf client program in executing the network test task to a target node, and the iperf task container and the custom network test resource monitoring module are located in different computing nodes; The iperf tool automatically plans a test plan for the network test task according to the parameters of the customized network test resources, automatically schedules and deploys the iperf client program and the iperf server program, so that the iperf client program completes the network test between the k8s nodes and reports the test results of the test link to the iperf tool after completing the network test. The iperf tool collects all the test results and outputs the test results to the specified output direction. Wherein, in the case of multiple link test tasks that do not affect each other, the iperf tool executes the multiple network test tasks in parallel.
2. The method according to claim 1, in, The iperf tool automatically tests the network connectivity and point-to-point network bandwidth between pairs of nodes in the k8s cluster and sends the result report via email.
3. The method according to claim 2, in, The custom network test resource monitoring module is defined by the following operations: Deploy IperfTask CRD resources. IperfTask is used to describe a test task. Create a namespace and mail server information, Deploy IperfOperator, which is responsible for monitoring IperfTask and executing tasks. Deploy an IperfTask test task, which performs TCP test bandwidth by default, and Deploy UDP test delay and packet loss rate tasks.
4. A device for automatically testing the quality of k8s cluster network, include: a memory having instructions stored therein; as well as A processor configured to execute instructions stored on the memory to perform the method according to any one of claims 1 to 3.
5. A computer-readable storage medium, comprising computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the method for automatically testing the quality of a k8s cluster network according to any one of claims 1 to 3.
