Computing cluster

Abstract

The invention relates to a computing cluster. The computing cluster comprises a management platform, a Kubernetes cluster and a high-performance computing cluster, and the Kubernetes cluster comprises a master control Kubernetes node and a plurality of common Kubernetes nodes. Wherein the management platform is used for issuing an instruction of newly adding the high-performance computing node to the master control Kubernetes node under the condition of receiving a node newly adding request of the high-performance computing cluster. And the master control Kubernetes node is used for responding to the instruction of the newly added high-performance computing node and starting a target Kubernetes node, and the target Kubernetes node is an idle common Kubernetes node. And the target Kubernetes node is used for generating a high-performance computing node configuration file and requesting the high-performance computing cluster to be added to the high-performance computing cluster. According to the computing cluster provided by the invention, the high-performance computing cluster and the K8S cluster are fused as one computing cluster, and the operation efficiency of the high-performance computing cluster and the K8S cluster is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to a computing cluster. Background Art

[0002] Due to the characteristics of high-performance computing clusters and Kubernetes (K8S for short) clusters, each has its own advantages and disadvantages and application scenarios. High-performance computing clusters can handle parallel tasks and handle high performance with job scheduling strategies, but they cannot provide rich container management functions such as image pulling, container management, and load balancing like K8S; while K8S clusters can be elastically shrunk and containers can be built as needed, which has a good elasticity advantage, but it cannot apply job scheduling strategies and does not support high-performance computing scenarios. However, with the development of artificial intelligence, some research requires parallel computing and also requires ordinary computing such as AI, so computing clusters are needed to support mixed loads of parallel computing (high-performance computing) and AI (ordinary computing) at the same time.

[0003] In the related technology, the high-performance computing cluster and the K8S cluster are integrated. When the high-performance computing cluster and the K8S cluster are integrated, there are mixed computing nodes. The mixed computing nodes do not directly belong to the high-performance computing cluster or the K8S cluster. The mixed computing nodes are deployed with two cluster configuration files and two node images. When the number of nodes in the high-performance computing cluster or the K8S cluster is insufficient, these mixed computing nodes will be called. Therefore, this will cause scheduling conflicts between the high-performance computing cluster and the K8S cluster, and will also affect their respective operating efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a computing cluster that can avoid scheduling conflicts and improve operating efficiency in response to the above technical problems.

[0005] The present application provides a computing cluster, which includes a management platform, a Kubernetes cluster, and a high-performance computing cluster. The Kubernetes cluster includes a master Kubernetes node and multiple common Kubernetes nodes, wherein:

[0006] The management platform is used to send an instruction to add a high-performance computing node to the master Kubernetes node when receiving a request to add a node to the high-performance computing cluster;

[0007] The master Kubernetes node is used to start the target Kubernetes node in response to the instruction of the newly added high-performance computing node; the target Kubernetes node is an idle common Kubernetes node;

[0008] The target Kubernetes node is used to generate a high-performance computing node configuration file and request the high-performance computing cluster to join the high-performance computing cluster.

[0009] In this embodiment, the high-performance computing cluster and the K8S cluster are integrated into one computing cluster, and the elasticity of the K8S cluster is utilized to elastically add computing nodes to the high-performance computing cluster. On the one hand, the computing cluster can perform high-performance computing through the included high-performance computing cluster, and at the same time, it can perform ordinary computing tasks through the included K8S cluster. On the other hand, each node in the computing cluster has a clear cluster to which it belongs, and there will be no situation where two clusters compete for one node. The high-performance computing cluster and the K8S cluster do not affect each other during operation, and each will not affect the operating efficiency of the other. In addition, by utilizing the elasticity of the K8S cluster, when the high-performance computing cluster is not sufficient to run high-performance computing, the number of nodes in the high-performance computing cluster is elastically increased to improve the computing power and operating efficiency of the high-performance computing cluster to meet the elastic scaling of the high-performance computing cluster.

[0010] In one of the embodiments, the master Kubernetes node is used to start the target Kubernetes node, obtain the high-performance computing node image from the image warehouse and copy it to the target Kubernetes node.

[0011] In this embodiment, the environment deployment of the target Kubernetes node is quickly completed through the image repository, thereby improving the efficiency of adding new high-performance computing nodes.

[0012] In one of the embodiments, the high performance computing cluster includes a master high performance computing node and a common high performance computing node;

[0013] After the target Kubernetes node is started, it is used to request the authentication key from the master high-performance computing node and start the high-performance computing cluster authentication service.

[0014] In this embodiment, an authentication program is added to the target Kubernetes node so that the target Kubernetes node, as a high-performance computing node, can communicate with each high-performance computing node of the original high-performance computing cluster more securely and quickly.

[0015] In one of the instances, the target Kubernetes node is used to generate a high-performance computing node configuration file, and then send the node information of the target Kubernetes node to the high-performance computing cluster, so that the high-performance computing cluster adds the target Kubernetes node as a high-performance computing node to the high-performance computing cluster.

[0016] In this embodiment, the target Kubernetes node sends the node information of the node to the high-performance computing cluster, so that the high-performance computing cluster issues corresponding job tasks based on the node information of the target Kubernetes node when performing job scheduling.

[0017] In one of the examples, each of the common Kubernetes nodes is pre-divided into a container node set and an elastic high-performance computing node set;

[0018] The master Kubernetes node is used to respond to the instruction of adding high-performance computing, select a Kubernetes node as the target Kubernetes node in the elastic high-performance computing node set and start it.

[0019] In this embodiment, the role of each node in the Kubernetes cluster is pre-divided to reduce scheduling failures caused by insufficient number of nodes in the Kubernetes cluster itself.

[0020] In one of the embodiments, the management platform is used to send an instruction to add a new high-performance computing cluster to the master Kubernetes node when receiving a request to add a new high-performance computing cluster;

[0021] The master Kubernetes node is used to respond to the newly added instruction of the newly added high-performance computing cluster and start a number of Kubernetes nodes based on the node quantity information carried by the instruction;

[0022] The master Kubernetes node is used to obtain the image of the high-performance computing node in the image warehouse and copy it to each of the Kubernetes nodes, use one of the Kubernetes nodes as an elastic master high-performance computing node, and use the other Kubernetes nodes as elastic common high-performance computing nodes;

[0023] The elastic master high-performance computing node is used to generate a master configuration file;

[0024] The elastic common high-performance computing node is used to generate an operation configuration file and request the elastic master high-performance computing node to join the elastic high-performance computing cluster.

[0025] In this embodiment, when the management platform receives a request to create a new high-performance computing cluster, it controls the K8S cluster to start a batch of nodes and configure them to obtain a new, independent high-performance computing cluster.

[0026] In one of the embodiments, each of the common Kubernetes nodes is pre-divided into a container node set and an elastic node set;

[0027] The master Kubernetes node responds to the newly added instruction of the newly added high-performance computing cluster, selects and starts several Kubernetes nodes in the elastic node set based on the node quantity information carried by the instruction.

[0028] In this embodiment, the role of each node in the Kubernetes cluster is pre-divided to reduce scheduling failures caused by insufficient number of nodes in the Kubernetes cluster itself.

[0029] In one of the embodiments, the elastic common high-performance computing node is used to request an authentication key from the elastic master high-performance computing node after startup, and start a high-performance computing cluster authentication service.

[0030] In this embodiment, an authentication procedure is added to each elastic high performance computing node to enable the elastic high performance computing nodes to communicate with each other more securely and quickly.

[0031] In one of the embodiments, the elastic ordinary high-performance computing node is used to generate a high-performance computing node configuration file, and then send the node information of the target Kubernetes to the elastic master high-performance computing node, so that the elastic master high-performance computing node adds the elastic ordinary high-performance computing node as a high-performance computing node to the high-performance computing cluster.

[0032] In this embodiment, each elastic ordinary high-performance computing node sends the node information of the node to the elastic master high-performance computing node, so that the elastic master high-performance computing node can issue corresponding job tasks based on the node information of each elastic ordinary high-performance node when performing job scheduling.

[0033] In one of the embodiments, the master Kubernetes node is used to monitor the first Kubernetes node of each high-performance computing node; when it is monitored that the length of time the first Kubernetes node is in an idle state reaches a recycling condition, the data information of the first Kubernetes node is cleared and the first Kubernetes node is recycled.

[0034] In this embodiment, when it is determined that the high-performance computing cluster resources are sufficient, the Kubernetes nodes that are in an idle state and converted into high-performance computing nodes are recovered.

[0035] In one of the embodiments, the management platform is used to submit the job to the master high-performance computing node or the master Kubernetes node according to the job type of the job when receiving the job.

[0036] In this embodiment, the management platform can send jobs to a high-performance computing cluster or a K8S cluster based on different job types.

[0037] In one of the embodiments, the high performance computing node is a Slurm node, and the high performance computing cluster is a Slurm cluster.

[0038] The above-mentioned computing cluster integrates the high-performance computing cluster and the K8S cluster as one computing cluster, and uses the elasticity advantage of the K8S cluster to elastically add computing nodes to the high-performance computing cluster. On the one hand, the computing cluster can perform high-performance computing through the included high-performance computing cluster, and at the same time, it can perform ordinary computing tasks through the included K8S cluster. On the other hand, each node in the computing cluster has a clear cluster to which it belongs, and there will be no situation where two clusters compete for one node. The high-performance computing cluster and the K8S cluster do not affect each other during operation, and each will not affect the operating efficiency of the other. In addition, by using the elasticity advantage of the K8S cluster, when the high-performance computing cluster is not enough to run high-performance computing, the number of nodes in the high-performance computing cluster is elastically increased to improve the computing power and operating efficiency of the high-performance computing cluster to meet the elastic scaling of the high-performance computing cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the composition of a computing cluster in one embodiment;

[0040] Figure 2 A schematic diagram of a cluster scheduling method in one embodiment;

[0041] Figure 3 A schematic diagram of a flow chart of a cluster scheduling method in another embodiment;

[0042] Figure 4 A schematic diagram of a flow chart of a cluster scheduling method in another embodiment;

[0043] Figure 5 A schematic diagram of a flow chart of a cluster scheduling method in another embodiment;

[0044] Figure 6 A schematic diagram of a flow chart of a cluster scheduling method in another embodiment;

[0045] Figure 7 A schematic diagram of the composition of a computing cluster in another embodiment;

[0046] Figure 8A schematic diagram of a process for expanding a Slurm cluster in one embodiment;

[0047] Fig. 9 A schematic diagram of a process of creating a new Slurm cluster in one embodiment;

[0048] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The units that make up a high-performance computing cluster are generally virtual machines (high-performance computing nodes), which have a complete physical environment and operating system. After the high-performance computing cluster receives the computing tasks submitted by the user, it reasonably allocates the tasks to each computing node through the job scheduling strategy.

[0051] Kubernetes, or K8s for short, is an abbreviation that uses "8" to replace the eight characters in the middle of the name "ubernete". It is a container orchestration system. The units that make up a K8S cluster are generally containers. There is no need to install a host operating system. Containers can be directly installed on the host operating system, so a node can run multiple containers. After receiving computing tasks submitted by users, the K8S cluster dynamically adjusts the number of containers according to the tasks, and containers can be built and used as needed.

[0052] High-performance computing clusters and K8S clusters each have their own advantages in computing. However, as the business develops, some business calculations require the advantages of both clusters. They need both the job scheduling advantages of high-performance computing clusters to provide high-performance computing, and the elastic scaling advantages of K8S clusters to build and use containers as the business evolves.

[0053] In related technologies, in order to achieve the integration of two clusters, the nodes of the two computing clusters are directly mixed, that is, some computing nodes are mixed by the two clusters, and the mixed computing nodes are deployed with two cluster configuration files and two node images. Among them, the hybrid node does not directly belong to the high-performance computing cluster or the K8S cluster, or it can be said that it belongs to both the high-performance computing cluster and the K8S cluster. When the number of nodes in the high-performance computing cluster is insufficient, the hybrid node can be called to run some computing tasks. When the number of nodes in the K8S cluster is insufficient, the hybrid node can be called to run some computing tasks. This causes scheduling conflicts between the high-performance computing cluster and the K8S cluster, that is, two calculations call the same hybrid node at the same time, and if the same node runs the computing tasks of the two clusters at the same time, the operation efficiency is low, which will affect the operation efficiency of the two clusters.

[0054] Based on this, the present application proposes a computing cluster, including a management platform, a Kubernetes cluster, and a high-performance computing cluster, wherein the Kubernetes cluster includes a master Kubernetes node and multiple ordinary Kubernetes nodes. Among them: the management platform is used to issue an instruction to add a new high-performance computing node to the master Kubernetes node when receiving a request to add a node to the high-performance computing cluster. The master Kubernetes node is used to start the target Kubernetes node in response to the instruction to add a new high-performance computing node, wherein the target Kubernetes node is an idle ordinary Kubernetes node. The target Kubernetes node is used to generate a high-performance computing node configuration file and request the high-performance computing cluster to join the high-performance computing cluster.

[0055] The computing cluster provided in this application integrates the high-performance computing cluster and the K8S cluster into one computing cluster, and utilizes the elasticity advantage of the K8S cluster to elastically add computing nodes to the high-performance computing cluster. On the one hand, the computing cluster can perform high-performance computing through the included high-performance computing cluster, and at the same time, it can perform ordinary computing tasks through the included K8S cluster. On the other hand, each node in the computing cluster has a clear cluster to which it belongs, and there will be no situation where two clusters compete for one node. The high-performance computing cluster and the K8S cluster do not affect each other during operation, and each will not affect the operating efficiency of the other. In addition, by utilizing the elasticity advantage of the K8S cluster, when the high-performance computing cluster is not sufficient to run high-performance computing, the number of nodes in the high-performance computing cluster is elastically increased to improve the computing power and operating efficiency of the high-performance computing cluster to meet the elastic scaling of the high-performance computing cluster.

[0056] Next, the computer cluster provided by this application is described in detail.

[0057] In one embodiment, Figure 1 As shown, a computing cluster is provided, which includes a management platform, a high-performance computing cluster and a Kubernetes cluster. The Kubernetes cluster includes a master Kubernetes node and multiple common Kubernetes nodes.

[0058] The high-performance computing cluster in the computing cluster is used to execute the high-performance computing jobs received by the computing cluster. High-performance computing jobs are generally parallel computing jobs, which require the high-performance computing cluster to dispatch multiple high-performance computing nodes to complete a high-performance computing job. The K8S cluster in the computing cluster is used to execute ordinary computing jobs such as AI received by the computing cluster. Ordinary computing jobs have less computing volume. K8S generally allocates one container for one job. When the job needs it, the container is created. After the job is completed, the container is destroyed.

[0059] The management platform in the computing cluster is used to receive various jobs submitted by various users and submit the jobs to the high-performance computing cluster or Kubernetes cluster of the computing cluster according to the job type.

[0060] In one embodiment, after the high-performance computing cluster receives a high-performance computing job, if the number of computing nodes in the high-performance computing cluster is sufficient, then when the high-performance computing cluster is idle, it calls a corresponding number of high-performance computing nodes to run the job; if the number of computing nodes in the high-performance computing cluster is insufficient, that is, the computing nodes included in the high-performance computing cluster are insufficient to run the high-performance computing job, the high-performance computing cluster sends a node addition request to the management platform.

[0061] After receiving the node addition request of the high-performance computing cluster, the management platform of the computing cluster sends an instruction to add a high-performance computing node to the master Kubernetes node according to the node request quantity carried in the node addition request of the high-performance computing cluster.

[0062] After the master Kubernetes node of the computing cluster receives the instruction to add a high-performance computing node sent by the management platform, it responds to the instruction to add a high-performance computing node, selects a Kubernetes node from each idle ordinary Kubernetes node as the target Kubernetes node and starts it.

[0063] After the target Kubernetes node of the computing cluster is started, the target Kubernetes node generates a high-performance computing node configuration file to complete the necessary configuration of the target Kubernetes node to be converted into a high-performance computing node. After that, the target Kubernetes node requests the high-performance computing cluster to join the high-performance computing cluster to complete the node addition request of the high-performance computing cluster.

[0064] The high-performance computing node configuration file is the configuration information required to run the high-performance computing job, including the operating environment, device information, etc., so that the high-performance computing cluster can obtain the relevant information of the high-performance computing node through the high-performance computing node configuration file for scheduling. For the high-performance computing cluster as a Slurm cluster, the node configuration file of the Slurm cluster is Slurm_node.conf. The Slurm_node.conf file contains a lot of configuration information of the node, such as the node name, IP address, architecture type, number of CPU cores, memory capacity, GPU type and number, etc. The details of the high-performance computing node configuration file can be referred to the description in the relevant technology, and will not be described in detail here.

[0065] like Figure 2 , which is a flow chart of the cluster scheduling method shown in the present application.

[0066] Step 201: During the job scheduling process, the high performance computing cluster has insufficient computing resources and sends a node addition request to the management platform.

[0067] The node addition request may include the number of newly added nodes.

[0068] Step 202: In response to the node addition request of the high-performance computing cluster, the management platform sends an instruction to add a high-performance computing node to the master Kubernetes node in Kubernetes.

[0069] Step 203: The master Kubernetes node responds to the instruction of adding a new high-performance computing node, selects a node from each idle ordinary Kubernetes node as the target Kubernetes node and starts it.

[0070] Step 204: The target Kubernetes node generates a high-performance computing node configuration file and sends a node joining request to the high-performance computing cluster.

[0071] Step 205: The high-performance computing cluster responds to the node joining request of the target Kubernetes node, adds the target Kubernetes node to the high-performance computing cluster as a high-performance computing node, and completes the expansion of the high-performance computing cluster.

[0072] In this embodiment, the high-performance computing cluster and the K8S cluster are integrated into one computing cluster, and the elasticity of the K8S cluster is utilized to elastically add computing nodes to the high-performance computing cluster. On the one hand, the computing cluster can perform high-performance computing through the included high-performance computing cluster, and at the same time, it can perform ordinary computing tasks through the included K8S cluster. On the other hand, each node in the computing cluster has a clear cluster to which it belongs, and there will be no situation where two clusters compete for one node. The high-performance computing cluster and the K8S cluster do not affect each other during operation, and each will not affect the operating efficiency of the other. In addition, by utilizing the elasticity of the K8S cluster, when the high-performance computing cluster is not sufficient to run high-performance computing, the number of nodes in the high-performance computing cluster is elastically increased to improve the computing power and operating efficiency of the high-performance computing cluster to meet the elastic scaling of the high-performance computing cluster.

[0073] In one embodiment, the master Kubernetes node in the computing cluster is used to start the target Kubernetes node, obtain the high-performance computing node image from the image repository and copy it to the target Kubernetes node, and complete the environment initialization of the target Kubernetes node, including the system, memory and other information required by the high-performance computing node, and the relevant configuration files for the master Kubernetes node to control the target Kubernetes node.

[0074] Taking Kubernetes node virtualization implemented by KubeVirt as an example, the master Kubernetes node needs to obtain and copy the relevant image files for Kubernetes node virtualization in the image repository, such as libvirt, qemu, etc. After the copy is completed, the master Kubernetes node can schedule the target Kubernetes node as a container, but the target Kubernetes node is essentially a virtualized node, not a container.

[0075] At this time, if Figure 3 As shown, the cluster scheduling method is shown in steps 301 to 306:

[0076] Step 301: During the job scheduling process, the high performance computing cluster has insufficient computing resources and sends a request for adding a node to the management platform.

[0077] The node addition request may include the number of newly added nodes.

[0078] Step 302: In response to the node addition request of the high-performance computing cluster, the management platform sends an instruction to add a high-performance computing node to the master Kubernetes node in Kubernetes.

[0079] Step 303: The master Kubernetes node responds to the instruction of adding a new high-performance computing node, selects a node from each idle ordinary Kubernetes node as the target Kubernetes node and starts it.

[0080] Step 304: After the master Kubernetes node starts the target Kubernetes node, it obtains the high-performance computing node image from the image repository and copies it to the target Kubernetes node.

[0081] Step 305: The target Kubernetes node generates a high-performance computing node configuration file and sends a node joining request to the high-performance computing cluster.

[0082] Step 306: The high-performance computing cluster responds to the node joining request of the target Kubernetes node and adds the target Kubernetes node to the high-performance computing cluster as a high-performance computing node.

[0083] In this embodiment, the environment deployment of the target Kubernetes node is quickly completed through the image repository, thereby improving the efficiency of adding new high-performance computing nodes.

[0084] In one embodiment, the high-performance computing cluster in the computing cluster includes a master high-performance computing cluster and a common high-performance computing cluster. After the target Kubernetes node is started, it is used to request an authentication key from the master high-performance computing node and start the high-performance computing cluster authentication service.

[0085] Among them, the authentication key is the key used for communication between high-performance computing nodes.

[0086] Taking the Slurm cluster as an example, after the target Kubernetes node is started, it needs to be added to the high-performance computing cluster as a high-performance computing node and communicate with other high-performance computing nodes in the high-performance computing cluster to achieve job collaboration. Therefore, the target Kubernetes node requests the authentication key from the master high-performance computing node, and then starts the high-performance computing cluster authentication service, such as the Munge process, and sends the requested authentication key to the Munge process, so that when the subsequent target Kubernetes node communicates with other high-performance computing cluster nodes, the Munge process automatically completes verification based on the authentication key to achieve communication with other high-performance computing cluster nodes.

[0087] At this time, if Figure 4 As shown, after executing step 304 and before executing step 305, the cluster scheduling method further includes the following steps:

[0088] Step 401: The target Kubernetes node requests an authentication key, i.e., a message authentication code, from the master high-performance computing node.

[0089] Step 402: The target Kubernetes node starts the high-performance computing cluster authentication service, so that the high-performance computing cluster authentication service uses the authentication key to verify the information sent by other high-performance computing clusters.

[0090] In this embodiment, an authentication program is added to the target Kubernetes node so that the target Kubernetes node, as a high-performance computing node, can communicate with each high-performance computing node of the original high-performance computing cluster more securely and quickly.

[0091] In one embodiment, the target Kubernetes node is used to generate a high-performance computing node configuration file and then send the node information of the target Kubernetes node to the high-performance computing cluster so that the high-performance computing cluster adds the target Kubernetes node as a high-performance computing node to the high-performance computing cluster.

[0092] The high-performance computing cluster schedules jobs based on the node information of each high-performance computing node. Therefore, before scheduling the target Kubernetes node as a high-performance computing node, it is necessary to obtain the node information of the target Kubernetes node so that the job can be scheduled based on the node information of the target Kubernetes node in the future. Therefore, after the target Kubernetes node generates the high-performance computing node configuration file, the node information of the target Kubernetes node is sent to the high-performance computing cluster so that the high-performance computing cluster adds the target Kubernetes node as a high-performance computing node to the high-performance computing cluster.

[0093] The node information of the target Kubernetes node includes address information, number of CPUs or GPUs, memory information, etc.

[0094] At this time, if Figure 5 As shown, the cluster scheduling method includes steps 501 to 505:

[0095] Step 501: During the job scheduling process, the high performance computing cluster has insufficient computing resources and sends a node addition request to the management platform.

[0096] The node addition request may include the number of newly added nodes.

[0097] Step 502: In response to the node addition request of the high-performance computing cluster, the management platform sends an instruction to add a high-performance computing node to the master Kubernetes node in Kubernetes.

[0098] Step 503: The master Kubernetes node responds to the instruction of adding a high-performance computing node, selects a node from each idle ordinary Kubernetes node as the target Kubernetes node and starts it.

[0099] Step 504: The target Kubernetes node generates a high-performance computing node configuration file, and sends a node joining request carrying the node information of the target Kubernetes node to the high-performance computing cluster.

[0100] Step 505: The high-performance computing cluster responds to the node joining request of the target Kubernetes node, saves the node information of the target Kubernetes node, and adds the target Kubernetes node to the high-performance computing cluster as a high-performance computing node.

[0101] In this embodiment, the target Kubernetes node sends the node information of the node to the high-performance computing cluster, so that the high-performance computing cluster issues corresponding job tasks based on the node information of the target Kubernetes node when performing job scheduling.

[0102] In one embodiment, each common Kubernetes node is pre-divided into a container node set and an elastic high-performance computing node set. At this time, the master Kubernetes node is used to respond to the instruction of adding high-performance computing, select a Kubernetes node in the elastic high-performance computing node set as the target Kubernetes node and start it.

[0103] In some embodiments, in order to reduce the occurrence of insufficient resource scheduling, sufficient Kubernetes nodes can be deployed in the Kubernetes cluster, and the number of Kubernetes nodes depends on the peak number of job requests received by the computing cluster. At this time, each ordinary Kubernetes node is pre-divided into a container node set and an elastic high-performance computing node set. When the Kubernetes cluster needs to create a new container, a Kubernetes node is selected from the ordinary Kubernetes nodes divided into the container node set and a container is built in the selected node. When the Kubernetes cluster needs to create a new high-performance computing node, a Kubernetes node is selected from the ordinary Kubernetes nodes divided into the elastic high-performance computing node set and the selected node is used as the target Kubernetes node.

[0104] At this time, if Figure 6 As shown, the cluster scheduling method includes steps 601 to 605:

[0105] Step 601: During the job scheduling process, the high performance computing cluster has insufficient computing resources and sends a request for adding a node to the management platform.

[0106] The node addition request may include the number of newly added nodes.

[0107] Step 602: In response to the node addition request of the high-performance computing cluster, the management platform sends an instruction to add a high-performance computing node to the master Kubernetes node in Kubernetes.

[0108] Step 603: The master Kubernetes node responds to the instruction to add a new high-performance computing node, selects a Kubernetes node from the elastic high-performance computing node set as the target Kubernetes node and starts it.

[0109] Step 604: The target Kubernetes node generates a high-performance computing node configuration file, and sends a node joining request carrying the node information of the target Kubernetes node to the high-performance computing cluster.

[0110] Step 605: The high-performance computing cluster responds to the node joining request of the target Kubernetes node, saves the node information of the target Kubernetes node, and adds the target Kubernetes node to the high-performance computing cluster as a high-performance computing node.

[0111] In this embodiment, the role of each node in the Kubernetes cluster is pre-divided to reduce scheduling failures caused by insufficient number of nodes in the Kubernetes cluster itself.

[0112] In one embodiment, the management platform is used to send an instruction to add a new high-performance computing cluster to the master Kubernetes node when receiving a request to add a new high-performance computing cluster.

[0113] Specifically, some users need a separate high-performance computing cluster to run high-performance jobs, so they will request the computing cluster management platform to add a new high-performance computing cluster for user use. After the management platform receives the high-performance computing cluster addition request sent by the user, it sends the instruction to add the new high-performance computing cluster to the master Kubernetes node.

[0114] The master Kubernetes node is used to respond to the newly added instructions of the high-performance computing cluster and start several Kubernetes nodes based on the node quantity information carried in the instructions.

[0115] The master Kubernetes node is used to obtain the image of the high-performance computing node from the image repository and copy it to each Kubernetes node. One of the Kubernetes nodes is used as the elastic master high-performance computing node, and the other Kubernetes nodes are used as elastic common high-performance computing nodes.

[0116] Elastic master high-performance computing node, used to generate master configuration files.

[0117] In a high-performance computing cluster, the master high-performance computing node is responsible for job scheduling and distributing high-performance computing jobs to each high-performance computing node. Each high-performance computing node collaborates to complete the high-performance computing job. Therefore, after the master Kubernetes node starts several Kubernetes nodes, the Kubernetes node that serves as an elastic master high-performance computing node needs to generate a master configuration file, including the initialization of the job scheduling strategy and the initialization of the high-performance computing cluster.

[0118] The elastic common high-performance computing node is used to generate a running configuration file and request the elastic master high-performance computing node to join the elastic high-performance computing cluster.

[0119] The elastic common high-performance computing node configuration file is the configuration information required to run high-performance computing jobs, including the operating environment, device information, etc., so that the high-performance computing cluster can obtain the relevant information of the high-performance computing node through the high-performance computing node configuration file for scheduling. For the high-performance computing cluster as a Slurm cluster, the node configuration file of the Slurm cluster is Slurm_node.conf. The Slurm_node.conf file contains a lot of configuration information of the node, such as the node name, IP address, architecture type, number of CPU cores, memory capacity, GPU type and number, etc. The details of the high-performance computing node configuration file can be referred to the description in the relevant technology, and will not be described in detail here.

[0120] When the computing cluster receives a request to create a new high-performance computing cluster, the cluster scheduling method is as shown in steps 701 to 705.

[0121] Step 701: Upon receiving a request to add a high-performance computing cluster, the management platform sends an instruction to add a new high-performance computing cluster to the master Kubernetes node.

[0122] Step 702: The master Kubernetes node responds to the newly added instruction of the high-performance computing cluster and starts several Kubernetes nodes based on the node quantity information carried in the instruction.

[0123] Step 703: The master Kubernetes node obtains the image of the high-performance computing node from the image repository and copies it to each Kubernetes node, using one of the Kubernetes nodes as an elastic master high-performance computing node and the other Kubernetes nodes as elastic common high-performance computing nodes.

[0124] Step 704: The elastic master high-performance computing node generates a master configuration file.

[0125] Step 705: The elastic common high-performance computing node generates a running configuration file, and requests the elastic master high-performance computing node to join the elastic high-performance computing cluster.

[0126] In this embodiment, when the management platform receives a request to create a new high-performance computing cluster, it controls the K8S cluster to start a batch of nodes and configure them to obtain a new, independent high-performance computing cluster.

[0127] In one embodiment, each common Kubernetes node is pre-divided into a container node set and an elastic node set. At this time, the master Kubernetes node is used to respond to the newly added instruction of the high-performance computing cluster, select and start several Kubernetes nodes in the elastic node set based on the node quantity information carried by the instruction.

[0128] In some embodiments, in order to reduce the occurrence of insufficient resource scheduling, sufficient Kubernetes nodes can be deployed in the Kubernetes cluster, and the number of Kubernetes nodes depends on the peak number of job requests received by the computing cluster. At this time, each ordinary Kubernetes node is pre-divided into a container node set and an elastic high-performance computing node set. When the Kubernetes cluster needs to create a new container, a Kubernetes node is selected from the ordinary Kubernetes nodes divided into the container node set and a container is built in the selected node. When the Kubernetes cluster needs to create a new high-performance computing cluster, several Kubernetes nodes are selected from the ordinary Kubernetes nodes divided into the elastic high-performance computing node set as elastic high-performance computing nodes in the elastic high-performance computing cluster, wherein each elastic high-performance computing node in the elastic high-performance computing cluster is converted from an ordinary K8S node in the K8S cluster.

[0129] In this embodiment, the role of each node in the Kubernetes cluster is pre-divided to reduce scheduling failures caused by insufficient number of nodes in the Kubernetes cluster itself.

[0130] In one embodiment, after the elastic common high-performance computing node is started, it is used to request an authentication key from the elastic master high-performance computing node and start the high-performance computing cluster authentication service.

[0131] Among them, the authentication key is the key used for communication between high-performance computing nodes.

[0132] Taking the high-performance computing cluster as a Slurm cluster as an example, the elastic master high-performance computing node starts the high-performance computing cluster authentication service, confirms the user information through the getpeername() function, and generates a credential information. The credential information includes: user ID, group ID, timestamp, life cycle, pseudo-random number, etc. After that, the high-performance computing cluster authentication service uses the private key to generate an authentication key. After the elastic ordinary high-performance computing node is started, it requests the authentication key from the elastic master high-performance computing node, and then starts the high-performance computing cluster authentication service, such as the Munge process, and sends the requested authentication key to the Munge process, so that when the elastic ordinary high-performance computing node communicates with other elastic ordinary high-performance computing nodes after starting, the Munge process automatically completes the verification based on the authentication key to achieve communication with other elastic ordinary high-performance computing nodes.

[0133] In this embodiment, an authentication procedure is added to each elastic high performance computing node to enable the elastic high performance computing nodes to communicate with each other more securely and quickly.

[0134] In one embodiment, after the elastic ordinary high-performance computing node generates a high-performance computing node configuration file, it is used to send the node information of the elastic ordinary high-performance computing node to the elastic master high-performance computing node, so that the elastic master high-performance computing node adds the elastic ordinary high-performance computing node as a high-performance computing node to the high-performance computing cluster.

[0135] The high-performance computing cluster schedules jobs based on the node information of each high-performance computing node. Therefore, before scheduling an elastic ordinary high-performance computing node as a high-performance computing node, it is necessary to obtain the node information of the elastic ordinary high-performance computing node so that jobs can be scheduled based on the node information of the elastic ordinary high-performance computing node in the future. Therefore, after the elastic ordinary high-performance computing node generates a high-performance computing node configuration file, the node information of the elastic ordinary high-performance computing node is sent to the elastic master high-performance computing node, so that the elastic master high-performance computing node uses the elastic ordinary high-performance computing node as the high-performance computing cluster where the elastic master high-performance computing node is located.

[0136] The node information of the elastic master high-performance computing node includes address information, number of CPUs or GPUs, memory information, etc.

[0137] In this embodiment, each elastic ordinary high-performance computing node sends the node information of the node to the elastic master high-performance computing node, so that the elastic master high-performance computing node can issue corresponding job tasks based on the node information of each elastic ordinary high-performance node when performing job scheduling.

[0138] In one of the embodiments, the master Kubernetes node monitors the first Kubernetes node of each high-performance computing node; when it is monitored that the length of time the first Kubernetes node is in an idle state reaches a recycling condition, the data information of the first Kubernetes node is cleared and the first Kubernetes node is recycled.

[0139] In a Kubernetes cluster, containers are generally created as tasks are requested. If a container fails to execute a task or completes a task, the container will be automatically destroyed and recycled.

[0140] In the computing cluster of the present application, the first Kubernetes node that becomes a high-performance computing node needs to run corresponding high-performance computing according to the scheduling of the high-performance computing cluster. Therefore, the first Kubernetes node may be in an idle state or in a running state depending on the scheduling of the high-performance computing cluster. Therefore, the first Kubernetes node will not fail to execute a task like a container, or be automatically destroyed and recycled after the task is completed.

[0141] Therefore, the master Kubernetes node can monitor the operating status of the first Kubernetes node that becomes a high-performance computing node. When it is detected that the length of time the first Kubernetes node is idle reaches the recycling condition, it indicates that the current high-performance computing cluster resources are sufficient, and the data information of the first Kubernetes node is cleared and the first Kubernetes node is recycled.

[0142] In this embodiment, when it is determined that the high-performance computing cluster resources are sufficient, the Kubernetes nodes that are in an idle state and converted into high-performance computing nodes are recovered.

[0143] In one of the embodiments, the management platform is used to submit the job to a master high-performance computing node or a master Kubernetes node when receiving the job according to the job type of the job.

[0144] Specifically, high-performance computing jobs differ from ordinary computing jobs in terms of running time, computing accuracy, hardware requirements, computing amount, and application fields. After the management platform receives the job submitted by the user, it determines whether the job type is a high-performance computing job or an ordinary computing job based on the job's predicted running time, computing accuracy requirements, hardware requirements, estimated computing amount, and application fields. If the job type of the job received by the management platform is a high-performance computing job, the job is submitted to the master high-performance computing node, and the master high-performance computing node schedules the corresponding resources in the high-performance computing cluster in the computing cluster to run the job; if the job type of the job received by the management platform is a ordinary computing job, the job is submitted to the master Kubernetes node, and the master Kubernetes node schedules the corresponding resources in the Kubernetes cluster in the computing cluster to run the job.

[0145] In this embodiment, the management platform can send jobs to a high-performance computing cluster or a K8S cluster based on different job types.

[0146] In one embodiment, the high performance computing node is a Slurm node and the high performance computing cluster is a Slurm cluster.

[0147] For the relevant contents about Slurm cluster and Slurm node, please refer to the description in the related technology, which will not be described in detail here.

[0148] Next, the specific embodiments provided in this application are described in detail.

[0149] like Figure 7 As shown, it is a schematic diagram of the composition of a cluster cluster according to an embodiment of the present application. The computing cluster includes a Slurm cluster and a K8S cluster. The addition and recovery of virtual nodes in the K8S1 cluster are realized by the KubeVirt cluster. Among them, the nodes marked with H are fixed nodes, nodes of the Slurm cluster, running HPC loads; the nodes marked with TH are elastic nodes, which are Slurm nodes that the K8S cluster extends from the original Slurm cluster, running HPC loads; the nodes marked with DH are elastic nodes, which are nodes in the newly created independent Slurm cluster of K8S, running HPC loads; nodes without any markings are idle nodes, which are ordinary nodes without load in the K8S cluster.

[0150] like Figure 8 As shown in the figure, the steps to extend the Slurm cluster to run high performance computing (HPC) workloads for the compute cluster are:

[0151] 801. The user logs in to the platform and submits the HPC task to the Slurm cluster through the management platform.

[0152] 802. The master node of the Slurm cluster determines whether the current Slurm cluster can run the HPC task submitted by the user based on the HPC task submitted by the user and the Slurm nodes included in the Slurm cluster.

[0153] 803. When there are sufficient Slurm nodes, the Slurm cluster adds the HPC tasks submitted by the user to the waiting queue, waiting to be directly run by the Slurm cluster.

[0154] 804. When it is determined that the number of Slurm nodes is insufficient, a request for adding a new node is sent to the management platform.

[0155] 805. The management platform sends instructions to the master K8S node of the K8S cluster to add a new high-performance computing node.

[0156] 806. The master K8S node of the K8S cluster starts one or more new K8S nodes as target K8S nodes.

[0157] 807. Configure and start the munge authentication program in the target K8S node.

[0158] 808. Generate a Slurm configuration file in the target K8S node.

[0159] 809. The target K8S node sends the host name and IP of the target K8S node to the Slurm cluster.

[0160] 810. The target K8S node starts the Slurm client service.

[0161] 811. After the Slurm cluster completes elastically adding nodes, it adds the HPC tasks submitted by the user to the queue to be run, waiting to be run by the Slurm cluster.

[0162] When users need a separate cluster to run HPC tasks, the computing cluster can create an independent Slurm cluster for the user. Fig. 9 As shown below, the steps to create a new Slurm cluster for the compute cluster to run HPC workloads are:

[0163] 901. When the management platform receives a request for adding a Slurm cluster submitted by a user, the management platform responds to the request for adding a Slurm cluster and sends an instruction for adding a Slurm cluster to the master K8S node of the K8S cluster.

[0164] 902. The master K8S node of the K8S cluster starts a batch of elastic nodes as elastic high-performance computing nodes.

[0165] 903. Configure and start the munge authentication service in the first elastic high-performance computing node.

[0166] 904. Configure and start the Slurm master service in the first elastic high performance computing node.

[0167] 905. Configure and start the munge authentication program in other elastic high-performance computing nodes.

[0168] 906. Generate Slurm configuration files on other elastic high performance computing nodes.

[0169] 907. The other elastic high performance computing nodes send their respective host names and IP addresses to the first elastic high performance computing node.

[0170] 908. Start the Slurm client service in other elastic high performance computing nodes.

[0171] 909. The management platform successfully created a new independent Slurm cluster.

[0172] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0173] In one embodiment, a computer device is provided, which may be the above-mentioned high-performance computing node or K8S node, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, and a communication interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the steps performed by the above-mentioned high-performance computing node or K8S node are implemented.

[0174] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0176] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0177] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A computing cluster, characterized in that: The computing cluster includes a management platform, a container orchestration system Kubernetes cluster, and a high-performance computing cluster. The Kubernetes cluster includes a master Kubernetes node and multiple common Kubernetes nodes, wherein: The management platform is used to send an instruction to add a high-performance computing node to the master Kubernetes node when receiving a request to add a node to the high-performance computing cluster; The master Kubernetes node is used to start the target Kubernetes node in response to the instruction of the newly added high-performance computing node; the target Kubernetes node is an idle common Kubernetes node; The target Kubernetes node is used to generate a high-performance computing node configuration file and request the high-performance computing cluster to join the high-performance computing cluster.

2. According to the computing cluster of claim 1, the master Kubernetes node is used to start the target Kubernetes node, obtain the high-performance computing node image in the image warehouse and copy it to the target Kubernetes node.

3. The computing cluster according to claim 1, characterized in that: The high performance computing cluster includes a master high performance computing node and common high performance computing nodes; After the target Kubernetes node is started, it is used to request the authentication key from the master high-performance computing node and start the high-performance computing cluster authentication service.

4. The computing cluster according to claim 1, characterized in that: The target Kubernetes node is used to generate a high-performance computing node configuration file and then send the node information of the target Kubernetes node to the high-performance computing cluster, so that the high-performance computing cluster adds the target Kubernetes node to the high-performance computing cluster as a high-performance computing node.

5. The computing cluster according to claim 1, characterized in that: Each of the common Kubernetes nodes is pre-divided into a container node set and an elastic high-performance computing node set; The master Kubernetes node is used to respond to the instruction of adding high-performance computing, select a Kubernetes node as the target Kubernetes node in the elastic high-performance computing node set and start it.

6. The computing cluster according to claim 1, characterized in that: The management platform is used to send an instruction to add a new high-performance computing cluster to the master Kubernetes node when receiving a request to add a new high-performance computing cluster; The master Kubernetes node is used to respond to the newly added instruction of the newly added high-performance computing cluster and start several Kubernetes nodes based on the node quantity information carried by the instruction; The master Kubernetes node is used to obtain the image of the high-performance computing node in the image warehouse and copy it to each of the Kubernetes nodes, use one of the Kubernetes nodes as an elastic master high-performance computing node, and use the other Kubernetes nodes as elastic common high-performance computing nodes; The elastic master high-performance computing node is used to generate a master configuration file; The elastic common high-performance computing node is used to generate an operation configuration file and request the elastic master high-performance computing node to join the elastic high-performance computing cluster.

7. The computing cluster according to claim 6, characterized in that: Each of the common Kubernetes nodes is pre-divided into a container node set and an elastic node set; The master Kubernetes node is used to respond to the newly added instruction of the newly added high-performance computing cluster, select and start several Kubernetes nodes in the elastic node set based on the node quantity information carried by the instruction.

8. The computing cluster according to claim 6, characterized in that: The elastic common high-performance computing node is used to request an authentication key from the elastic master high-performance computing node after startup, and start a high-performance computing cluster authentication service.

9. The computing cluster according to claim 5, characterized in that: The elastic common high-performance computing node is used to generate a high-performance computing node configuration file and then send the node information of the target Kubernetes to the elastic master high-performance computing node, so that the elastic master high-performance computing node adds the elastic common high-performance computing node as a high-performance computing node to the high-performance computing cluster.

10. The computing cluster according to any one of claims 1 to 8, characterized in that: The master Kubernetes node is used to monitor the first Kubernetes node of each high-performance computing node; when it is monitored that the length of time the first Kubernetes node is in an idle state reaches a recycling condition, the data information of the first Kubernetes node is cleared and the first Kubernetes node is recycled.

11. The computing cluster according to any one of claims 1 to 8, characterized in that: The management platform is used to submit the job to the master high-performance computing node or the master Kubernetes node according to the job type of the job when receiving the job.

12. The computing cluster according to any one of claims 1 to 8, characterized in that: The high performance computing node is a Slurm node, and the high performance computing cluster is a Slurm cluster.

Images