High-performance computing cluster deployment method and device, computer equipment and storage medium

Abstract

The invention relates to a high-performance computing cluster deployment method and device, computer equipment and a storage medium. The method comprises: in response to a service deployment instruction for a target cluster, determining service deployment information set for the target cluster, the service deployment information comprising a corresponding relationship between a node in the target cluster and a service in a plurality of services, the plurality of services comprising a target scheduling service; determining matching services of nodes in the target cluster according to the service deployment information; determining a matching instruction template of the node under the matching service from at least one instruction template associated with the matching service, and generating a service deployment instruction of the node under the matching service according to the matching instruction template; and issuing a service deployment instruction of the node under the matching service to the node, so that the node executes the service deployment instruction, and deploying the target cluster as a high-performance computing cluster based on the target scheduling service. By adopting the method, the deployment efficiency of the high-performance computing cluster can be improved.

Description

Technical Field

[0001] The present application relates to the fields of Internet and computer technology, and in particular to a high-performance computing cluster deployment method, apparatus, computer equipment, and storage medium. Background Art

[0002] In a high-performance computing (HPC) environment, the scheduler is a key component responsible for managing and allocating computing resources to tasks. Different HPC systems may use different schedulers, each with its own unique characteristics and differences, depending on the system architecture and user requirements.

[0003] In supercomputing applications, users need to select the appropriate scheduler based on their specific business scenarios and domain characteristics. Examples of schedulers include SLURM, PBS, or LSF. Because different schedulers vary in functionality, flexibility, performance, scalability, and configuration management, supercomputing platforms must provide different scheduler adaptability for different customers.

[0004] Currently, in order to deploy (i.e., build) a high-performance computing cluster, scripts are usually used for deployment. The scripts are only applicable to specific schedulers and are not applicable to different types of schedulers, resulting in low deployment efficiency of high-performance computing clusters. Summary of the Invention

[0005] Based on this, it is necessary to provide a high-performance computing cluster deployment method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the deployment efficiency of high-performance computing clusters in response to the above technical problems.

[0006] On the one hand, the present application provides a high-performance computing cluster deployment method, including: in response to a service deployment instruction for a target cluster, determining service deployment information set for the target cluster, the service deployment information including a correspondence between nodes in the target cluster and services in a plurality of services, the plurality of services including a target scheduling service; determining a matching service for the node in the target cluster based on the service deployment information, the matching service for the node being a service corresponding to the node; determining a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service, and generating a service deployment instruction for the node under the matching service based on the matching instruction template; sending the service deployment instruction for the node under the matching service to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

[0007] On the other hand, the present application also provides a high-performance computing cluster deployment device, including: a deployment information determination module, used to determine the service deployment information set for the target cluster in response to a service deployment instruction for the target cluster, the service deployment information including the correspondence between the nodes in the target cluster and the services in multiple services, the multiple services including a target scheduling service; a matching service determination module, used to determine the matching service of the node in the target cluster according to the service deployment information, the matching service of the node is the service corresponding to the node; an instruction generation module, used to determine the matching instruction template of the node under the matching service from at least one instruction template associated with the matching service, and generate the service deployment instruction of the node under the matching service according to the matching instruction template; an instruction issuing module, used to issue the service deployment instruction of the node under the matching service to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

[0008] On the other hand, the present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned high-performance computing cluster deployment method when executing the computer program.

[0009] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above-mentioned high-performance computing cluster deployment method when executed by a processor.

[0010] On the other hand, the present application also provides a computer program product, including a computer program, which implements the steps in the above-mentioned high-performance computing cluster deployment method when executed by a processor.

[0011] The above-mentioned high-performance computing cluster deployment method, apparatus, computer equipment, computer-readable storage medium and computer program product, since the service deployment information is set for the target cluster, and the service deployment information includes the correspondence between the nodes in the target cluster and the services in multiple services, and the multiple services include the target scheduling service, can build the required high-performance computing cluster by setting the required scheduling service and the service adapted to the scheduling service. Determine the matching instruction template of the node under the matching service from at least one instruction template associated with the matching service, generate the service deployment instruction of the node under the matching service according to the matching instruction template, and send the service deployment instruction of the node under the matching service to the node so that the node executes the service deployment instruction. This can realize the process from service deployment information to service deployment instruction, and then realize the construction of a high-performance computing cluster based on the service deployment instruction. It can be applied to build or deploy any high-performance computing cluster based on any scheduling service, thereby improving the deployment efficiency of the high-performance computing cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A diagram of an application environment of a high-performance computing cluster deployment method in some embodiments;

[0014] Figure 2 A schematic diagram of a process for deploying a high-performance computing cluster in some embodiments;

[0015] Figure 3 An architectural diagram of a high-performance computing cluster deployment method in some embodiments;

[0016] Figure 4 A timing diagram of a high performance computing cluster deployment method in some embodiments;

[0017] Figure 5 A structural block diagram of a high-performance computing cluster deployment device in some embodiments;

[0018] Figure 6 is a diagram of the internal structure of a computer device in some embodiments;

[0019] Figure 7 1 is a diagram of the internal structure of a computer device in some other embodiments. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0021] The high performance computing cluster deployment method provided in the embodiment of the present application can be applied to Figure 1 In the illustrated application environment, the application scenario includes a cluster deployment device 102, a first target cluster 104, and a second target cluster 106. The first target cluster 104 includes nodes 1 to 3, and the second target cluster 106 includes nodes 4 to 6. The cluster deployment device 102, the first target cluster 104, and the second target cluster 106 can communicate via a network. Figure 1 The number of clusters and the number of nodes in a cluster are just examples. In actual implementation, there can be more clusters and more nodes in a cluster.

[0022] Specifically, the cluster deployment device 102 can deploy a high-performance computing system on a cluster. High-performance computing is a computational approach that utilizes computing resources to solve complex scientific, engineering, and business problems. A scheduler is a key component responsible for managing and allocating computing resources (such as compute nodes, processor cores, or memory) to tasks to be executed. Different high-performance computing systems may use different schedulers, each with its own characteristics and differences depending on the system architecture and user needs. Schedulers, also known as scheduling services, can include, but are not limited to, SLURM (Simple Linux Utility for Resource Management), PBS (PortableBatch System), or LSF (Load Sharing Facility). SLURM is an open-source scheduler used in universities and supercomputing centers. It supports multiple scheduling strategies, such as FIFO, Fairshare, and priority scheduling. It is suitable for working with arrays and facilitates the simultaneous submission of multiple similar tasks. PBS, which includes the commercial version PBS Professional and the open-source version Open PBS, offers high flexibility and scalability, making it suitable for HPC environments of various sizes. It supports scheduling strategies such as shortest job first and first-come, first-served. LSF is a commercial scheduler suitable for large-scale clusters. It supports multiple scheduling algorithms and provides customizable job scheduling rules. Open source schedulers are free to use, while commercial schedulers require authorization. Different schedulers vary in functionality and flexibility and should be selected based on specific application scenarios. Scheduler performance may vary depending on cluster size and complexity. Schedulers vary in configuration and management complexity, and ease of maintenance should be considered. When selecting an HPC scheduler, you need to make an appropriate choice based on your HPC environment and requirements. Regardless of the scheduler used, good job scheduling and resource management are key factors in the operation of high-performance computing clusters.

[0023] The cluster deployment device 102 can deploy at least one high-performance computing system on the same cluster, or deploy the same or different types of high-performance computing systems on different clusters. For example, a high-performance computing system based on a first scheduler can be deployed on the first target cluster 104, and a high-performance computing system based on a second scheduler can be deployed on the second target cluster 106. The first scheduler and the second scheduler can be the same or different. Deployment can be understood as installation or construction. That is, the cluster deployment device 102 supports the deployment of high-performance computing systems with multiple schedulers.

[0024] The cluster deployment device 102 can be, but is not limited to, a desktop computer or laptop computer, and can also be a server. The nodes in the first target cluster 104 and the second target cluster 106 can be independent physical servers, or they can be a server cluster or distributed system composed of multiple physical servers, or cloud servers providing cloud computing services. Cloud servers are used to provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The cluster deployment device 102, the first target cluster 104, and the second target cluster 106 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0025] In some embodiments, as Figure 2 As shown, a high performance computing cluster deployment method is provided, which can be executed by a terminal or a server, or by a terminal and a server together. Figure 1 Taking the cluster deployment device 102 in FIG. 1 as an example, the method includes the following steps 202 to 208. Among them:

[0026] Step 202 : In response to a service deployment instruction for a target cluster, determining service deployment information set for the target cluster, the service deployment information including a correspondence between nodes in the target cluster and services in a plurality of services, the plurality of services including a target scheduling service.

[0027] The cluster deployment device 102 includes a cluster building system, which is used to build a high-performance computing cluster. The high-performance computing cluster deployment method provided in this application can be implemented based on the cluster building system in the cluster deployment device 102. The target cluster includes multiple nodes, and the roles of the nodes in the target cluster can be the same or different.

[0028] The node corresponding to the service refers to the node where the service needs to be installed or deployed. The deployment method of the service can be, but is not limited to, standalone (single-machine deployment), server-client (server-client) or classic (classic layered architecture). The service configuration page can be used to configure the node where the service needs to be installed and the role of the node in the service. Taking the Launcher service as an example, the deployment method of the Launcher service can be server-client. You can choose to install the Launcher on two different nodes, with one node role as server and the other node role as client. The service deployment information can also include the role of the node in the service. Each service can adapt to different types of operating systems, and the type of operating system can be, but is not limited to, Linux or Windows.

[0029] Control nodes and compute nodes play different roles in the target cluster. Control nodes run the scheduler, assigning computing tasks to compute nodes and monitoring cluster status, such as node load and resource usage. Schedulers are also called scheduling services, and the target scheduling service can be any scheduler. Compute nodes execute computing tasks. To enable control and compute nodes to function as desired, services must be deployed on these nodes.

[0030] In some embodiments, high-performance computing relies on various services. For example, building a SLURM cluster requires relying on related services such as MUNGE, LDAP, and MYSQL. The required services can be created through service definitions. For example, the services that the cluster depends on can be abstractly defined to describe the name, version, deployment method, and direct dependency of each service, and establish a directory tree of interdependent services. The terminal device or cluster deployment device displays a service management page, and a service list is displayed on the service management page. The service list includes multiple service items, and different service items represent different services. The service item may include at least one of the service name, deployment method, version, serial number, or type. The services in the service list are adapted to different operating systems.

[0031] In some embodiments, the terminal device or cluster deployment device displays a service configuration page corresponding to the target cluster. The service configuration page is used to set service deployment information. For example, the service configuration page allows users to select a service and set at least one node on which the selected service is to be installed and the node's role in the service. Service deployment instructions can be sent from the terminal device to the cluster deployment device or triggered by the cluster deployment device.

[0032] In some embodiments, the terminal device or cluster deployment device displays a cluster management page, which displays a cluster list. The cluster list displays at least one cluster item, and different cluster items represent different clusters. The target cluster can be any cluster in the cluster list.

[0033] In some embodiments, a terminal device or a cluster deployment device creates a cluster in response to a creation operation to create multiple nodes as a cluster. To create a cluster, resources must be defined in advance (for example, abstracting IAAS (Infrastructure as a Service) resources involved in high-performance computing as the medium for high-performance computing nodes). Each resource can be understood as a node, and information such as the type of processor (e.g., CPU or GPU, machine type (e.g., physical machine or cloud host), system, memory, specifications, and hardware type) of each resource must be determined. Before or after cluster creation, nodes can be labeled to clearly indicate whether they are control nodes or compute nodes.

[0034] In some embodiments, in response to a trigger operation on a cluster item corresponding to a target cluster, a terminal device or cluster deployment device displays a node list containing at least one node item, with different node items representing different nodes in the target cluster. The node item may include the node's machine model and processor type. The machine model may include, but is not limited to, a physical machine or a cloud host. The processor type may include, but is not limited to, a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).

[0035] Step 204: Determine the matching service of the node in the target cluster according to the service deployment information. The matching service of the node is the service corresponding to the node.

[0036] Each node can correspond to at least one service, and thus each node can have at least one matching service. A service can correspond to at least one node, meaning a service can be deployed on at least one node. For the same service, different deployment results can vary depending on the node's role within the service. For example, the deployment results for the client and server roles within the same service can be different.

[0037] In some embodiments, the terminal device or cluster deployment device may display a cluster management page, which includes a cluster list. The cluster list includes at least one cluster item, with different cluster items representing different clusters. When the cluster item corresponding to the target cluster is selected, the terminal device or cluster deployment device displays a service configuration page corresponding to the target cluster in response to the service deployment operation.

[0038] In some embodiments, as Figure 3 As shown, an architecture diagram corresponding to a high-performance computing cluster deployment method is provided, wherein the page provided by the cluster deployment device supports cluster configuration, node configuration and service configuration. The cluster configuration is used to create multiple nodes into a cluster. The node configuration is used to configure the nodes in the cluster, such as configuring the role of the node in the cluster (control node or computing node). The service configuration is used to add or delete services, and can also be used to configure the correspondence between nodes and services, that is, to determine the correspondence between services and nodes.

[0039] In the figure, the cluster deployment device includes a master service and slave services. The master and slave services can be synchronized. The API (Application Programming Interface) provides the interface for interface calls. Both the master and slave services include a cluster module, a node module, and a service module. Cluster management provides cluster creation and maintenance functions. Cluster state maintenance maintains the status of created clusters. Service instance management maintains service instances. Cluster monitoring monitors created clusters. Node management in the node module maintains cluster nodes. Node state maintenance maintains node status. Node service installation installs services on nodes. Node monitoring monitors node operational data. Service management in the service module maintains services. Service component management maintains service components. Service dependency management manages inter-service dependencies. Instruction template management maintains instruction templates. Service role management manages service-related roles.

[0040] In some embodiments, after the cluster deployment device obtains the service deployment information, it may generate service instance records corresponding to the multiple services respectively, and insert the service instance records into the service instance record table of the database according to the dependency relationship between the services.

[0041] In some embodiments, the cluster deployment device can determine the services that need to be installed on the node based on the service deployment information, generate a service instance node record, and the service instance node record contains the corresponding relationship between the service instance and the node. Then, based on the dependency relationship between the services to be installed, the service instance node record is inserted into the service instance node record table. For example, if service A and service B need to be installed on node 1, service instance node record 1 and service instance node record 2 are generated. Service instance node record 1 contains the corresponding relationship between node 1 and the instance of service A, and service instance node record 2 contains the corresponding relationship between node 1 and the instance of service B.

[0042] In some embodiments, the cluster deployment device inserts a service instance node component record based on the component definition of the service, for example, inserting the service instance node component record into a service instance node component record table. The service instance node component record contains the correspondence between the service instance, the node, and the components in the service instance, where the components in the service instance are the components that need to be installed on the node. For example, if components 1 to 5 of service A need to be installed on node 1, the service instance node component record indicates that components 1 to 5 of service A need to be installed on node 1.

[0043] Step 206: Determine a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service, and generate a service deployment instruction for the node under the matching service according to the matching instruction template.

[0044] A command template is used to generate a series of commands, each of which is a service deployment command. By executing these commands, the service is deployed. Due to different deployment methods and the different roles of nodes in the service, the commands executed when deploying the service also vary. Therefore, each service is associated with at least one command template. The cluster deployment device is responsible for command generation and scheduling.

[0045] To use instruction templates, pre-defined instructions are required. For example, cluster-dependent service interactions can be broken down into atomic operations, and corresponding instructions can be designed for each service's deployment method. For example, for a standalone MySQL deployment, instructions such as database initialization and user-defined instructions are designed. Instruction templates also need to be pre-built, as different instructions are required for different services, deployment methods, and the roles of each node. By designing software deployment methods and resource node roles, relevant instruction templates can be adapted. When scheduling instructions, cluster deployment devices can distribute relevant instructions based on the current scheduler (scheduling service), resources, operating system, software, specific deployment methods, and dependencies.

[0046] Specifically, the cluster deployment device may determine a matching instruction template for a node from at least one instruction template associated with the matching service based on the matching service deployment method and the node's role in the matching service. Based on the node's matching instruction template, the cluster deployment device may generate a series of instructions for deploying the matching service on the node. The cluster deployment device may also write the generated series of instructions into an instruction table corresponding to the node.

[0047] In some embodiments, for each node in the target cluster, the cluster deployment device determines the current service instance node record from the service instance node record table according to the order of the service instance node records in the service instance node record table corresponding to the node, determines the service instance identifier in the current service instance node record, and determines the service instance corresponding to the service instance identifier as the current service instance. It can be understood that the current service instance is an instance of a matching service of the node. The cluster deployment device can generate a series of service deployment instructions based on the environment variables and matching instruction template of the current service instance, and write the generated series of instructions into the instruction table corresponding to the node.

[0048] Step 208: Send the service deployment instruction of the node under the matching service to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

[0049] Among them, the nodes in the target cluster include executors, such as Figure 3 In [1], the executor can provide functions such as node registration, node monitoring data push, and command execution. Node registration is used to apply for adding a node to the target cluster. Node monitoring data push is used to monitor the node model to which it belongs and push the monitoring data to the cluster deployment device. Command execution is used to execute commands.

[0050] Specifically, the cluster deployment device can write service deployment instructions to the node's corresponding message queue. The node's executor can retrieve the service deployment instructions from the message queue and then execute the service deployment instructions. The node can then feedback the execution results to the cluster deployment device. If the execution result is failure, the cluster deployment device can retry the instruction. If the execution result is success, the cluster deployment device can continue to issue new service deployment instructions to the node.

[0051] In some embodiments, the multiple services to be deployed in the target cluster can have an installation order. When at least two services need to be installed / deployed on a node, the at least two services can be installed sequentially, and the installation order can be set based on the dependencies between the services. For example, if Service A and Service B need to be installed on Node 1, and if Service B depends on Service A, Service A can be installed first, followed by Service B.

[0052] In some embodiments, for each node in the target cluster, the cluster deployment device determines whether there are instructions to be executed in the instruction table corresponding to the node. If there are no instructions to be executed, the status of the current service instance node record is updated to installed in the service instance node record table, and the instance identifier corresponding to the service instance in the current service instance node is determined. The service instance corresponding to the instance identifier is determined from the service instance record table (recording the current service instance), and it is determined whether there are other nodes under the current service instance that have not been fully installed. If there are no other nodes that have not been fully installed, that is, the nodes that need to install the service represented by the current service instance have all installed the service, then the status of the current service instance in the service instance record table is updated to installed. After the status of the current service instance is updated to installed, the next service instance that needs to be installed for the node is determined, that is, a new current service instance node record is determined, and it is executed iteratively, so that all services that need to be installed for the node are installed. For example Figure 3 In the example, the slurm scheduler is installed on the control node, services 1 and 2 are installed on compute node 1, and services 2 and 3 are installed on compute node 2.

[0053] If there are instructions to be executed, the instructions to be executed are sent to the node. If there are other nodes that have not completed the installation, the installation of the current service instance is waited for by the other nodes.

[0054] In some embodiments, the node may monitor abnormal instruction execution and service status, and send an alarm notification to the cluster deployment service for abnormal behavior.

[0055] In the above-mentioned high-performance computing cluster deployment method, since the service deployment information is set for the target cluster, and the service deployment information includes the correspondence between the nodes in the target cluster and the services in the multiple services, and the multiple services include the target scheduling service, the required high-performance computing cluster can be built by setting the required scheduling service and the service adapted to the scheduling service. The matching instruction template of the node under the matching service is determined from at least one instruction template associated with the matching service, the service deployment instruction of the node under the matching service is generated according to the matching instruction template, and the service deployment instruction of the node under the matching service is sent to the node so that the node executes the service deployment instruction. This can realize the process from service deployment information to service deployment instruction, and then realize the construction of a high-performance computing cluster based on the service deployment instruction. It can be applied to the construction and deployment of any high-performance computing cluster based on any scheduling service, thereby improving the deployment efficiency of the high-performance computing cluster.

[0056] This application abstracts and decomposes the behaviors related to building a high-performance computing cluster, and abstracts the resources, operating system, etc. involved in the cluster, and proposes a complete system for building a high-performance computing cluster based on instruction interaction, which improves the stability, scalability and maintainability of the system.

[0057] In some embodiments, determining a matching instruction template for a node under a matching service from at least one instruction template associated with the matching service includes: determining a deployment mode of the matching service from service deployment information and determining a role of the node in the matching service; and determining a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service based on the deployment mode of the matching service and the role of the node in the matching service.

[0058] For the same service, different roles correspond to different matching instruction templates.

[0059] In this embodiment, the service deployment information contains the deployment method of the matching service and the role of the node in the matching service. Based on the deployment method of the matching service and the role of the node in the matching service, the matching instruction template is determined. Therefore, for different deployment methods and different roles, corresponding instructions can be generated through the matching instruction template, and the service can be installed by generating instructions.

[0060] In some embodiments, generating a service deployment instruction for a node under a matching service according to a matching instruction template includes: creating a service instance of the matching service and obtaining environment variables of the service instance; and generating a service deployment instruction for the node under the matching service according to the environment variables and the matching instruction template.

[0061] Each service instance can have at least one environment variable.

[0062] Specifically, there may be instruction parameters in the matching instruction template. The instruction parameters are parameters that need to be set. There is a one-to-one correspondence between the instruction parameters and the environment variables. The cluster deployment device can replace the instruction parameters in the matching instruction template with the corresponding environment variables to obtain an updated instruction template. Then, based on the updated instruction template, a service deployment instruction for the node under the matching service is generated.

[0063] In this embodiment, the service deployment instruction of the node under the matching service is generated according to the environment variables and the matching instruction template, which can conveniently generate the service deployment instruction.

[0064] In some embodiments, the target cluster is a heterogeneous cluster, each service is adapted to multiple architectures, and the node adapts and executes the service deployment instruction according to the architecture of the node.

[0065] Specifically, a service deployment instruction can be an abstract instruction. After the executor in a node obtains the service deployment instruction, it can switch or convert the service deployment instruction into an update deployment instruction that can be executed in the node based on the node's characteristics (such as architecture, system, or version). The difference between the update deployment instruction and the service deployment instruction is that the function is the same, but the instruction type is different. For example, the instruction type in Windows is different from that in Linux. A heterogeneous cluster means that the nodes in the cluster can adopt different architectures. For example, node 1 and node 2 in the cluster adopt different architectures. In other words, this high-performance computing cluster deployment method can deploy a heterogeneous cluster into a high-performance computing cluster. Therefore, the cluster deployment device supports the deployment of heterogeneous multi-scheduler high-performance computing clusters.

[0066] In this embodiment, heterogeneous multi-scheduler high-performance computing cluster deployment can be supported, expanding the application scenarios of the deployment.

[0067] In some embodiments, generating a service deployment instruction for a node under a matching service according to a matching instruction template includes: generating a service deployment instruction set for the node under the matching service according to the matching instruction template, and writing the service deployment instruction set into an instruction table of the node to send the service deployment instructions in the instruction table to the node; the method also includes: generating a service instance node record between the matching service and the node according to the service deployment information, and inserting the service instance node record into a service instance node record table, the service instance node record being used to record the correspondence between the service instance of the matching service and the node; when the service deployment instruction in the instruction table is executed, the status of the service instance node record is updated to executed.

[0068] The generation of the service deployment instruction set and the insertion of the service instance node record can be performed in parallel or in series, which is not limited here. The relevant description has been described above and will not be repeated here.

[0069] In this embodiment, by updating the status of the service instance node record, the installation status of the service instance in the node can be clarified in real time, so as to clarify the progress of the installation.

[0070] In some embodiments, the service deployment instruction of the node under the matching service is sent to the node, including: writing the service deployment instruction of the node under the matching service into the message queue corresponding to the node, so that the node reads the service deployment instruction from the message queue and executes the service deployment instruction to deploy the matching service in the node.

[0071] Specifically, after executing the service deployment instruction, the node can write the execution result to the corresponding message queue. The cluster deployment device reads the execution result from the message queue and determines whether the execution is successful based on the execution result. If it is successful, if there are still service deployment instructions that need to be executed, the next service deployment instruction will be sent. If there are no service deployment instructions that need to be executed under the service (that is, all instructions that need to be executed by the node under the service have been executed). Figure 3 As shown, if the message queue is the message queue corresponding to the control node, the service deployment instruction of the node under the matching service is written into the message queue. Then, the executor in the control node can read the service deployment instruction from the message queue and execute it.

[0072] In this embodiment, asynchronous communication between nodes and cluster deployment devices can be achieved through message queues, thereby improving the flexibility of communication.

[0073] In some embodiments, as Figure 4 As shown, a timing diagram of a high-performance computing cluster deployment method is provided, including:

[0074] 1. The cluster deployment device writes an initialization instruction to the message queue corresponding to the node;

[0075] The node can be any node in the target cluster. The initialization instruction is used to determine whether the node is ready for service deployment.

[0076] 2. When the node is in normal condition, it reads the initialization instruction from the message queue and returns the ready instruction to the cluster deployment device through the message queue;

[0077] The ready instruction is used to indicate that the node is currently in a normal state and is ready for service deployment.

[0078] 3. After receiving the ready instruction, the cluster deployment device inserts the service instance record according to the service dependency, inserts the service instance node record according to the service installation node, and inserts the service instance node component record according to the component definition of the service;

[0079] The service installation node refers to the node where the node needs to be installed.

[0080] 4. After the cluster deployment device sends one or a batch of service deployment instructions under the current service instance to the node and receives the execution result, it updates the execution status of the service deployment instruction. For example, if the execution result is successful, the execution status can be updated to executed.

[0081] The execution status is either success or failure. If successful, the cluster deployment device writes the next instruction to the message queue. If failed, the cluster deployment device resends the previous batch of instructions or displays a message indicating that the instruction execution failed. Service deployment instructions are stored in an instruction table, which records the execution status of service deployment instructions. The instructions in the instruction table are generated based on the environment variables of the current service instance and the matching instruction template for the node under the current service instance.

[0082] 6. Next, the cluster deployment device determines whether there are instructions to be executed in the node's instruction table. If so, the instructions to be executed are issued. If not, the service instance node status is updated to installed, and the instance variables are written to the environment variable table. Then, it is determined whether there are any unfinished nodes under the current service instance. If not, the current service instance is updated to installed, the environment variables are updated, and then a service to be installed of the node is requested, that is, the next current service instance is determined. The instruction template is determined and the instruction is generated according to the node's deployment role in the service, and the instruction is inserted into the instruction table. The instruction is read from the instruction table and issued; if there are other unfinished nodes under the current service instance, wait for other nodes to complete the installation of the current service instance.

[0083] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0084] Based on the same inventive concept, embodiments of the present application also provide a high-performance computing cluster deployment device for implementing the high-performance computing cluster deployment method described above. The implementation solution provided by this device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of one or more embodiments of the high-performance computing cluster deployment device provided below can be found in the above-mentioned limitations of the high-performance computing cluster deployment method and will not be repeated here.

[0085] In some embodiments, as Figure 5As shown, a high-performance computing cluster deployment device is provided, including: a deployment information determination module 502, a matching service determination module 504, an instruction generation module 506 and an instruction issuing module 508, wherein:

[0086] The deployment information determination module 502 is configured to determine service deployment information set for the target cluster in response to a service deployment instruction for the target cluster, wherein the service deployment information includes a correspondence between nodes in the target cluster and services in a plurality of services, wherein the plurality of services includes a target scheduling service.

[0087] The matching service determination module 504 is configured to determine a matching service for a node in the target cluster according to the service deployment information. The matching service for a node is a service corresponding to the node.

[0088] The instruction generation module 506 is configured to determine a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service, and generate a service deployment instruction for the node under the matching service according to the matching instruction template.

[0089] The instruction issuing module 508 is used to issue the service deployment instruction of the node under the matching service to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

[0090] In some embodiments, the instruction generation module is further used to determine the deployment mode of the matching service from the service deployment information and determine the role of the node in the matching service; based on the deployment mode of the matching service and the role of the node in the matching service, determine the matching instruction template of the node under the matching service from at least one instruction template associated with the matching service.

[0091] In some embodiments, the instruction generation module is further used to create a service instance of the matching service and obtain environment variables of the service instance; and generate service deployment instructions for the node under the matching service based on the environment variables and the matching instruction template.

[0092] In some embodiments, the target cluster is a heterogeneous cluster, each service is adapted to multiple architectures, and the node adapts and executes the service deployment instruction according to the architecture of the node.

[0093] In some embodiments, the instruction generation module is also used to generate a service deployment instruction set for the node under the matching service based on the matching instruction template, and write the service deployment instruction set into the instruction table of the node to send the service deployment instructions in the instruction table to the node; the device also includes a status recording module, the status recording module is used to: generate a service instance node record between the matching service and the node based on the service deployment information, insert the service instance node record into the service instance node record table, the service instance node record is used to record the correspondence between the service instance of the matching service and the node; when the service deployment instruction in the instruction table is executed, the status of the service instance node record is updated to executed.

[0094] In some embodiments, the instruction issuing module is also used to write the service deployment instruction of the node under the matching service into the message queue corresponding to the node, so that the node reads the service deployment instruction from the message queue and executes the service deployment instruction to deploy the matching service in the node.

[0095] Each module in the high-performance computing cluster deployment device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0096] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the high-performance computing cluster deployment method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a high-performance computing cluster deployment method is implemented.

[0097] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for deploying a high-performance computing cluster. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0098] Those skilled in the art will understand that Figure 6 and Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0099] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned high-performance computing cluster deployment method when executing the computer program.

[0100] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned high-performance computing cluster deployment method are implemented.

[0101] In some embodiments, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned high-performance computing cluster deployment method when executed by a processor.

[0102] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0103] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0104] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A high performance computing cluster deployment method, characterized in that: The method comprises: In response to a service deployment instruction for a target cluster, determining service deployment information set for the target cluster, the service deployment information including a correspondence between nodes in the target cluster and services in a plurality of services, the plurality of services including a target scheduling service; Determine a matching service for a node in the target cluster according to the service deployment information, where the matching service for the node is a service corresponding to the node; Determining a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service, and generating a service deployment instruction for the node under the matching service according to the matching instruction template; The service deployment instruction of the node under the matching service is sent to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

2. The method according to claim 1, characterized in that The determining, from at least one instruction template associated with the matching service, a matching instruction template for the node under the matching service, includes: Determining a deployment mode of the matching service from the service deployment information, and determining a role of the node in the matching service; Based on the deployment mode of the matching service and the role of the node in the matching service, a matching instruction template for the node under the matching service is determined from at least one instruction template associated with the matching service.

3. The method according to claim 2, characterized in that Generating the service deployment instruction for the node under the matching service according to the matching instruction template includes: Creating a service instance of the matching service and obtaining environment variables of the service instance; Generate a service deployment instruction for the node under the matching service according to the environment variable and the matching instruction template.

4. The method according to any one of claims 1 to 3, characterized in that The target cluster is a heterogeneous cluster, each of the services is adapted to multiple architectures, and the node adapts and executes the service deployment instruction according to the architecture of the node.

5. The method according to any one of claims 1 to 3, characterized in that Generating the service deployment instruction for the node under the matching service according to the matching instruction template includes: Generating a service deployment instruction set for the node under the matching service according to the matching instruction template, and writing the service deployment instruction set into an instruction table of the node, so as to send the service deployment instructions in the instruction table to the node; The method further comprises: Generate a service instance node record between the matching service and the node according to the service deployment information, and insert the service instance node record into a service instance node record table, wherein the service instance node record is used to record the corresponding relationship between the service instance of the matching service and the node; When the service deployment instruction in the instruction table is executed, the status of the service instance node record is updated to executed.

6. The method according to any one of claims 1 to 3, characterized in that The sending of the service deployment instruction of the node under the matching service to the node includes: Writing the service deployment instruction of the node under the matching service into the message queue corresponding to the node, so that the node reads the service deployment instruction from the message queue and executes the service deployment instruction to deploy the matching service in the node.

7. A high-performance computing cluster deployment device, characterized in that: The device comprises: a deployment information determination module, configured to determine, in response to a service deployment instruction for a target cluster, service deployment information set for the target cluster, the service deployment information including a correspondence between nodes in the target cluster and services in a plurality of services, the plurality of services including a target scheduling service; a matching service determination module, configured to determine a matching service for a node in the target cluster according to the service deployment information, where the matching service for the node is a service corresponding to the node; an instruction generation module, configured to determine a matching instruction template for the node under the matching service from at least one instruction template associated with the matching service, and generate a service deployment instruction for the node under the matching service according to the matching instruction template; An instruction issuing module is used to issue the service deployment instruction of the node under the matching service to the node, so that the node executes the service deployment instruction to deploy the target cluster as a high-performance computing cluster based on the target scheduling service.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images