Cluster deployment method and apparatus, computer device, readable storage medium and program product

Abstract

The application relates to a cluster deployment method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring cluster configuration information corresponding to each cluster category and disk information corresponding to each node server; based on the cluster configuration information, building a cluster architecture corresponding to each cluster category; respectively performing type analysis on the disk information of each node server to determine the disk type corresponding to each node server; for each node server, based on the disk type, determining a target architecture from the cluster architectures, wherein the target architecture is matched with the cluster category and the node server; and respectively deploying each node server in the corresponding target architecture to obtain a service cluster corresponding to each cluster category. The method can improve the cluster building efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a cluster deployment method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the development of computer technology field, cluster deployment technology appears, when building a cluster, technical personnel need to manually configure complete configuration information required by cluster deployment according to the on-site environment, deploy the cluster, and technical personnel also need to query logs to track the start condition.

[0003] However, since the configuration information required for building a cluster is complex, the cluster is built by manual method, and there is a problem of low cluster building efficiency. SUMMARY

[0004] Therefore, it is necessary to provide a cluster deployment method, device, computer equipment, computer readable storage medium and computer program product capable of improving the cluster building efficiency.

[0005] In a first aspect, the present application provides a cluster deployment method, comprising:

[0006] obtaining cluster configuration information corresponding to each of a plurality of cluster categories, and disk information corresponding to each of a plurality of node servers;

[0007] based on the cluster configuration information, building a cluster architecture of each of the cluster categories;

[0008] respectively analyzing the disk information of each of the node servers to determine the disk type corresponding to each of the node servers;

[0009] for each of the node servers, based on the disk type, determining a target architecture matching the node server from each of the cluster architectures;

[0010] deploying each of the node servers in the corresponding target architecture to obtain a service cluster corresponding to each of the cluster categories.

[0011] In one of the embodiments, the obtaining of the cluster configuration information corresponding to each of the plurality of cluster categories comprises:

[0012] obtaining cluster identification information corresponding to each of the plurality of cluster categories;

[0013] for each of the cluster categories, based on the cluster identification information, collecting information of the cluster category to obtain the cluster configuration information of the cluster category.

[0014] In one embodiment, determining, for each node server, a target architecture matching the cluster category of the node server from each of the cluster architectures based on the disk type includes:

[0015] For each of the node servers, determine the number of disk types included in the node server;

[0016] When the number of the types is one, a target architecture that matches the cluster type of the node server is determined from each of the cluster architectures based on the disk capacity of each disk contained in the node server.

[0017] In one embodiment, the method further includes:

[0018] When there are multiple types, determine the number of disks corresponding to each disk type;

[0019] If the number of disks corresponding to the first disk type among all the disk types is one, then the cluster architecture corresponding to the first disk type is determined as the target architecture that matches the node server.

[0020] In one embodiment, the method further includes:

[0021] In response to a cluster expansion command for a service cluster that has been deployed, determine the configuration file corresponding to the cluster expansion command;

[0022] The cluster expansion information contained in the configuration file is matched with the cluster configuration information. If the match is successful, the expansion content contained in the configuration file is added to the target service cluster corresponding to the target cluster configuration information that is successfully matched with the cluster expansion information.

[0023] If the matching fails, an update service cluster corresponding to the cluster expansion information will be created based on the cluster expansion information.

[0024] In one embodiment, adding the expansion content contained in the configuration file to the target service cluster corresponding to the target cluster configuration information that successfully matches the cluster expansion information includes:

[0025] Determine the expansion type of the expanded content;

[0026] If the expansion content is a disk, then determine the target node server that matches the disk, and add the disk to the target node server;

[0027] If the expansion involves node servers, then the node servers will be added to the target service cluster corresponding to the target cluster configuration information that successfully matches the node servers.

[0028] Secondly, this application also provides a cluster deployment apparatus, comprising:

[0029] The information acquisition module is used to acquire the cluster configuration information corresponding to each of the multiple cluster categories, as well as the disk information corresponding to each of the multiple node servers.

[0030] The cluster architecture building module is used to build the respective cluster architecture for each of the cluster categories based on the cluster configuration information.

[0031] The type analysis module is used to perform type analysis on the disk information of each node server to determine the disk type corresponding to each node server.

[0032] The target architecture determination module is used to determine, for each of the node servers, a target architecture that matches the cluster category of the node server from each of the cluster architectures based on the disk type.

[0033] The cluster deployment module is used to deploy each of the node servers in the corresponding target architecture to obtain the service clusters corresponding to each of the cluster categories.

[0034] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0035] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described above.

[0036] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0037] The aforementioned cluster deployment method, apparatus, computer equipment, computer-readable storage medium, and computer program products acquire cluster configuration information corresponding to multiple cluster categories and disk information corresponding to multiple node servers. This allows for the determination of the basic cluster deployment situation. Based on the cluster configuration information, the cluster architecture for each cluster category is built. This allows for the initial steps of cluster deployment to be completed first, establishing an initial cluster framework without server deployment. Subsequently, the disk information of each node server is analyzed to determine its corresponding disk type. For each node server, based on the disk type, a target architecture matching the cluster category and the node server is determined from each cluster architecture. Each node server is then deployed in its corresponding target architecture, resulting in the service cluster corresponding to each cluster category. Cluster architecture allocation for node servers can be performed according to disk type, ensuring accurate cluster deployment while achieving automated cluster deployment and improving deployment efficiency. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is an application environment diagram of a cluster deployment method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a cluster deployment method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating cluster deployment information in one embodiment;

[0042] Figure 4 This is a flowchart illustrating the cluster deployment method in another embodiment;

[0043] Figure 5 This is a flowchart illustrating the steps of matching the target architecture in one embodiment;

[0044] Figure 6 This is a scenario diagram illustrating the expansion and network connection of three types of clusters in one embodiment;

[0045] Figure 7 This is a flowchart illustrating the cluster expansion and network connection steps in one embodiment;

[0046] Figure 8 This is a flowchart illustrating the cluster deployment method in yet another embodiment;

[0047] Figure 9 This is a structural block diagram of a cluster deployment device in one embodiment;

[0048] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] The cluster deployment method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the management terminal 102 communicates with multiple node servers 104 via a network. The management terminal 102, for example, could be AnsibleTower (Ansible management node), which is essentially a unified management interface for Ansible, similar to a management platform in virtualization. It allows users to intuitively see the status of the managed system through a graphical interface, while providing automated data center management functions combined with operations, logging, auditing, and version control. Node servers 104, in short, are servers that act as independent computing and storage units in the cluster environment. In the cluster, each such server is considered a node, and they are interconnected via a network to collaborate on large-scale computing tasks or store massive amounts of data. Specifically, during the cluster deployment process, the management terminal 102 can obtain the cluster configuration information corresponding to each of the multiple cluster categories, as well as the disk information corresponding to each of the multiple node servers 104; based on the cluster configuration information, it builds the cluster architecture for each cluster category; it performs type analysis on the disk information of each node server 104 to determine the disk type corresponding to each node server 104; for each node server 104, based on the disk type, it determines the target architecture that matches the cluster category with the node server 104 from each cluster architecture; and it deploys each node server 104 in the corresponding target architecture to obtain the service cluster corresponding to each cluster category.

[0051] In one exemplary embodiment, such as Figure 2 As shown, a cluster deployment method is provided, which can be applied to... Figure 1 Taking the management terminal 102 as an example, the explanation includes the following steps S202 to S210. Wherein:

[0052] Step S202: Obtain the cluster configuration information corresponding to each of the multiple cluster categories, as well as the disk information corresponding to each of the multiple node servers.

[0053] A cluster consists of multiple node servers that are interconnected via a network and collaborate to provide high-performance, scalable services. There are several types of clusters, such as management clusters and storage clusters. Cluster configuration information refers to the information required for cluster deployment, such as network configuration information for different network interfaces of node servers, system kernel architecture information, software versions and disk array information, storage multipath configuration, etc. A node server, simply put, is a server that acts as an independent computing and storage unit in a cluster environment. In a cluster, each such server is considered a node, and they are interconnected via a network to collaborate on large-scale computing tasks or store massive amounts of data. Disk information refers to the information about the disks contained in the node servers; for example, disk information may include disk type, disk identifier, disk address, etc.

[0054] Specifically, when deploying a cluster, the management terminal needs to first determine the cluster category, the cluster configuration information corresponding to each cluster category, and the disk information of all node servers involved in the cluster deployment process. This establishes the initial deployment status for subsequent cluster deployment. For example, obtaining the cluster configuration information for multiple cluster categories and the disk information for multiple node servers can be done actively or passively. In one specific embodiment, the management terminal can directly obtain the cluster identification information for each cluster to be built, and for each cluster, collect information based on the cluster identification information to obtain the cluster configuration information. In another specific embodiment, the management terminal can also obtain the cluster configuration information for multiple cluster categories and the disk information for multiple node servers collected by the information collection device.

[0055] Step S204: Based on the cluster configuration information, build the respective cluster architecture for each cluster category.

[0056] In this context, cluster architecture refers to the architectural design of combining multiple node servers into a whole system. In this embodiment, cluster architecture specifically refers to a cluster framework structure that already has complete cluster configuration information but has not yet deployed node servers.

[0057] Specifically, after obtaining the cluster configuration information corresponding to each of the multiple cluster categories, the management terminal can build the corresponding cluster architecture for each cluster type based on the above cluster configuration information, so as to facilitate the subsequent deployment of node servers.

[0058] Step S206: Perform type analysis on the disk information of each node server to determine the corresponding disk type of each node server.

[0059] The disk types can include: OST (Object Storage Target): a component device in the Lustre file system used to store the actual data of files; MDT (Metadata Target): a component device in the Lustre file system used to store the metadata of files, such as file size, creation time, permissions, etc.; and MGT (Management Target): a component device in the Lustre file system used to store file system management information and configuration data.

[0060] Specifically, after building the cluster architecture for each cluster category, node servers can be deployed for each cluster architecture. First, the node servers need to be distinguished. That is, the management end needs to perform type analysis on the disk information of each node server to determine the corresponding disk type of each node server.

[0061] Step S208: For each node server, based on the disk type, determine the target architecture that matches the cluster category and the node server from each cluster architecture.

[0062] The target architecture is the cluster architecture selected for a specific operation.

[0063] Specifically, after determining the disk type corresponding to each node server, the management terminal can determine the corresponding cluster architecture for each node server. That is, for each node server, the management terminal can determine the cluster category corresponding to each cluster architecture based on the disk type of the node server, and determine the target architecture that matches the cluster category of the node server from each cluster architecture. In some specific embodiments, the management terminal can determine the number of disk types contained in each node server. If the number of types is one, the target architecture that matches the cluster category of the node server is determined from each cluster architecture based on the disk capacity of each disk contained in the node server. If the number of types is multiple, the number of disks corresponding to each disk type is determined. If the number of disks corresponding to the first disk type is one, then the cluster architecture corresponding to the first disk type is determined as the target architecture that matches the node server.

[0064] Step S210: Deploy each node server in the corresponding target architecture to obtain the service clusters corresponding to each cluster category.

[0065] Among them, the service cluster is a cluster that has been fully deployed and is ready to provide services.

[0066] Specifically, after determining the target architecture that each node server matches, the management end can deploy each node server in the corresponding target architecture to obtain the service clusters corresponding to each cluster category.

[0067] The above-described cluster deployment method obtains the cluster configuration information corresponding to each of the multiple cluster categories and the disk information corresponding to each of the multiple node servers. This allows for the determination of the basic cluster deployment situation. Based on the cluster configuration information, the cluster architecture for each cluster category is built. This method can first complete the preliminary steps of cluster deployment, building an initial cluster framework without server deployment. Then, the disk information of each node server is analyzed to determine the disk type corresponding to each node server. For each node server, based on the disk type, the target architecture matching the cluster category and the node server is determined from each cluster architecture. Each node server is then deployed in the corresponding target architecture, resulting in the service cluster corresponding to each cluster category. Cluster architecture allocation for node servers can be performed according to disk type. This ensures the accuracy of cluster deployment while achieving automated cluster deployment and improving the efficiency of cluster deployment.

[0068] In an exemplary embodiment, obtaining cluster configuration information corresponding to each of multiple cluster categories includes: obtaining cluster identification information corresponding to each of the multiple cluster categories; and for each cluster category, collecting information on the cluster category based on the cluster identification information to obtain the cluster configuration information of the cluster category.

[0069] Among them, cluster identification information is specific information used to uniquely identify and distinguish clusters.

[0070] Specifically, the management terminal can first obtain the cluster identification information corresponding to each of the multiple cluster categories, and based on the cluster identification information, collect information for each cluster category to obtain the cluster configuration information corresponding to each of the multiple cluster categories.

[0071] In one specific embodiment, the management terminal can first obtain cluster identification information corresponding to each of the multiple cluster categories for cluster deployment. This information is relatively simple and cannot be used to build a complete cluster at the moment. Then, the Ansible module of Python is used to parse out a new configuration file. Based on the script file with the configuration items written, it performs the function of information collection, such as collecting network configuration information of different network interfaces of node servers, system kernel architecture information, software version and disk array information, storage multi-path configuration, etc. Finally, after completing the collection and organization of information about the controlled node servers, the cluster configuration information required to build the cluster is obtained.

[0072] In this embodiment, the cluster identification information corresponding to each cluster category is obtained first, and then information is collected for each cluster category based on the cluster identification information. This is different from the traditional deployment method that requires manual configuration of cluster configuration information, and can avoid the inefficiency caused by manual configuration.

[0073] In an exemplary embodiment, for each node server, based on the disk type, a target architecture matching the cluster category of the node server is determined from each cluster architecture, including: for each node server, determining the number of disk types contained in the node server; and if the number of types is one, determining the target architecture matching the cluster category of the node server from each cluster architecture based on the disk capacity of each disk contained in the node server.

[0074] Disk capacity, also known as hard disk capacity, is a parameter that measures the ability of a computer's storage device to store data.

[0075] Specifically, the matching process between cluster architecture and node servers is related to the number and types of disks, as well as their capacity. For each node server, the management terminal needs to first determine the number of disk types it contains. If the number of disk types is only one, it indicates that the node server contains only one type of disk. Then, based on the capacity of each disk within the node server, the target architecture for matching the cluster type with the node server needs to be determined from the various cluster architectures. In a specific embodiment, if all disk types meet the predefined OST (Object Storage Target) specification range, then the cluster architecture OSS is used as the target architecture for matching the node server; if all disk types meet the predefined MDT (Metadata Target) specification range, then the cluster architecture MDS is used as the target architecture for matching the node server; and if all disk types meet the predefined MGT (Management Target) rules, then the cluster architecture MGS is used as the target architecture for matching the node server.

[0076] After the initial assessment, the system will again check whether the number of MDS and OSS required for this deployment meets the requirements. That is, whether there is at least one pair of MGS (Management Server) and one pair of OSS to build the cluster. If the requirements are not met, it means that there is an integration error and the requirements are not met. If the requirements are met, the subsequent deployment steps will continue.

[0077] In this embodiment, the number of disk types contained in the node server is first determined. If the number of disk types is one, the target architecture is determined based on the disk capacity of each disk contained in the node server, which can ensure the accuracy of the matching between the cluster architecture and the node server.

[0078] In an exemplary embodiment, the cluster deployment method further includes: when there are multiple types, determining the number of disks corresponding to each disk type; if the number of disks corresponding to the first disk type is one, then determining the cluster architecture corresponding to the first disk type as the target architecture matching the node server.

[0079] In this embodiment, the first disk type refers to the disk type corresponding to mgt.

[0080] Specifically, when there are multiple disk types, the number of disks corresponding to each disk type is determined. If the number of disks corresponding to the first disk type (mgt) is one, and the number of disks corresponding to the other disk types is mdt, then the cluster architecture corresponding to the first disk type is determined as the target architecture that matches the node server, which can ensure the accuracy of the matching between the cluster architecture and the node server.

[0081] In an exemplary embodiment, the cluster deployment method further includes: in response to a cluster expansion instruction for a service cluster after deployment, determining a configuration file corresponding to the cluster expansion instruction; matching the cluster expansion information contained in the configuration file with the configuration information of each cluster; if the match is successful, adding the expansion content contained in the configuration file to the target service cluster corresponding to the target cluster configuration information that matches the cluster expansion information; if the match fails, creating an update service cluster corresponding to the cluster expansion information based on the cluster expansion information.

[0082] Among them, cluster expansion commands refer to instructions for expanding the capacity of a cluster, such as expanding the cluster itself or expanding the capacity of individual node servers. Configuration files are files that configure or initialize the parameters corresponding to cluster expansion. Specifically, configuration files contain cluster expansion information, which mainly represents the specific details of the cluster expansion, such as the specific content of the expansion.

[0083] Specifically, when the original cluster capacity and carrying capacity face significant challenges due to increased user numbers or user demands for new types of storage devices and network types, technicians need to expand and network the underlying storage cluster to enable it to operate effectively in large-scale HPC and other application scenarios requiring high throughput and low latency. There are three types of cluster expansion and networking scenarios: The first is linear expansion of Lustre storage components OST and MDT, which involves adding storage hardware media to server nodes and adding the new node servers to the cluster to achieve linear capacity expansion; the second is distributed expansion by adding Lustre disks MDS (Metadata Server) and OSS (Object Storage Server), where the node server has newly mapped disk arrays, expanding the original cluster with new nodes and storage media to achieve distributed expansion, increasing capacity and improving IO read / write performance; the third is expanding the original management terminal to build new types of clusters, such as clusters with high-performance storage media or high-performance network bandwidth.

[0084] For example, in response to a cluster expansion command for a service cluster after deployment, the management terminal can determine the configuration file corresponding to the cluster expansion command, match the cluster expansion information contained in the configuration file with the configuration information of each cluster, and if the match is successful, it indicates that the original cluster contains the above-mentioned cluster expansion information. Therefore, the expansion content contained in the configuration file needs to be added to the target service cluster corresponding to the target cluster configuration information that has successfully matched the cluster expansion information; if the match fails, an update service cluster corresponding to the cluster expansion information is created based on the cluster expansion information.

[0085] In this embodiment, after receiving the cluster expansion instruction, the cluster can be expanded according to the cluster expansion information contained in the cluster expansion instruction, thereby achieving better load balancing and performance optimization.

[0086] In an exemplary embodiment, adding the expansion content contained in the configuration file to the target service cluster corresponding to the target cluster configuration information that successfully matches the cluster expansion information includes: determining the expansion type of the expansion content; if the expansion content is a disk, determining the target node server that matches the disk and adding the disk to the target node server; if the expansion content is a node server, adding the node server to the target service cluster corresponding to the target cluster configuration information that successfully matches the node server.

[0087] Specifically, if the cluster expansion information contained in the configuration file matches the configuration information of each cluster, expansion can be performed based on the expansion type of the expansion content. If the expansion content is a disk, the target node server that matches the disk is determined and the disk is added to the target node server. If the expansion content is a node server, the node server is added to the target service cluster corresponding to the target cluster configuration information that matches the node server.

[0088] In this embodiment, further expansion can be carried out based on the expansion content, thereby achieving better load balancing and performance optimization.

[0089] In one specific embodiment, a cluster deployment method for a specific application scenario is provided, including:

[0090] First, build a highly readable and easily extensible configuration file based on the inventory file.

[0091] Table 1. Example of deployment configuration in Inventory format

[0092]

[0093] As shown in Table 1, the Inventory file is the configuration file for the Ansible operations and maintenance tool to manage host information. It can easily build a list of host names for various types of hosts. Variable configurations for hosts are specified using common custom host types and "vars" and "children" configurations. For example, the configuration items under "all:vars" represent public configuration information for global nodes, and the configuration items under "typeX:vars" represent configuration information for nodes of type X. Nodes under "typeX:children" represent multiple nodes of that type. Table 1 shows a simplified inventory configuration file required for deployment. cm* represents the management node, and cm:vars represents the public network configuration required for the management node.

[0094] Traditional deployment methods rely on manually configuring the complete configuration information required for one-click deployment. This process is cumbersome, prone to errors, and difficult to troubleshoot. This method aims to collect essential configuration items that cannot be detected from nodes, such as specific configurations like the keepalived virtual IP, CN2 gateway, and cluster name. The process of parsing, collecting, and building the cluster deployment information required for one-click deployment is as follows: Figure 3 As shown:

[0095] First, the cluster identification information used for cluster deployment is obtained. This information is relatively simple and cannot be used to build a complete cluster at this stage. Then, the Ansible module in Python is used to parse out a new configuration file. Based on a script file containing pre-written configuration items, it performs information collection functions, such as collecting network configuration information for different network interfaces of the node servers, system kernel architecture information, software versions, disk array information, storage multipath configuration, etc. Finally, after collecting and organizing the information about the controlled node servers, the cluster configuration information required to build the cluster is obtained. This inventory-based approach allows for highly flexible configuration, addition, and modification of the required cluster environment configuration.

[0096] Secondly, Ansible's multi-point control automated deployment strategy based on integration features.

[0097] The above describes the configuration files that need to be prepared in advance when building a cluster. These files can be parsed into configuration items required for management and backend storage. After obtaining these configuration items, they can be applied to the deployment of Lustre storage clusters and management clusters. The initial deployment process is as follows: Figure 4 As shown, the detailed steps are as follows:

[0098] S401. Determine the cluster configuration information corresponding to each cluster category:

[0099] This configuration file contains the specified information required by the cluster, such as cluster name, lnet network type, management database type, various network devices and IP addresses, and information such as master-slave host pairs for high availability; after being parsed by the ansible module of Python, this type of configuration file obtains complete deployment configuration items for management and backend nodes.

[0100] S402. Parallel distribution of data packets and scripts to be deployed on the node server based on Ansible:

[0101] By leveraging Ansible's multi-point control and data distribution capabilities, the Lustre and control installation packages required for deployment, as well as the scripts called during the deployment process, are transmitted.

[0102] S403. Determine the target architecture for each node server;

[0103] In this step, the system architecture and network configuration information of the cluster are first collected in s3.1. This information is used to install the installation dependency packages for the corresponding architecture and kernel version, set up the firewall, configure the lnet network, configure array multipath, and collect features to be sent back to the deployment nodes. Then, during the deployment process in s3.2, the component identity (target architecture) of the node server can be identified based on the disk and high availability host information. After identification, the disk is formatted and the cluster is built.

[0104] The general process of identifying the identity based on the integrated component is as follows: Figure 5 As shown, the detailed steps are as follows:

[0105] Step S501: For each node server, determine the number of disk types contained in the node server.

[0106] Step S502: When there is only one type, based on the disk capacity of each disk contained in the node server, determine the target architecture that matches the cluster type and the node server from each cluster architecture.

[0107] Step S503: When there are multiple types, determine the number of disks corresponding to each disk type. If the number of disks corresponding to the first disk type is one, then determine the cluster architecture corresponding to the first disk type as the target architecture that matches the node server.

[0108] Specifically, the ranges of the Lustre components MDT, MGT, and OST disks are pre-defined. MDT is used to store metadata, MGT to store management data, and OST to store cluster data. In a production environment, the capacities of these three types of disks have clear boundaries. Initially, the sequence numbers of MDT and OST are both 0. The node server is serialized and traversed, and the disks requiring formatting are identified based on the storage multipath of the disk array. The disks are traversed sequentially to obtain their names and capacities. After traversing all array disks of the storage server node, its Lustre component identity is determined based on integration characteristics, such as satisfying the criteria of a single MGT, ... If the remaining conditions are MDT, the node is identified as MGS. If the conditions of the three types of servers are met, the component's identity is marked, and the MDT and OST sequence numbers are updated. The process continues to traverse the next pair of storage server nodes. Otherwise, if the conditions of the three types of storage servers are not met, it is determined that the integration configuration is incorrect and does not meet the requirements, and the execution fails. After traversing all nodes, it will again check whether the number of MDS nodes and OSS nodes required for this deployment meets the requirements. There must be at least one pair of MGS and one pair of OSS for building the cluster. If this is not met, it means that the integration is incorrect and does not meet the requirements. If it is met, the subsequent deployment steps continue.

[0109] s404. Build a highly available cluster;

[0110] After the cluster is built, it does not yet have high availability capabilities. High availability needs to be configured based on the network information of the host pairs.

[0111] S405. Cluster underlying performance test;

[0112] At this point, the high-availability cluster has been successfully built, and the underlying testers can use it to test the performance of the underlying storage cluster without being limited by the control end, thus achieving the separation of control and backend deployment;

[0113] S406. Collect network and disk information for deploying the underlying storage;

[0114] S407. Deploy the management terminal (management cluster) and the database used.

[0115] The above deployment process achieves separate deployment of management and storage clusters, and identifies component identities based on disk arrays. This process only requires modification of the configuration file during expansion, and can realize management reuse, mixed deployment of multiple types of storage clusters, and cluster expansion and networking in the production environment.

[0116] The previous section introduced the initial deployment process of the Lustre cluster. However, as the number of users increases or users have new requirements for storage devices and network types, the original cluster capacity and carrying capacity may face significant challenges. At this point, technicians need to expand and connect the underlying storage cluster to the network so that it can operate effectively in large-scale HPC and other application scenarios that require high throughput and low latency. Figure 6 The demonstration showcased three scenarios for cluster expansion and networking. The first scenario involves linear expansion of Lustre storage components OST and MDT, which allows for the addition of storage hardware media to server nodes, thereby linearly expanding the cluster capacity. The second scenario involves adding Lustre disks MDS (Metadata Server) and OSS (Object Storage Server) to achieve distributed expansion. These node servers have newly mapped disk arrays, expanding the original cluster with new nodes and storage media to achieve distributed expansion, increasing capacity and improving IO read / write performance. The third scenario involves expanding the original control terminal to create new types of clusters, such as clusters with high-performance storage media or high-performance network bandwidth.

[0117] The flowchart for cluster expansion and network connection is as follows: Figure 7 As shown, the steps for cluster expansion and network connection are as follows:

[0118] Step S701: In response to the cluster expansion command for the service cluster after deployment, determine the configuration file corresponding to the cluster expansion command.

[0119] Step S702: Match the cluster expansion information contained in the configuration file with the configuration information of each cluster; Step S703: If the match is successful, determine the expansion type of the expansion content.

[0120] Step S704: If the expansion content is a disk, then determine the target node server that matches the disk and add the disk to the target node server.

[0121] Step S705: If the expansion content is a node server, then add the node server to the target service cluster corresponding to the target cluster configuration information that successfully matches the node server.

[0122] Step S706: If the matching fails, a new update service cluster corresponding to the cluster expansion information is created based on the cluster expansion information.

[0123] Specifically, the process first checks whether the cluster expansion information in the configuration file already exists in the management cluster. If it exists, it determines that the cluster already in the management cluster is being expanded and networked. If it does not exist, the newly created Lustre storage cluster on the management terminal is being reused. If the cluster already exists, it checks whether the node servers are already in the cluster. If they exist, it collects the newly added storage media on the node servers and merges these storage disks into the storage cluster. If the node servers are not yet in the cluster, it collects the integration information such as storage media, network configuration, and system architecture on the new node servers, and after deployment, merges these new storage servers into the existing cluster. If a new type of cluster is being built, the deployment process is the same as the initial deployment, and it must finally be merged into the management terminal. The above cluster expansion and network integration operation realizes the reuse of the same management terminal and allows for flexible expansion configuration of cluster capacity, new distributed nodes, and new types of clusters. After expansion, better load balancing and performance optimization can be achieved.

[0124] In a specific embodiment, such as Figure 8 As shown, a cluster deployment method is also provided, including:

[0125] Step S801: Obtain the cluster identification information corresponding to each of the multiple cluster categories, and the disk information corresponding to each of the multiple node servers;

[0126] Step S802: For each cluster category, information is collected based on the cluster identification information to obtain the cluster configuration information of the cluster category;

[0127] Step S803: Based on the cluster configuration information, build the cluster architecture for each cluster category, and perform type analysis on the disk information of each node server to determine the corresponding disk type for each node server.

[0128] Step S804: For each node server, determine the number of disk types contained in the node server.

[0129] Step S805: When there is only one type, based on the disk capacity of each disk contained in the node server, determine the target architecture that matches the cluster type and the node server from each cluster architecture.

[0130] Step S806: When there are multiple types, determine the number of disks corresponding to each disk type. If the number of disks corresponding to the first disk type is one, then determine the cluster architecture corresponding to the first disk type as the target architecture to match the node server.

[0131] Step S807: Deploy each node server in the corresponding target architecture to obtain the service clusters corresponding to each cluster category.

[0132] Step S808: In response to the cluster expansion command for the service cluster after deployment, determine the configuration file corresponding to the cluster expansion command.

[0133] Step S809: Match the cluster expansion information contained in the configuration file with the configuration information of each cluster; Step S810: If the match is successful, determine the expansion type of the expansion content.

[0134] Step S811: If the expansion content is a disk, then determine the target node server that matches the disk and add the disk to the target node server.

[0135] Step S812: If the expansion content is a node server, then add the node server to the target service cluster corresponding to the target cluster configuration information that successfully matches the node server.

[0136] Step S813: If the matching fails, a new update service cluster corresponding to the cluster expansion information is created based on the cluster expansion information.

[0137] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0138] Based on the same inventive concept, this application also provides a cluster deployment apparatus for implementing the cluster deployment method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more cluster deployment apparatus embodiments provided below can be found in the limitations of the cluster deployment method described above, and will not be repeated here.

[0139] In one exemplary embodiment, such as Figure 9 As shown, a cluster deployment device 900 is provided, including: an information acquisition module 902, a cluster architecture building module 904, a type analysis module 906, a target architecture determination module 908, and a cluster deployment module 910, wherein:

[0140] The information acquisition module 902 is used to acquire the cluster configuration information corresponding to each of the multiple cluster categories, as well as the disk information corresponding to each of the multiple node servers.

[0141] The cluster architecture building module 904 is used to build the cluster architecture for each cluster category based on the cluster configuration information.

[0142] Type analysis module 906 is used to perform type analysis on the disk information of each node server to determine the disk type of each node server.

[0143] The target architecture determination module 908 is used to determine the target architecture that matches the cluster category and the node server based on the disk type from each cluster architecture for each node server.

[0144] The cluster deployment module 910 is used to deploy each node server in the corresponding target architecture to obtain the service clusters corresponding to each cluster category.

[0145] In an exemplary embodiment, the information acquisition module 902 is specifically used to: acquire cluster identification information corresponding to each of the multiple cluster categories;

[0146] For each cluster category, information is collected based on the cluster identification information to obtain the cluster configuration information for that cluster category.

[0147] In an exemplary embodiment, the target architecture determination module 908 is specifically used to: determine the number of disk types contained in each node server;

[0148] When there is only one type, the target architecture that matches the cluster type and the node server is determined from each cluster architecture based on the disk capacity of each disk contained in the node server.

[0149] In an exemplary embodiment, the cluster deployment device 900 further includes a target architecture matching module, specifically used to: determine the number of disks corresponding to each disk type when there are multiple types;

[0150] If the number of disks corresponding to the first disk type in each disk type is one, then the cluster architecture corresponding to the first disk type will be determined as the target architecture to match the node server.

[0151] In one exemplary embodiment, the cluster deployment apparatus 900 further includes a cluster expansion module, comprising:

[0152] The instruction response unit is used to respond to cluster expansion instructions for the service cluster after deployment and to determine the configuration file corresponding to the cluster expansion instructions.

[0153] The content addition unit is used to match the cluster expansion information contained in the configuration file with the configuration information of each cluster. If the match is successful, the expansion content contained in the configuration file is added to the target service cluster corresponding to the target cluster configuration information that has successfully matched the cluster expansion information.

[0154] The cluster creation unit is used to create a new update service cluster corresponding to the cluster expansion information if a match fails.

[0155] In one exemplary embodiment, the content addition unit is specifically used to: determine the expansion type of the expanded content;

[0156] If the expansion involves a disk, then identify the target node server that matches the disk and add the disk to the target node server.

[0157] If the expansion involves node servers, then the node servers will be added to the target service cluster corresponding to the target cluster configuration information that successfully matches the node servers.

[0158] Each module in the aforementioned cluster deployment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0159] In one exemplary embodiment, a computer device is provided, which may be a management terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with an external management terminal; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a cluster deployment method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0160] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0161] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A cluster deployment method, characterized in that, The method includes: Obtain cluster configuration information for each of the multiple cluster categories, as well as disk information for each of the multiple node servers; Based on the cluster configuration information, construct the respective cluster architecture for each of the aforementioned cluster categories; Each node server's disk information is analyzed to determine its corresponding disk type. The disk type includes metadata target, management target, and object data storage target. The metadata target is used to store metadata, the management target is used to store management data, and the object data storage target is a cluster data storage disk. For each node server, based on the disk type, a target architecture that matches the cluster category of the node server is determined from each of the cluster architectures. Each of the node servers is deployed in its corresponding target architecture to obtain the service cluster corresponding to each of the cluster categories.

2. The method according to claim 1, characterized in that, The process of obtaining cluster configuration information corresponding to each of the multiple cluster categories includes: Obtain the cluster identifier information corresponding to each of the multiple cluster categories; For each cluster category, information is collected based on the cluster identification information to obtain the cluster configuration information for that cluster category.

3. The method according to claim 1, characterized in that, For each node server, determining the target architecture that matches the cluster category of the node server from each cluster architecture based on the disk type includes: For each of the node servers, determine the number of disk types included in the node server; When the number of the types is one, a target architecture that matches the cluster type of the node server is determined from each of the cluster architectures based on the disk capacity of each disk contained in the node server.

4. The method according to claim 2, characterized in that, The method further includes: When there are multiple types, determine the number of disks corresponding to each disk type; If the number of disks corresponding to the first disk type among all the disk types is one, then the cluster architecture corresponding to the first disk type is determined as the target architecture that matches the node server.

5. The method according to claim 1, characterized in that, The method further includes: In response to a cluster expansion command for a service cluster that has been deployed, determine the configuration file corresponding to the cluster expansion command; The cluster expansion information contained in the configuration file is matched with the cluster configuration information. If the match is successful, the expansion content contained in the configuration file is added to the target service cluster corresponding to the target cluster configuration information that is successfully matched with the cluster expansion information. If the matching fails, an update service cluster corresponding to the cluster expansion information will be created based on the cluster expansion information.

6. The method according to claim 5, characterized in that, The step of adding the expansion content contained in the configuration file to the target service cluster corresponding to the target cluster configuration information that successfully matches the cluster expansion information includes: Determine the expansion type of the expanded content; If the expansion content is a disk, then determine the target node server that matches the disk, and add the disk to the target node server; If the expansion involves node servers, then the node servers will be added to the target service cluster corresponding to the target cluster configuration information that successfully matches the node servers.

7. A cluster deployment device, characterized in that, The device includes: The information acquisition module is used to acquire the cluster configuration information corresponding to each of the multiple cluster categories, as well as the disk information corresponding to each of the multiple node servers. The cluster architecture building module is used to build the respective cluster architecture for each of the cluster categories based on the cluster configuration information. The type analysis module is used to perform type analysis on the disk information of each node server to determine the disk type corresponding to each node server. The disk type includes metadata target, management target and object data storage target. The metadata target is used to store metadata, the management target is used to store management data, and the object data storage target is a cluster data storage disk. The target architecture determination module is used to determine, for each of the node servers, a target architecture that matches the cluster category of the node server from each of the cluster architectures based on the disk type. The cluster deployment module is used to deploy each of the node servers in the corresponding target architecture to obtain the service clusters corresponding to each of the cluster categories.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images