A system and method for deploying big data clusters on a cloud service platform

Abstract

The present invention provides a system and method for deploying a big data cluster on a cloud service platform, the method steps: generating a resource library module for virtual machine cluster deployment, storing image libraries, storage pools, host clusters and network resources, and creating a query interface for resources in the resource library module to form a resource library interface module; when the user interface module obtains the user's cluster deployment startup, it outputs the corresponding query interface of the resource library interface module to the user to obtain the specific parameters of the resource library module selected by the user; according to the specific parameters of the resource library module selected by the user, the corresponding resources are selected from the resource library module, the virtual machine cluster deployment is executed, and the management virtual machine and the working virtual machine are deployed respectively; the virtual machine cluster that has been deployed is run, and each virtual machine is set. The present invention realizes the high efficiency of cloud service platform deployment of big data clusters, saves deployment time, has good security, and can bring a good user experience.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud platform cluster deployment, and specifically relates to a system and method for deploying a big data cluster on a cloud service platform. Background Art

[0002] The existing cloud service platform uses a separate deployment method to deploy big data clusters. The separate deployment method requires first deploying virtual machines separately, and then wait for all virtual machines to be deployed before configuring the cluster relationship. If there are many virtual machines, the waiting time will be very long. This separate deployment method has the following problems: first, the deployment time is long, so the efficiency is extremely low; second, the technical requirements are high, so it is not convenient for customers to use; third, the technical requirements for operation and maintenance personnel in the event of a failure are high, and the cost of use is high.

[0003] This is a shortcoming of the prior art. Therefore, in view of the above-mentioned defects in the prior art, it is very necessary to provide a system and method for deploying a big data cluster on a cloud service platform. Summary of the invention

[0004] In view of the defects of the prior art that the existing cloud server platform deploys clusters in a separate deployment manner, which has a long deployment time, low efficiency, high technical requirements and high usage costs, the present invention provides a system and method for deploying big data clusters on a cloud service platform to solve the above technical problems.

[0005] In a first aspect, the present invention provides a system for deploying a big data cluster on a cloud service platform, comprising:

[0006] The resource library module is used to store image libraries, virtual machine templates, storage pools, host clusters, and network resources; virtual machine templates include management virtual machine templates and working virtual machine templates;

[0007] The resource library interface module is used to provide image library list query interface, template list query interface, storage pool list query interface, host cluster list query interface, network list query interface and big data cluster deployment interface;

[0008] The user interface module is used to obtain the corresponding query interface of the resource library interface module to the user when the user's cluster deployment is started, so as to obtain the specific parameters of the resource library module selected by the user;

[0009] The cluster deployment execution module is used to select the corresponding resources of the resource library module according to the specific parameters of the resource library module selected by the user, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine respectively;

[0010] The cluster operation module is used to run the deployed virtual machine cluster and set up each virtual machine.

[0011] Furthermore, the resource library module includes:

[0012] The image library unit is used to store the image library required for the deployment of the virtual machine cluster;

[0013] A virtual machine template unit, used to store virtual machine templates required for virtual machine cluster deployment;

[0014] The storage pool unit is used to store the system disks and data disks required for the deployment of the virtual machine cluster, and to generate a storage pool for the system disks and data disks;

[0015] A host cluster unit is used to store the host cluster required for virtual machine cluster deployment;

[0016] The network storage unit is used to store the management network and business network required for the deployment of the virtual machine cluster.

[0017] Furthermore, the resource library interface module includes:

[0018] The image library list query interface is used to obtain all image libraries required for virtual machine cluster deployment, generate an image library list, and provide it to users for image library configuration query;

[0019] The template list query interface is used to obtain the working virtual machine templates and management virtual machine templates required for virtual machine cluster deployment, generate a template list, and provide it to users for template configuration query;

[0020] The storage pool list query interface is used to obtain the storage pools required for virtual machine cluster deployment, generate a storage pool list, and provide it to users for storage pool configuration query;

[0021] The host cluster list interface is used to obtain the host clusters required for virtual machine cluster deployment, generate a host cluster list, and provide it to users for host cluster configuration query;

[0022] The network list interface is used to obtain the network IP, gateway, and mask required for the user's virtual machine cluster deployment, generate a network list, and provide it to the user for network information configuration query;

[0023] The big data cluster deployment interface is used to receive operations from users to start virtual machine cluster deployment.

[0024] Furthermore, the user interface module includes:

[0025] A user virtual machine cluster deployment startup unit, used to start the virtual machine cluster deployment when obtaining the user's deployment startup through the big data cluster deployment interface;

[0026] A user virtual machine cluster name interface unit is used to generate a prompt interface, obtain the virtual machine cluster name and cluster description information, and perform duplication check on the virtual machine cluster name;

[0027] The user management virtual machine configuration interface unit is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the management virtual machine template selected by the user through the template list query interface, and configure the virtual machine name prefix, CPU, memory, data disk size and number of data disks;

[0028] A user working virtual machine interface configuration unit is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the working virtual machine template selected by the user through the template list query interface, and configure the name prefix, CPU, memory, data disk size, and number of data disks;

[0029] A user host cluster interface unit, used to generate a host cluster selection interface, and select hosts under a host cluster through a host cluster list interface, wherein the hosts include hosts under the same host cluster and hosts under different host clusters;

[0030] The user host configuration interface unit is used to generate a host configuration interface, obtain the host selected by the user, and configure the host's system disk, data disk, management network, and business network options;

[0031] A user virtual machine network information interface unit is used to generate a virtual machine information display interface, display virtual machine information, and obtain the user's configuration of virtual machine node network information through a network information query interface;

[0032] The parameter object generation unit is used to obtain the virtual machine cluster name, cluster description information, name prefixes of the management virtual machine and the working virtual machine, CPU, memory, data disk size and number of data disks, the selected host, the storage and network configuration of the host, and the virtual machine node network information, generate a parameter object in a set format, and provide the parameter object to the cluster deployment execution module.

[0033] Furthermore, the cluster deployment execution module includes:

[0034] A parameter parsing unit is used to obtain specific parameters selected by the user and parse out the user's configuration parameters for the virtual machine cluster;

[0035] The configuration parameter verification unit is used to verify the configuration parameters of the virtual machine cluster, and prompt the user to reconfigure when the verification fails, and proceed to the next step of deployment when the verification passes;

[0036] A working virtual machine deployment unit is used to start a working virtual machine deployment main task, generate a number of asynchronous subtasks from the working virtual machine deployment main task, and perform deployment on all working virtual machines at the same time, where each asynchronous subtask corresponds to the deployment of a working virtual machine;

[0037] An asynchronous subtask completion judgment unit is used to judge whether there is an asynchronous subtask deployment failure in the working virtual machine deployment, and when the asynchronous subtask deployment fails, exit the working virtual machine deployment main task;

[0038] A management virtual machine deployment unit, used to start the management virtual machine deployment main task and perform deployment on the management virtual machine;

[0039] The cluster operation modules include:

[0040] A virtual machine startup unit, used to start the working virtual machine and start the management virtual machine after all the working virtual machines are started;

[0041] A virtual machine startup success judgment unit is used to judge whether the management virtual machine is successfully started, and if the management virtual machine is not successfully started, wait for the management virtual machine to be successfully started;

[0042] The storage mounting unit is used to format and mount disks through the Linux command interface at the bottom of the virtual machine;

[0043] A host name setting unit, used to set host names for hosts of all virtual machines in the virtual machine cluster;

[0044] The configuration file generating unit is used to generate a configuration file according to a set format based on the IP, host name and host role of each virtual machine in the virtual machine cluster.

[0045] In a second aspect, the present invention provides a method for deploying a big data cluster on a cloud service platform, comprising the following steps:

[0046] S1. Generate a resource library module for virtual machine cluster deployment, store image libraries, storage pools, host clusters, and network resources, and create query interfaces for each resource in the resource library module. Each query interface forms a resource library interface module;

[0047] S2. When the user interface module obtains the user's cluster deployment startup, it outputs the corresponding query interface of the resource library interface module to the user to obtain the specific parameters of the resource library module selected by the user;

[0048] S3. According to the specific parameters of the resource library module selected by the user, select the corresponding resources from the resource library module, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine separately;

[0049] S4. Run the deployed virtual machine cluster and set up each virtual machine.

[0050] Furthermore, the specific steps of step S1 are as follows:

[0051] S11. Configure the environment for virtual machine cluster deployment and create a resource library module;

[0052] S12 generates a management virtual machine template, a working virtual machine template and an image library, and puts the management virtual machine template and the working virtual machine template into the image library, and then puts the image library into the resource library module;

[0053] S13. Generate a storage pool with the system disk and data disk required for configuring the virtual machine cluster, and then put the storage pool into the resource library module;

[0054] S14. Generate a host cluster from the physical hosts required to configure the virtual machine cluster, and then put the host cluster into the resource library module;

[0055] S15. Place the management network and data network required to configure the virtual machine cluster into the resource library module;

[0056] S16. Obtain the image library list, template list, storage pool list, host cluster list, network list and network information, and generate the image library list query interface, template list query interface, storage pool list query interface, host cluster list interface, network list query interface and network information query interface;

[0057] S17. Generate a big data deployment cluster interface and provide it to users to start virtual machine cluster deployment. Prepare the virtual machine cluster deployment environment in advance, add the resources required for virtual machine cluster deployment to the resource library module, and establish an interface to connect with the resource library module; network information includes IP, network and subnet mask.

[0058] Furthermore, the specific steps of step S2 are as follows:

[0059] S21. When the user interface module obtains the user deployment startup through the big data cluster deployment interface, it starts the virtual machine cluster deployment;

[0060] S22. The user interface module generates a prompt interface, obtains the virtual machine cluster name and cluster description information, and performs a duplication check on the virtual machine cluster name;

[0061] S23. The user interface module manages virtual machine configuration, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the management virtual machine template selected by the user through the template list query interface, and configures the virtual machine name prefix, CPU, memory, data disk size, and number of data disks;

[0062] S24. The user interface module configures the working virtual machine, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the working virtual machine template selected by the user through the template list query interface, and configures the name prefix, CPU, memory, data disk size, and number of data disks;

[0063] S25. The user interface module generates a host cluster selection interface, and selects a host under the host cluster through the host cluster list interface, the host includes hosts under the same host cluster and hosts under different host clusters;

[0064] S26. The user interface module generates a host configuration interface, obtains the host selected by the user, and configures the host's system disk, data disk, management network, and business network options;

[0065] S27. The user interface module generates a virtual machine information display interface, displays the virtual machine information, and obtains the user's configuration of the virtual machine node network information through the network information query interface;

[0066] S28. The user interface module assembles the acquired virtual machine cluster name, cluster description information, name prefixes of the management virtual machine and the working virtual machine, CPU, memory, data disk size and number of data disks, the selected host, the storage and network configuration of the host, and the network information of the virtual machine node, generates a parameter object in a set format, and provides the parameter object to the cluster deployment execution module. In step S26, the number of virtual machines deployed on a single host can be selected, and multiple hosts can deploy multiple virtual machines to make full use of host resources. The big data cluster deployment interface is an interface for users to start the virtual machine cluster deployment; the prompt interface can output a list of precautions for virtual machine cluster deployment and perform a repeatability check on the cluster name.

[0067] Further, the specific steps of step S3 are as follows:

[0068] S31. The cluster deployment execution module obtains the specific parameters selected by the user and parses the user's configuration parameters for the virtual machine cluster;

[0069] S32. The cluster deployment execution module verifies the configuration parameters of the virtual machine cluster, and when the verification fails, prompts the user to reconfigure, and when the verification passes, proceeds to the next step of deployment;

[0070] S33. The cluster deployment execution module starts the main task of deploying the working virtual machine, generates several asynchronous subtasks from the main task of deploying the working virtual machine, and executes the deployment on all working virtual machines at the same time, each asynchronous subtask corresponds to the deployment of a working virtual machine;

[0071] S34. The cluster deployment execution module determines whether there is an asynchronous subtask deployment failure in the work virtual machine deployment;

[0072] If yes, output an error message, exit the main task of working virtual machine deployment, and end;

[0073] If not, wait until all asynchronous subtasks are deployed and proceed to step S35;

[0074] S35. The cluster deployment execution module starts the main task of managing virtual machine deployment, executes deployment on the management virtual machine, and enters the next step after the management virtual machine deployment is completed. The cluster deployment execution module verifies the configuration parameters of the virtual machine cluster, such as the limit on the number of virtual machines per host, and verifies whether the management network and business network of each virtual machine are in the same network segment; multiple working virtual machines are deployed simultaneously through asynchronous subtasks.

[0075] Furthermore, the specific steps of step S4 are as follows:

[0076] S41. The cluster operation module starts the working virtual machine, and after all the working virtual machines are turned on, the management virtual machine is turned on;

[0077] S42. The cluster operation module determines whether the management virtual machine is successfully powered on;

[0078] If yes, go to step S43;

[0079] If not, wait until the management virtual machine is successfully powered on and then proceed to step S43;

[0080] S43. The cluster operation module formats and mounts the disk through the Linux command interface at the bottom of the virtual machine;

[0081] S44. The cluster operation module sets the host name for all virtual machines in the virtual machine cluster;

[0082] S45. The cluster operation module generates a configuration file according to the set format for the IP, host name and host role of each virtual machine in the virtual machine cluster. After the virtual machine cluster is deployed, you can log in to the address of the management virtual machine to enter the cluster management interface to view the cluster status, each virtual machine information and the status of each server.

[0083] The beneficial effects of the present invention are:

[0084] The system and method for deploying a big data cluster on a cloud service platform provided by the present invention realize the simultaneous deployment of multiple virtual machines in the big data cluster of the cloud service platform, and deploy the working virtual machines and the management virtual machines separately. At the same time, the virtual machines can be deployed on the hosts of the same cluster or on the hosts of different clusters. The present invention is simple to operate, has low technical requirements for operators, saves time for deploying the big data cluster of the cloud service platform, improves deployment efficiency, has good security, and provides a high user experience.

[0085] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect.

[0086] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0088] Figure 1 It is a schematic diagram of system embodiment 1 of the cloud service platform of the present invention for deploying a big data cluster.

[0089] Figure 2 It is a schematic diagram of system embodiment 2 of the cloud service platform of the present invention for deploying a big data cluster.

[0090] Figure 3 It is a flow chart of Embodiment 3 of the method for deploying a big data cluster on a cloud service platform of the present invention.

[0091] Figure 4 This is a schematic diagram of the process of the method for deploying a big data cluster on a cloud service platform of the present invention in Embodiment 4 Figure 2 .

[0092] Figure 5 This is a schematic diagram of the process of the method for deploying a big data cluster on a cloud service platform of the present invention in Embodiment 4 Figure 2 .

[0093] In the figure, 1-resource library module; 1.1-image library unit; 1.2-virtual machine template unit; 1.3-storage pool unit; 1.4-host cluster unit; 1.5-network storage unit; 2-resource library interface module; 2.1-image library list query interface; 2.2-template list query interface; 2.3-storage pool list query interface; 2.4-host cluster list interface; 2.5-network list interface; 2.6-big data cluster deployment interface; 3-user interface module; 3.1-user virtual machine cluster deployment startup unit; 3.2-user virtual machine cluster name interface unit; 3.3-user management virtual machine configuration interface unit; 3.4-user work 1.3-Working virtual machine interface configuration unit; 3.5-User host cluster interface unit; 3.6-User host configuration interface unit; 3.7-User virtual machine network information interface unit; 3.8-Parameter object generation unit; 4-Cluster deployment execution module; 4.1-Parameter parsing unit; 4.2-Configuration parameter verification unit; 4.3-Working virtual machine deployment unit; 4.4-Asynchronous subtask completion judgment unit; 4.5-Management virtual machine deployment unit; 5-Cluster operation module; 5.1-Virtual machine startup unit; 5.2-Virtual machine startup success judgment unit; 5.3-Storage mounting unit; 5.4-Host name setting unit; 5.5-Configuration file generation unit. DETAILED DESCRIPTION

[0094] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0095] Embodiment 1:

[0096] like Figure 1 As shown, the present invention provides a system for deploying a big data cluster on a cloud service platform, comprising:

[0097] Resource library module 1, used to store image libraries, virtual machine templates, storage pools, host clusters and network resources; virtual machine templates include management virtual machine templates and working virtual machine templates;

[0098] Resource library interface module 2, used to provide image library list query interface, template list query interface, storage pool list query interface, host cluster list query interface, network list query interface and big data cluster deployment interface;

[0099] User interface module 3, used to obtain the corresponding query interface of the resource library interface module to the user when the user's cluster deployment is started, so as to obtain the specific parameters of the resource library module selected by the user;

[0100] The cluster deployment execution module 4 is used to select the corresponding resources of the resource library module according to the specific parameters of the resource library module selected by the user, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine respectively;

[0101] The cluster operation module 5 is used to operate the deployed virtual machine cluster and set up each virtual machine.

[0102] Embodiment 2:

[0103] like Figure 2 As shown, the present invention provides a system for deploying a big data cluster on a cloud service platform, comprising:

[0104] Resource library module 1 is used to store image libraries, virtual machine templates, storage pools, host clusters, and network resources; virtual machine templates include management virtual machine templates and working virtual machine templates; resource library module 1 includes:

[0105] Image library unit 1.1, used to store the image library required for virtual machine cluster deployment;

[0106] A virtual machine template unit 1.2, used for storing virtual machine templates required for virtual machine cluster deployment;

[0107] Storage pool unit 1.3, used to store system disks and data disks required for virtual machine cluster deployment, and to generate storage pools for system disks and data disks;

[0108] A host cluster unit 1.4, used for storing a host cluster required for virtual machine cluster deployment;

[0109] Network storage unit 1.5, used to store the management network and business network required for virtual machine cluster deployment;

[0110] Resource library interface module 2 is used to provide image library list query interface, template list query interface, storage pool list query interface, host cluster list query interface, network list query interface and big data cluster deployment interface; resource library interface module 2 includes:

[0111] Image library list query interface 2.1 is used to obtain all image libraries required for virtual machine cluster deployment, generate an image library list, and provide it to users for image library configuration query;

[0112] Template list query interface 2.2, used to obtain the working virtual machine templates and management virtual machine templates required for virtual machine cluster deployment, generate a template list, and provide it to users for template configuration query;

[0113] Storage pool list query interface 2.3 is used to obtain the storage pool required for virtual machine cluster deployment, generate a storage pool list, and provide it to users for storage pool configuration query;

[0114] Host cluster list interface 2.4 is used to obtain the host clusters required for virtual machine cluster deployment, generate a host cluster list, and provide it to users for host cluster configuration query;

[0115] Network list interface 2.5 is used to obtain the network IP, gateway and mask required for the user's virtual machine cluster deployment, generate a network list, and provide it to the user for network information configuration query;

[0116] Big data cluster deployment interface 2.6, used to receive the user's operation to start virtual machine cluster deployment;

[0117] The user interface module 3 is used to obtain the corresponding query interface of the resource library interface module when the user's cluster deployment is started, so as to obtain the specific parameters of the resource library module selected by the user; the user interface module 3 includes:

[0118] The user virtual machine cluster deployment startup unit 3.1 is used to start the virtual machine cluster deployment when obtaining the user's deployment startup through the big data cluster deployment interface;

[0119] The user virtual machine cluster name interface unit 3.2 is used to generate a prompt interface, obtain the virtual machine cluster name and cluster description information, and perform duplication check on the virtual machine cluster name;

[0120] The user management virtual machine configuration interface unit 3.3 is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the management virtual machine template selected by the user through the template list query interface, and configure the virtual machine name prefix, CPU, memory, data disk size and number of data disks;

[0121] The user working virtual machine interface configuration unit 3.4 is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the working virtual machine template selected by the user through the template list query interface, and configure the name prefix, CPU, memory, data disk size and number of data disks;

[0122] A user host cluster interface unit 3.5, used to generate a host cluster selection interface, and select hosts under a host cluster through a host cluster list interface, wherein the hosts include hosts under the same host cluster and hosts under different host clusters;

[0123] The user host configuration interface unit 3.6 is used to generate a host configuration interface, obtain the host selected by the user, and configure the system disk, data disk, management network and business network options of the host;

[0124] User virtual machine network information interface unit 3.7, used to generate a virtual machine information display interface, display virtual machine information, and obtain user configuration of virtual machine node network information through the network information query interface;

[0125] The parameter object generation unit 3.8 is used to obtain the name of the virtual machine cluster, the cluster description information, the name prefix of the management virtual machine and the working virtual machine, the CPU, memory, data disk size and the number of data disks, the selected host, the storage and network configuration of the host, and the virtual machine node network information, generate a parameter object in a set format, and provide the parameter object to the cluster deployment execution module;

[0126] The cluster deployment execution module 4 is used to select the corresponding resources of the resource library module according to the specific parameters of the resource library module selected by the user, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine respectively; the cluster deployment execution module 4 includes:

[0127] The parameter parsing unit 4.1 is used to obtain the specific parameters selected by the user and parse out the configuration parameters of the virtual machine cluster by the user;

[0128] The configuration parameter verification unit 4.2 is used to verify the configuration parameters of the virtual machine cluster, and prompt the user to reconfigure when the verification fails, and proceed to the next step of deployment when the verification passes;

[0129] The working virtual machine deployment unit 4.3 is used to start the working virtual machine deployment main task, generate a number of asynchronous subtasks from the working virtual machine deployment main task, and perform deployment on all working virtual machines at the same time, each asynchronous subtask corresponds to the deployment of a working virtual machine;

[0130] The asynchronous subtask completion judgment unit 4.4 is used to judge whether there is any asynchronous subtask deployment failure in the working virtual machine deployment, and when the asynchronous subtask deployment fails, exit the working virtual machine deployment main task;

[0131] The management virtual machine deployment unit 4.5 is used to start the management virtual machine deployment main task and perform deployment on the management virtual machine;

[0132] The cluster operation module 5 is used to run the deployed virtual machine cluster and set up each virtual machine; the cluster operation module 5 includes:

[0133] A virtual machine startup unit 5.1 is used to start the working virtual machine and start the management virtual machine after all the working virtual machines are started;

[0134] The virtual machine startup success judgment unit 5.2 is used to judge whether the management virtual machine is successfully started, and if the management virtual machine is not successfully started, wait for the management virtual machine to be successfully started;

[0135] Storage mount unit 5.3 is used to format and mount disks through the Linux command interface at the bottom of the virtual machine;

[0136] A host name setting unit 5.4 is used to set host names for hosts of all virtual machines in the virtual machine cluster;

[0137] The configuration file generating unit 5.5 is used to generate a configuration file according to a set format according to the IP, host name and host role of each virtual machine in the virtual machine cluster.

[0138] Embodiment 3:

[0139] like Figure 3 As shown, the present invention provides a method for deploying a big data cluster on a cloud service platform, comprising the following steps:

[0140] S1. Generate a resource library module for virtual machine cluster deployment, store image libraries, storage pools, host clusters, and network resources, and create query interfaces for each resource in the resource library module. Each query interface forms a resource library interface module;

[0141] S2. When the user interface module obtains the user's cluster deployment startup, it outputs the corresponding query interface of the resource library interface module to the user to obtain the specific parameters of the resource library module selected by the user;

[0142] S3. According to the specific parameters of the resource library module selected by the user, select the corresponding resources from the resource library module, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine separately;

[0143] S4. Run the deployed virtual machine cluster and set up each virtual machine.

[0144] Embodiment 4:

[0145] like Figure 4 and Figure 5 As shown, the present invention provides a method for deploying a big data cluster on a cloud service platform, comprising the following steps:

[0146] S1. Generate a resource library module for virtual machine cluster deployment, store image libraries, storage pools, host clusters, and network resources, and create query interfaces for each resource in the resource library module. Each query interface forms a resource library interface module. The specific steps are as follows:

[0147] S11. Configure the environment for virtual machine cluster deployment and create a resource library module;

[0148] S12 generates a management virtual machine template, a working virtual machine template and an image library, and puts the management virtual machine template and the working virtual machine template into the image library, and then puts the image library into the resource library module;

[0149] S13. Generate a storage pool with the system disk and data disk required for configuring the virtual machine cluster, and then put the storage pool into the resource library module;

[0150] S14. Generate a host cluster from the physical hosts required to configure the virtual machine cluster, and then put the host cluster into the resource library module;

[0151] S15. Place the management network and data network required to configure the virtual machine cluster into the resource library module;

[0152] S16. Obtain the image library list, template list, storage pool list, host cluster list, network list and network information, and generate the image library list query interface, template list query interface, storage pool list query interface, host cluster list interface, network list query interface and network information query interface;

[0153] S17. Generate a big data deployment cluster interface and provide it to users to start virtual machine cluster deployment;

[0154] S2. When the user interface module obtains the user's cluster deployment startup, it outputs the corresponding query interface of the resource library interface module to the user to obtain the specific parameters of the resource library module selected by the user; the specific steps are as follows:

[0155] S21. When the user interface module obtains the user deployment startup through the big data cluster deployment interface, it starts the virtual machine cluster deployment;

[0156] S22. The user interface module generates a prompt interface, obtains the virtual machine cluster name and cluster description information, and performs a duplication check on the virtual machine cluster name;

[0157] S23. The user interface module manages virtual machine configuration, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the management virtual machine template selected by the user through the template list query interface, and configures the virtual machine name prefix, CPU, memory, data disk size, and number of data disks;

[0158] S24. The user interface module configures the working virtual machine, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the working virtual machine template selected by the user through the template list query interface, and configures the name prefix, CPU, memory, data disk size, and number of data disks;

[0159] S25. The user interface module generates a host cluster selection interface, and selects a host under the host cluster through the host cluster list interface, the host includes hosts under the same host cluster and hosts under different host clusters;

[0160] S26. The user interface module generates a host configuration interface, obtains the host selected by the user, and configures the host's system disk, data disk, management network, and business network options;

[0161] S27. The user interface module generates a virtual machine information display interface, displays the virtual machine information, and obtains the user's configuration of the virtual machine node network information through the network information query interface;

[0162] S28. The user interface module assembles the acquired virtual machine cluster name, cluster description information, name prefix of the management virtual machine and the working virtual machine, CPU, memory, data disk size and number of data disks, the selected host, the storage and network configuration of the host, and the network information of the virtual machine node, generates a parameter object in a set format, and provides the parameter object to the cluster deployment execution module;

[0163] S3. According to the specific parameters of the resource library module selected by the user, select the corresponding resources from the resource library module, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine respectively; the specific steps are as follows:

[0164] S31. The cluster deployment execution module obtains the specific parameters selected by the user and parses the user's configuration parameters for the virtual machine cluster;

[0165] S32. The cluster deployment execution module verifies the configuration parameters of the virtual machine cluster, and when the verification fails, prompts the user to reconfigure, and when the verification passes, proceeds to the next step of deployment;

[0166] S33. The cluster deployment execution module starts the main task of deploying the working virtual machine, generates several asynchronous subtasks from the main task of deploying the working virtual machine, and executes the deployment on all working virtual machines at the same time, each asynchronous subtask corresponds to the deployment of a working virtual machine;

[0167] S34. The cluster deployment execution module determines whether there is an asynchronous subtask deployment failure in the work virtual machine deployment;

[0168] If yes, output an error message, exit the main task of working virtual machine deployment, and end;

[0169] If not, wait until all asynchronous subtasks are deployed and proceed to step S35;

[0170] S35. The cluster deployment execution module starts the main task of managing virtual machine deployment, executes deployment on the management virtual machine, and proceeds to the next step after the management virtual machine deployment is completed;

[0171] S4. Run the deployed virtual machine cluster and set up each virtual machine. The specific steps are as follows:

[0172] S41. The cluster operation module starts the working virtual machine, and after all the working virtual machines are turned on, the management virtual machine is turned on;

[0173] S42. The cluster operation module determines whether the management virtual machine is successfully powered on;

[0174] If yes, go to step S43;

[0175] If not, wait until the management virtual machine is successfully powered on and then proceed to step S43;

[0176] S43. The cluster operation module formats and mounts the disk through the Linux command interface at the bottom of the virtual machine;

[0177] S44. The cluster operation module sets the host name for all virtual machines in the virtual machine cluster;

[0178] S45. The cluster operation module generates a configuration file according to a set format based on the IP, host name and host role of each virtual machine in the virtual machine cluster.

[0179] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A system for deploying a big data cluster on a cloud service platform, characterized in that: include: A resource library module (1) is used to store image libraries, virtual machine templates, storage pools, host clusters, and network resources; The virtual machine template includes a management virtual machine template and a working virtual machine template; A resource library interface module (2) is used to provide an image library list query interface, a template list query interface, a storage pool list query interface, a host cluster list query interface, a network list query interface, and a big data cluster deployment interface; The user interface module (3) is used to obtain the corresponding query interface of the resource library interface module to the user when the user's cluster deployment is started, so as to obtain the specific parameters of the resource library module selected by the user; The cluster deployment execution module (4) is used to select the corresponding resources of the resource library module according to the specific parameters of the resource library module selected by the user, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine respectively; The cluster operation module (5) is used to operate the deployed virtual machine cluster and set up each virtual machine; The cluster deployment execution module (4) includes: A parameter parsing unit (4.1) is used to obtain specific parameters selected by the user and parse out the user's configuration parameters for the virtual machine cluster; The configuration parameter verification unit (4.2) is used to verify the configuration parameters of the virtual machine cluster, and prompt the user to reconfigure when the verification fails, and proceed to the next step of deployment when the verification passes; A working virtual machine deployment unit (4.3) is used to start a main task of working virtual machine deployment, generate a number of asynchronous subtasks from the main task of working virtual machine deployment, and perform deployment on all working virtual machines at the same time, where each asynchronous subtask corresponds to the deployment of a working virtual machine; The asynchronous subtask completion judgment unit (4.4) is used to judge whether there is an asynchronous subtask deployment failure in the working virtual machine deployment, and when the asynchronous subtask deployment fails, exit the working virtual machine deployment main task; A management virtual machine deployment unit (4.5), used to start the management virtual machine deployment main task and perform deployment on the management virtual machine; The cluster operation module (5) includes: A virtual machine startup unit (5.1), used to start the working virtual machine, and start the management virtual machine after all the working virtual machines are started; A virtual machine startup success judgment unit (5.2) is used to judge whether the management virtual machine has been successfully started, and if the management virtual machine has not been successfully started, wait for the management virtual machine to be successfully started; Storage mount unit (5.3), used to format and mount disks through the Linux command interface at the bottom of the virtual machine; A host name setting unit (5.4), used for setting host names for hosts of all virtual machines in the virtual machine cluster; The configuration file generation unit (5.5) is used to generate a configuration file according to a set format according to the IP address, host name and host role of each virtual machine in the virtual machine cluster.

2. The system for deploying a big data cluster on a cloud service platform as claimed in claim 1, characterized in that: The resource library module (1) includes: Image library unit (1.1), used to store the image library required for virtual machine cluster deployment; A virtual machine template unit (1.2), used to store virtual machine templates required for virtual machine cluster deployment; Storage pool unit (1.3), used to store system disks and data disks required for virtual machine cluster deployment, and to generate storage pools for system disks and data disks; A host cluster unit (1.4), used to store host clusters required for virtual machine cluster deployment; The network storage unit (1.5) is used to store the management network and business network required for the deployment of the virtual machine cluster.

3. The system for deploying a big data cluster on a cloud service platform as claimed in claim 1, characterized in that: The resource library interface module (2) includes: Image library list query interface (2.1), used to obtain all image libraries required for virtual machine cluster deployment, generate an image library list, and provide it to users for image library configuration query; Template list query interface (2.2), used to obtain the working virtual machine templates and management virtual machine templates required for virtual machine cluster deployment, generate a template list, and provide it to users for template configuration query; Storage pool list query interface (2.3), used to obtain the storage pool required for virtual machine cluster deployment, generate a storage pool list, and provide it to users for storage pool configuration query; Host cluster list interface (2.4), used to obtain the host clusters required for virtual machine cluster deployment, generate a host cluster list, and provide it to users for host cluster configuration query; The network list interface (2.5) is used to obtain the network IP, gateway, and mask required for the user's virtual machine cluster deployment, generate a network list, and provide it to the user for network information configuration query; The big data cluster deployment interface (2.6) is used to receive the user's operation to start the virtual machine cluster deployment.

4. The system for deploying a big data cluster on a cloud service platform as claimed in claim 1, characterized in that: The user interface module (3) includes: A user virtual machine cluster deployment startup unit (3.1), used to start the virtual machine cluster deployment when obtaining the user's deployment startup through the big data cluster deployment interface; A user virtual machine cluster name interface unit (3.2), used to generate a prompt interface, obtain the virtual machine cluster name and cluster description information, and perform duplication check on the virtual machine cluster name; The user management virtual machine configuration interface unit (3.3) is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the management virtual machine template selected by the user through the template list query interface, and configure the virtual machine name prefix, CPU, memory, data disk size and number of data disks; A user working virtual machine interface configuration unit (3.4) is used to generate an image library selection interface, obtain the image library selected by the user through the image library list query interface, obtain the working virtual machine template selected by the user through the template list query interface, and configure the name prefix, CPU, memory, data disk size and number of data disks; A user host cluster interface unit (3.5), used to generate a host cluster selection interface, and select hosts under a host cluster through a host cluster list interface, wherein the hosts include hosts under the same host cluster and hosts under different host clusters; The user host configuration interface unit (3.6) is used to generate a host configuration interface, obtain the host selected by the user, and configure the system disk, data disk, management network and business network options of the host; A user virtual machine network information interface unit (3.7), used to generate a virtual machine information display interface, display virtual machine information, and obtain user configuration of virtual machine node network information through a network information query interface; The parameter object generation unit (3.8) is used to obtain the name of the virtual machine cluster, cluster description information, name prefixes of the management virtual machine and the working virtual machine, CPU, memory, data disk size and number of data disks, the selected host, storage and network configuration of the host, and virtual machine node network information, generate a parameter object in a set format, and provide the parameter object to the cluster deployment execution module.

5. A method for deploying a big data cluster on a cloud service platform, characterized in that: The steps include: S1. Generate a resource library module for virtual machine cluster deployment, store image libraries, storage pools, host clusters, and network resources, and create query interfaces for each resource in the resource library module. Each query interface forms a resource library interface module; S2. When the user interface module obtains the user's cluster deployment startup, it outputs the corresponding query interface of the resource library interface module to the user to obtain the specific parameters of the resource library module selected by the user; S3. According to the specific parameters of the resource library module selected by the user, select the corresponding resources from the resource library module, execute the virtual machine cluster deployment, and deploy the management virtual machine and the working virtual machine separately; S4. Run the deployed virtual machine cluster and set up each virtual machine; The specific steps of step S3 are as follows: S31. The cluster deployment execution module obtains the specific parameters selected by the user and parses the user's configuration parameters for the virtual machine cluster; S32. The cluster deployment execution module verifies the configuration parameters of the virtual machine cluster, and when the verification fails, prompts the user to reconfigure, and when the verification passes, proceeds to the next step of deployment; S33. The cluster deployment execution module starts the main task of deploying the working virtual machine, generates several asynchronous subtasks from the main task of deploying the working virtual machine, and executes the deployment on all working virtual machines at the same time, each asynchronous subtask corresponds to the deployment of a working virtual machine; S34. The cluster deployment execution module determines whether there is an asynchronous subtask deployment failure in the work virtual machine deployment; If yes, output an error message, exit the main task of working virtual machine deployment, and end; If not, wait until all asynchronous subtasks are deployed and proceed to step S35; S35. The cluster deployment execution module starts the main task of managing virtual machine deployment, executes deployment on the management virtual machine, and proceeds to the next step after the management virtual machine deployment is completed; The specific steps of step S4 are as follows: S41. The cluster operation module starts the working virtual machine, and after all the working virtual machines are turned on, the management virtual machine is turned on; S42. The cluster operation module determines whether the management virtual machine is successfully powered on; If yes, go to step S43; If not, wait until the management virtual machine is successfully powered on and then proceed to step S43; S43. The cluster operation module formats and mounts the disk through the Linux command interface at the bottom of the virtual machine; S44. The cluster operation module sets the host name for all virtual machines in the virtual machine cluster; S45. The cluster operation module generates a configuration file according to a set format based on the IP, host name and host role of each virtual machine in the virtual machine cluster.

6. The method for deploying a big data cluster on a cloud service platform as claimed in claim 5, characterized in that: The specific steps of step S1 are as follows: S11. Configure the environment for virtual machine cluster deployment and create a resource library module; S12 generates a management virtual machine template, a working virtual machine template and an image library, and puts the management virtual machine template and the working virtual machine template into the image library, and then puts the image library into the resource library module; S13. Generate a storage pool with the system disk and data disk required for configuring the virtual machine cluster, and then put the storage pool into the resource library module; S14. Generate a host cluster from the physical hosts required to configure the virtual machine cluster, and then put the host cluster into the resource library module; S15. Place the management network and data network required to configure the virtual machine cluster into the resource library module; S16. Obtain the image library list, template list, storage pool list, host cluster list, network list and network information, and generate the image library list query interface, template list query interface, storage pool list query interface, host cluster list interface, network list query interface and network information query interface; S17. Generate a big data deployment cluster interface and provide it to users to start virtual machine cluster deployment.

7. The method for deploying a big data cluster on a cloud service platform as claimed in claim 5, characterized in that: The specific steps of step S2 are as follows: S21. When the user interface module obtains the user deployment startup through the big data cluster deployment interface, it starts the virtual machine cluster deployment; S22. The user interface module generates a prompt interface, obtains the virtual machine cluster name and cluster description information, and performs a duplication check on the virtual machine cluster name; S23. The user interface module manages virtual machine configuration, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the management virtual machine template selected by the user through the template list query interface, and configures the virtual machine name prefix, CPU, memory, data disk size, and number of data disks; S24. The user interface module configures the working virtual machine, generates an image library selection interface, obtains the image library selected by the user through the image library list query interface, obtains the working virtual machine template selected by the user through the template list query interface, and configures the name prefix, CPU, memory, data disk size, and number of data disks; S25. The user interface module generates a host cluster selection interface, and selects a host under the host cluster through the host cluster list interface, the host includes hosts under the same host cluster and hosts under different host clusters; S26. The user interface module generates a host configuration interface, obtains the host selected by the user, and configures the host's system disk, data disk, management network, and business network options; S27. The user interface module generates a virtual machine information display interface, displays the virtual machine information, and obtains the user's configuration of the virtual machine node network information through the network information query interface; S28. The user interface module assembles the acquired virtual machine cluster name, cluster description information, name prefix of the management virtual machine and the working virtual machine, CPU, memory, data disk size and number of data disks, the selected host, the storage and network configuration of the host, and the network information of the virtual machine node to generate a parameter object in a set format, and provides the parameter object to the cluster deployment execution module.

Images