Method, device and cloud computing system for controlling cluster resources
By determining the binding relationship between the resources to be expanded and the application and generating the corresponding data packets, the problem of poor applicability of resource expansion in the existing technology is solved, and the applicability of resource expansion for different services is realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
- Filing Date
- 2019-12-05
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies are not applicable to resource expansion for different business operations, resulting in poor applicability.
By determining the binding relationship between the resources to be scaled and the application, the initialized resources to be scaled are added to the resource pool of the corresponding application, and an execution data package is generated according to the deployment type and deployed on the corresponding resources to be scaled for execution, including the processing of physical machines and container images.
It enables resource expansion applicable to different business operations, thus improving the applicability of resource expansion.
Smart Images
Figure CN110968427B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a method for controlling cluster resources, a device for controlling cluster resources, a cloud computing system, and a computer-readable storage medium. Background Technology
[0002] As cloud systems consume resources during use, resource control (such as scaling up and down) of various products in the cluster has become a regular operation that operations and maintenance personnel need to perform.
[0003] In related technologies, expansion methods are developed for cluster applications of a certain business in a specific scenario. Summary of the Invention
[0004] The inventors of this disclosure have discovered the following problem in the aforementioned related technologies: they cannot be applied to resource expansion for different business applications, resulting in poor applicability.
[0005] In view of this, this disclosure proposes a cluster resource control technology scheme that can improve the applicability of resource expansion.
[0006] According to some embodiments of this disclosure, a method for controlling cluster resources is provided, including: when the resource to be controlled is a resource to be expanded, determining the binding relationship between the resource to be expanded and the corresponding application; adding the initialized resource to be expanded to the resource pool of the corresponding application according to the binding relationship; generating an execution data packet for the application to be processed according to the deployment type of the application to be processed; and deploying the execution data packet on the corresponding resource to be expanded in the resource pool of the application to be processed for execution.
[0007] In some embodiments, adding the initialized resource to be expanded to the resource pool of the corresponding application includes: passing the relevant information of the resource to be expanded to the pre-script of the corresponding application; and executing the pre-script to complete the initialization of the resource to be expanded.
[0008] In some embodiments, generating the application to be processed execution data package includes: when the deployment type is package deployment, the resource to be scaled is a physical machine, generating the application package to be processed as the execution data package; when the deployment type is image deployment, the resource to be scaled is a container image, generating the application package to be processed, and generating the execution data package based on the application package and the running image of the application to be processed.
[0009] In some embodiments, deploying the data packet to be executed on the corresponding resource to be expanded in the resource pool of the application to be processed includes: when the resource to be expanded is a physical machine, sending the data packet to be executed to the physical machine for execution; when the resource to be expanded is a container image, sending the data packet to be executed to an idle physical machine in the corresponding resource pool for execution.
[0010] In some embodiments, deploying the data packet to be executed on the corresponding resource to be expanded in the resource pool of the application to be processed includes: obtaining relevant information about the resource to be expanded; and sending the relevant information about the resource to be expanded to a third-party program of the application to be processed through a deployment interface configured for the application to be processed, so that the third-party program can deploy the data packet to be executed on the resource to be expanded according to its own deployment method.
[0011] In some embodiments, the method further includes: executing a post-script for the application to be processed, the post-script being used for at least one of the following processes: returning the scaling result to the cluster management node; creating corresponding volumes for the corresponding scaling resources; and cleaning up the garbage generated during the scaling process.
[0012] In some embodiments, the method further includes: establishing SSH (Secure Shell) connections with each resource to be expanded, for executing relevant scripts of the corresponding application, wherein only one SSH connection can be established with each resource to be expanded at the same time.
[0013] In some embodiments, the method further includes: retaining the corresponding SSH connection for a preset time period after the execution of the data packet to be executed by the corresponding application is completed.
[0014] In some embodiments, the method further includes: if the resource to be controlled is a resource to be scaled down, determining whether there is important data in the resource to be scaled down and whether there are services that depend on the resource to be scaled down; if there is no important data and no services that depend on the resource to be scaled down, removing the resource to be scaled down from the cluster.
[0015] In some embodiments, the relevant information of the resource to be scaled up is passed through the configured scaling-down interface so as to remove the resource to be scaled down from the cluster.
[0016] In some embodiments, the method further includes: obtaining the scaling-down result by polling the configured query interface.
[0017] In some embodiments, after removing the resource to be scaled up from the cluster, the method further includes: if the resource to be scaled up is a physical machine, adding the resource to a resource pool; if the resource to be scaled up is a container image, destroying the container image and adding the resource to a resource pool.
[0018] In some embodiments, the method further includes: when the resource to be scaled up is a physical machine, executing a post-processing script for scaling up, the post-processing script being used to initiate an installation process to reinstall the operating system on the resource to be scaled up.
[0019] According to some other embodiments of this disclosure, a cluster resource control device is provided, comprising: a determining unit, configured to determine the binding relationship between the resource to be expanded and the corresponding application when the resource to be controlled is a resource to be expanded; an adding unit, configured to add the initialized resource to be expanded to the resource pool of the corresponding application according to the binding relationship; a generating unit, configured to generate an execution data packet of the application to be processed according to the deployment type of the application to be processed; and an execution unit, configured to deploy the execution data packet on the corresponding resource to be expanded in the resource pool of the application to be processed for execution.
[0020] In some embodiments, the adding unit passes the relevant information of the resource to be expanded to the pre-script of the corresponding application, executes the pre-script, and completes the initialization of the resource to be expanded.
[0021] In some embodiments, when the deployment type is package deployment, the resource to be expanded is a physical machine, and the generation unit generates a program package of the application to be processed as an executable data package; when the deployment type is image deployment, the resource to be expanded is a container image, the generation unit generates a program package of the application to be processed, and generates an executable data package based on the program package and the running image of the application to be processed.
[0022] In some embodiments, when the resource to be expanded is a physical machine, the execution unit sends the data packet to be executed to the physical machine for execution; when the resource to be expanded is a container image, the execution unit sends the data packet to be executed to an idle physical machine in the corresponding resource pool for execution.
[0023] In some embodiments, the execution unit obtains the relevant information of the resource to be expanded, and sends the relevant information of the resource to be expanded to the third-party program of the application to be processed through the deployment interface configured for the application to be processed, so that the third-party program can deploy the data packet to be executed on the resource to be expanded for execution according to its own deployment method.
[0024] In some embodiments, the execution unit executes a post-script of the application to be processed, the post-script being used for at least one of the following processes: returning the scaling result to the cluster management node; creating the corresponding volume for the corresponding scaling resources; and cleaning up the garbage generated by the scaling process.
[0025] In some embodiments, the apparatus further includes an establishment unit for establishing SSH connections with each resource to be expanded, for executing relevant scripts of the corresponding application, and for establishing only one SSH connection with each resource to be expanded at any given time.
[0026] In some embodiments, the establishment unit retains the corresponding SSH connection for a preset time period after the execution of the data packet to be executed by the corresponding application is completed.
[0027] In some embodiments, the apparatus further includes a scaling-down unit, configured to determine whether the resource to be scaled down contains important data and whether there are services that depend on the resource to be scaled down when the resource to be controlled is a resource to be scaled down; if the resource to be scaled down does not contain important data and there are no services that depend on the resource to be scaled down, the scaling-down unit removes the resource to be scaled down from the cluster.
[0028] In some embodiments, the scaling-down unit transmits relevant information about the resource to be scaled down through a configured scaling-down interface, so as to remove the resource to be scaled down from the cluster.
[0029] In some embodiments, the scaling-down unit obtains the scaling-down results by polling the configured query interface.
[0030] In some embodiments, after the shrinking unit removes the resource to be shrunk from the cluster, if the resource to be shrunk is a physical machine, the adding unit adds the resource to the resource pool; if the resource to be shrunk is a container image, the shrinking unit destroys the container image, and the adding unit adds the resource to the resource pool.
[0031] In some embodiments, when the resource to be scaled up is a physical machine, the execution unit executes a post-script for the scaling-up process. The post-script is used to initiate the installation process to reinstall the operating system on the resource to be scaled up.
[0032] According to further embodiments of the present disclosure, a cluster resource control device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the cluster resource control method of any of the above embodiments based on instructions stored in the memory device.
[0033] According to further embodiments of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the cluster resource control method of any of the above embodiments.
[0034] According to further embodiments of this disclosure, a cloud computing system is provided, including: a cluster resource control device for executing the cluster resource control method in any of the above embodiments.
[0035] In the above embodiments, based on the binding relationship between the resources to be expanded and the applications, the application's data packets are deployed and executed on the corresponding resources to be expanded. This allows for resource expansion applicable to applications with different business needs, thereby improving the applicability of resource expansion. Attached Figure Description
[0036] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.
[0037] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description:
[0038] Figure 1 Flowcharts illustrating some embodiments of the cluster resource control method of this disclosure;
[0039] Figure 2 Flowcharts illustrating other embodiments of the cluster resource control method of this disclosure;
[0040] Figure 3 Schematic diagrams illustrating some embodiments of the cluster resource control apparatus of this disclosure;
[0041] Figure 4 Flowcharts illustrating further embodiments of the cluster resource control method of this disclosure;
[0042] Figure 5 Schematic diagrams illustrating other embodiments of the cluster resource control apparatus of this disclosure;
[0043] Figure 6 Schematic diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure;
[0044] Figure 7 Schematic diagrams illustrating some embodiments of the cluster resource control method of this disclosure;
[0045] Figure 8 Schematic diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure;
[0046] Figure 9 Schematic diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure;
[0047] Figure 10 Block diagrams illustrating some embodiments of the cluster resource control apparatus of this disclosure;
[0048] Figure 11 Block diagrams illustrating other embodiments of the cluster resource control apparatus of this disclosure;
[0049] Figure 12 Block diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure;
[0050] Figure 13 Block diagrams illustrating some embodiments of the control system for cluster resources of this disclosure are shown. Detailed Implementation
[0051] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.
[0052] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.
[0053] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.
[0054] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.
[0055] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.
[0056] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0057] As mentioned earlier, when faced with numerous products integrated into a cloud computing system (such as a private cloud system), operations and maintenance personnel not only need to understand and master the scaling up and down processes of various products, but also need to be clear about the operational differences between them. This undoubtedly increases the later-stage operation and maintenance costs of the private cloud. More importantly, manual operation is prone to errors, leading to scaling up or down incidents, which in turn reduces the success rate of scaling up and down.
[0058] To address the aforementioned technical issues, this disclosure integrates multiple most frequently used products into a single expansion and contraction process. This allows maintenance personnel to easily expand and contract the resource machines of these products with simple operations. For example, this can be achieved through the following embodiment.
[0059] Figure 1 Flowcharts illustrating some embodiments of the cluster resource control method of this disclosure are shown.
[0060] like Figure 1 As shown, the method includes: step S11, determining the binding relationship; step S12, adding resources to be expanded; step S13, generating a data packet to be executed; and step S14, deploying the data packet to be executed.
[0061] In step S11, if the resource to be controlled is a resource to be expanded, the binding relationship between the resource to be expanded and the corresponding application is determined. For example, the resource to be expanded can be a physical machine or a container image.
[0062] In step S12, the initialized resources to be expanded are added to the resource pool of the corresponding application according to the binding relationship.
[0063] In some embodiments, the relevant information of the resource to be expanded is passed to the pre-script of the corresponding application; the pre-script is executed to complete the initialization of the resource to be expanded.
[0064] In step S13, an execution data packet for the application to be processed is generated according to the deployment type of the application to be processed.
[0065] In some embodiments, when the deployment type is package deployment, a package of the application to be processed is generated as an executable data package. In this case, the resource to be scaled up is a physical machine.
[0066] In some embodiments, when the deployment type is image deployment, a package of the application to be processed is generated, and an executable data package is generated based on the package and the running image of the application to be processed. In this case, the resource to be scaled up is a container image.
[0067] In step S14, the data packet to be executed is deployed on the corresponding resources to be expanded in the resource pool of the application to be processed.
[0068] In some embodiments, when the resource to be expanded is a physical machine, the data packet to be executed is sent to the physical machine for execution; when the resource to be expanded is a container image, the data packet to be executed is sent to an idle physical machine in the corresponding resource pool (standby pool) for execution. For example, the standby pool is a pool of spare physical machine resources in a cloud computing system.
[0069] In some embodiments, relevant information about the resource to be expanded is obtained; the relevant information about the resource to be expanded is sent to the third-party program of the application to be processed through the deployment interface configured for the application to be processed; the third-party program deploys the data packet to be executed on the resource to be expanded for execution according to its own deployment method.
[0070] In some embodiments, a post-script for the application to be processed is executed. The post-script is used for at least one of the following processes: returning the scaling result to the cluster management node; creating the corresponding volume for the corresponding scaling resources; and cleaning up the garbage generated by the scaling process.
[0071] In some embodiments, SSH connections are established with each resource to be expanded to execute relevant scripts of the corresponding application. Only one SSH connection can be established with each resource to be expanded at a time.
[0072] In some embodiments, after the execution of the application's pending data packet is completed, the corresponding SSH connection is retained for a preset time period so that it can be reused the next time the application's pending data packet is executed.
[0073] Figure 2 Flowcharts illustrating some other embodiments of the cluster resource control method of this disclosure are shown.
[0074] like Figure 2 As shown, the method further includes: step S21, determining important data and dependent services; and S22, removing resources to be scaled down.
[0075] In step S21, if the resource to be controlled is a resource to be scaled down, it is determined whether there is important data in the resource to be scaled down, and whether there are services that depend on the resource to be scaled down.
[0076] In step S22, if there is no important data and no service that depends on the resource to be scaled down, the resource to be scaled down is removed from the cluster.
[0077] In some embodiments, the relevant information of the resource to be scaled up can be transmitted in the system through the configured scaling-down interface so as to remove the resource to be scaled down from the cluster; and the scaling-down result can be obtained by polling the configured query interface.
[0078] In some embodiments, if the resource to be scaled up is a physical machine, the resource to be scaled up is added to a resource pool (standby pool); if the resource to be scaled up is a container image, the container image is destroyed and the resource to be scaled up is added to the resource pool.
[0079] In some embodiments, when the resource to be scaled up is a physical machine, a post-processing script for the scaling-up process is executed. This post-processing script initiates the installation process to reinstall the operating system on the resource to be scaled up. For example, this reinstallation can be achieved through PXE (PrebooteXecution Environment).
[0080] In the above embodiments, based on the binding relationship between the resources to be expanded and the applications, the application's data packets are deployed and executed on the corresponding resources to be expanded. This allows for resource expansion applicable to applications with different business needs, thereby improving the applicability of resource expansion.
[0081] Figure 3 Schematic diagrams illustrating some embodiments of the cluster resource control apparatus of this disclosure are shown.
[0082] like Figure 3As shown, users can interact with the backend scaling (or scaling down) controller (i.e., the control device) through a front-end web page. For example, the front-end web page can be implemented using an Nginx (engine x) server; the control device can be implemented using the Golang programming language.
[0083] The internal structure of the control unit can be a unified resource control system designed for complex private cloud clusters, supporting concurrent scaling up and down of multiple components. Scripts from various business lines can connect to the control unit.
[0084] For example, the control device may include multiple modules such as a multi-component scaling controller, a heterogeneous component resource management module, a concurrent scaling actuator, a unified component construction system, a multi-component deployment system, and a component scaling-down and recycling system.
[0085] In some embodiments, a multi-component scaling controller (including a determination unit, an addition unit, etc.) is responsible for controlling the overall scaling process. For example, the multi-component scaling controller can coordinate the work of other modules; control the invocation of each module used in the scaling process; and provide necessary data parameters.
[0086] In some embodiments, the heterogeneous component resource management module is responsible for managing all metadata required during the expansion and contraction process. For example, metadata includes server (physical machine resource) management IP (Internet Protocol), server specifications, expansion and contraction application data, etc.
[0087] For example, the heterogeneous component resource management module can use the relational database MySQL (My Structured Query Language) to persistently store data.
[0088] For example, the heterogeneous component resource management module can be deployed independently and provide an OpenAPI (Open Application Programming Interface) call interface to the outside world.
[0089] In some embodiments, the concurrent scaling executor (which may include an execution unit, a creation unit, etc.) is responsible for executing the scripts that each resource server needs to run at different stages.
[0090] For example, the underlying layer of a concurrent scaling executor can be a connection pool based on the SSH protocol. The concurrent scaling executor can be implemented using the Golang programming language to ensure high concurrency. The concurrent scaling executor can establish an SSH connection between the control device and the resource to be scaled.
[0091] For example, concurrent scaling executors can execute custom scaling-up and scaling-down processes. This is particularly relevant for IaaS (Infrastructure as a Service) applications.
[0092] For example, concurrent scaling executors can execute standardized scaling processes. Custom scaling processes can be executed for applications related to big data, cloud storage, and services other than IaaS.
[0093] For example, concurrent scaling executors can trigger unified component building systems, multi-component deployment systems, and component scaling-down and recycling systems. These systems handle the scaling-up and scaling-down processes of the cloud computing system cluster.
[0094] In some embodiments, the unified component build system (which may include a generation unit) is responsible for compiling the source code deployed on each resource server; it prepares different compilation environments for different resource servers based on Docker containers. The unified component build system can isolate the compilation environment, ensuring that the dependent services required for compilation do not interfere with each other, thereby guaranteeing the smooth progress of the compilation process.
[0095] In some embodiments, the multi-component deployment system (which may include an execution unit) is responsible for deploying the compiled program to a specified physical server or container.
[0096] For example, a multi-component deployment system can include two types: package deployment and image deployment. Package deployment will put the compiled program online to a specified physical machine; image deployment will deploy and start containers based on the runtime image packaged during compilation and according to the required resources (including the number of CPU cores, memory size, hard disk size, etc. required for running).
[0097] In some embodiments, the component scaling-down and recycling system (which may include scaling-down units and adding units) is responsible for recycling the scaled-down resources.
[0098] For example, recycled container resources are put back into the resource pool, while recycled physical machines are put into the standby pool. The component scaling-down and recycling system destroys container resources and reinstalls and formats the data disks of recycled physical machines.
[0099] Figure 4 Flowcharts illustrating further embodiments of the cluster resource control method of this disclosure are shown.
[0100] like Figure 4As shown, the method can include a scaling up process (left side of the diagram) and a scaling down process (right side of the diagram). The scaling up process can include a standardized scaling up process and a custom scaling up process; the scaling down process can include a standardized scaling down process and a custom scaling down process.
[0101] For example, the scaling up and scaling down of various resource nodes related to IaaS and cloud database products (such as MySQL resource nodes, large-capacity storage resource nodes, etc.) are all custom scaling up and scaling down processes; while the scaling up and scaling down of various resource nodes related to cloud storage and data cloud (such as ds2-datanode resource nodes, big data datanode resource nodes, etc.) are all standardized scaling up and scaling down processes.
[0102] The scaling up and scaling down processes of the aforementioned products all require three common steps: allocation of usage (binding the resources to be scaled up or down to the corresponding product application), execution of pre-scripts, and execution of post-scripts. These common steps can be extracted to establish a highly applicable resource control method.
[0103] In some embodiments, the standardized expansion process mainly includes: mounting the standby machine (resource) to be expanded to the standby machine pool of the specified product application; compiling and building the application's program package or runtime image; deploying the program package or image to the servers in the standby machine pool; and polling the deployment results. For example, the expansion process of a data cloud product mainly targets big data datanode resource nodes, and this standardized expansion process may include the following steps.
[0104] In step 1, the purpose of the expansion host is assigned. For example, a tag is assigned to the datanode resource node of the server (resource) to be expanded, thereby binding the resource to be expanded with the application.
[0105] In step 2, a pre-script is executed. For example, a multi-component scaling controller can call a pre-configured script (pre-configured script) in the data cloud's FTP (File Transfer Protocol) directory on the control machine, passing the list of IPs of the servers to be scaled as command-line arguments to the pre-configured script. For example, the control machine can be a computer in a cluster running the control method disclosed herein.
[0106] By executing the expansion script, the initialization of the server to be expanded can be completed. The pre-configured script is mainly responsible for initializing the physical machine to be expanded, such as installing some basic software packages.
[0107] In step 3, the expansion host is attached. For example, the physical machine to be expanded is attached to the standby pool of the corresponding product line in the product service tree to facilitate unified resource allocation. For example, the resources to be expanded can be added to the resource pool of the product line to which the bound application belongs, based on the relevant information of the product to be deployed within the cloud computing system.
[0108] In some embodiments, information about products to be deployed within a cloud computing system can be stored in a tree structure. For example, it can be stored in a five-level tree structure of department, product line, product, system, and application.
[0109] In step 4, the data package is compiled and created. For example, depending on the application's deployment type, the application's package is compiled and built. If the big data datanode resource node uses package deployment, the corresponding package is compiled as the processing data package; for image deployment, the package and the running image are packaged together to generate the data package to be processed. The compiled and built package is then transferred to the bound machine to be expanded in the standby pool, and the startup script is executed.
[0110] In step 5, the scaling results are polled. For example, the deployment status can be polled periodically, and the deployment results can be recorded in the deployment unit of the Information Management Database (CMDB). The web frontend can query the deployment results through the API interface and present them to the user.
[0111] In step 6, a post-processing script is executed to handle post-expansion transactions. For example, the post-processing script can inform the management node in the cluster about the expanded resource nodes, thus completing the cluster expansion.
[0112] In some embodiments, the cloud storage expansion process mainly involves expanding the ds2-datanode resource nodes, and its steps are similar to those of the data cloud described above. However, the post-script of the cloud storage expansion process can perform the process of creating volumes.
[0113] In some embodiments, a standardized scaling-down process may include business data inspection, dependent service inspection, removal from the cluster, resource reclamation, polling results, and execution of post-scripts. For example, the standardized scaling-down process may include the following steps.
[0114] In step 1, a business data check is performed on the resource to be scaled down to determine if there is any important data in the resource. If important data is found, the resource to be scaled down will not support the scaling-down operation.
[0115] In step 2, a dependency service check is performed on the resource to be scaled down to determine if any services depend on it for operation. If dependent services exist, the resource to be scaled down does not support the scaling-down operation.
[0116] In step 3, the cluster management node is notified to move the resources to be scaled down out of the cluster.
[0117] In step 4, the component scaling down and recycling system is invoked to reclaim the scaling down resources.
[0118] In step 5, the scaling-down results are polled.
[0119] In step 6, a post-script is executed to handle post-scaledown transactions. For example, it can trigger a PXE installation operation to reclaim resources.
[0120] In some embodiments, the customized expansion process may include: mounting the standby machine to be expanded to the designated product application standby machine pool; having a third party provide its own expansion program (third-party program), which must provide two standardized expansion interfaces, namely, a trigger expansion interface (deployment interface) and a query expansion result interface (query interface); the multi-component scaling controller calls the trigger expansion interface to pass the standby machine information to be expanded in the form of parameters; and polling the expansion results through the query expansion result interface.
[0121] For example, a custom scaling process for applications of IaaS products can be implemented through the following steps.
[0122] In step 1, a purpose is assigned to the host to be expanded. For example, the server to be expanded is assigned a tag for a resource node of an IaaS product application.
[0123] In step 2, the pre-installation script is executed. For example, the multi-component scaling controller calls the data cloud pre-installation script in the management machine's FTP directory and passes the list of IP addresses of the machines to be scaled as command-line arguments to the pre-installation script; the scaling script is then executed to complete the machine initialization. The pre-installation script is mainly responsible for initializing the physical machines to be scaled, for example, by installing some basic software packages.
[0124] In step 3, the machine is attached to the expansion host. For example, the physical machine to be expanded is attached to the backup machine pool of the corresponding product line in the product's service tree, so that third-party programs can access it uniformly.
[0125] In step 4, the expansion host is passed. For example, the multi-component scaling controller triggers the IaaS deployment interface to obtain relevant information about the server to be expanded from the information management CMDB; the relevant information is passed to the IaaS product in the form of parameters; the IaaS product deployment program obtains the server to be expanded from the standby pool; the IaaS deployment service is deployed according to its own business.
[0126] In step 5, the polling interface is invoked. For example, the multi-component scaling controller periodically polls the IaaS scaling results; the deployment results are recorded in the deployment unit of the information management module CMDB; the web frontend can query the deployment results through the API interface and present them to the user.
[0127] In step 6, execute the post-script. This step can be left blank if the product does not have a post-script.
[0128] In some embodiments, a custom scaling-down process may include business data checks, dependency service checks, calling a custom scaling-down interface, polling for scaling-down results, reclaiming resources to a resource pool or standby pool, and executing post-processing scripts. For example, a custom scaling-down process can be implemented through the following steps.
[0129] In step 1, a business data check is performed on the resource to be scaled down to determine if there is any important data in the resource. If important data is found, the resource to be scaled down will not support the scaling-down operation.
[0130] In step 2, a dependency service check is performed on the resource to be scaled down to determine if any services depend on it for operation. If dependent services exist, the resource to be scaled down does not support the scaling-down operation.
[0131] In step 3, a custom scaling-down interface is called to pass relevant information about the resource to be scaled down.
[0132] In step 4, the custom scaling-down result interface (query interface) is polled to obtain the scaling-down results.
[0133] In step 5, the recovered resources are placed into the resource pool or the standby pool.
[0134] Figure 5 Schematic diagrams illustrating other embodiments of the cluster resource control apparatus of this disclosure are shown.
[0135] like Figure 5 As shown, the heterogeneous component resource management module mainly consists of a service layer and a data layer, and provides metadata to other service modules and the web front-end UI (User Interface) through the HTTP (Hypertext Transfer Protocol) API. Supervisor can monitor the services of the heterogeneous component resource management module.
[0136] In some embodiments, the service layer includes an API server, a server information management unit, a container information management unit, a product information management unit, a compilation information management unit, and a deployment information management unit.
[0137] For example, an API server can be an HTTP server developed based on Golang. The API server provides a RESTful (Representational State Transfer) style interface to the outside world and is responsible for securely and legally providing internally managed data to users.
[0138] For example, the server information management unit is primarily responsible for managing information about all physical machines within the cluster. This physical machine information can include management node servers, currently used resource node servers, and standby servers for scaling up or down. The server information management unit can manage all server information, providing necessary information for subsequent server deployment, configuration, and recycling.
[0139] For example, the Container Information Management Unit manages information about all containers within the cluster. Container information can include the number of CPU cores used by the container, memory size, disk size, as well as the physical machine it belongs to and the application it is used for.
[0140] For example, the product information management unit is responsible for managing product information deployed within the private cloud. Product information can be structured in a tree structure. This tree structure can be divided into five levels: department, product line, product, system, and application. Before scaling up or down, the physical machines to be scaled up or down need to be assigned to specific system applications to facilitate centralized management of the scaling up / down process.
[0141] For example, the Build Information Management Unit is responsible for managing the build process for each product application. The Build Information Management Unit can be developed based on Jenkins and supports automatically pulling code from source code repositories (GitLab). Based on configuration information, the Build Information Management Unit can compile and build application binaries or images (including runtime images and application packages).
[0142] For example, the deployment information management unit is responsible for deploying the compiled program package or packaged image to the specified physical machine or container, and recording all deployment records.
[0143] The data layer can include primary storage nodes, secondary storage nodes, and storage. The primary storage nodes store various information from the storage layer and synchronously back it up to the secondary storage nodes. For example, data in the primary storage nodes can be backed up to storage periodically to prevent data loss if both the primary and secondary storage nodes are contaminated.
[0144] Figure 6 Schematic diagrams showing further embodiments of the cluster resource control apparatus of this disclosure are provided.
[0145] like Figure 6As shown, the multi-component scaling controller can be an SSH controller built using Golang. Scripts from various business lines can connect to the internal structure of the multi-component scaling controller.
[0146] In some embodiments, the multi-component scaling controller needs to utilize a concurrent scaling executor to execute pre-scripts and post-scripts. For example, the underlying layer of the concurrent scaling executor is a high-performance SSH concurrent connection pool based on Golang, which includes SSH connections to each resource to be scaled.
[0147] For example, a series of operation interfaces, such as connecting to physical machines and executing script commands, can be encapsulated on top of the connection pool. Due to the high concurrency characteristics of the Golang language, combined with the connection pool described above, it is possible to ensure large-scale concurrent operations on servers and containers, thereby improving the efficiency of scaling up and down.
[0148] For example, to ensure operational security, the connection pool establishes only one SSH connection for each resource (physical machine, container image) to be scaled up or down, and periodically cleans up expired SSH connections.
[0149] Figure 7 Schematic diagrams illustrating some embodiments of the cluster resource control method of this disclosure are shown.
[0150] like Figure 7 As shown, the method includes: the user submits relevant code to the Git distributed version control system; Git triggers the Jenkins automated server's compilation, build, and deployment process. For example, the compilation and build process can be triggered manually or automatically after the user submits code.
[0151] Jenkins uses Docker to compile, build, and deploy images. For example, it can generate either a package or a runtime image based on the application type. For instance, big data datanode resource nodes use package deployment, meaning the compilation and build output only contains a package, not a runtime image; while image deployment packages both the package and the runtime image together into a single runtime image.
[0152] Figure 8 Schematic diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure are shown.
[0153] like Figure 8 As shown, once the application has been compiled and built, the multi-component scaling controller, based on the type of application being scaled, transmits the data packets compiled and built by the unified component building system to the physical machine to be scaled or an idle machine in the resource pool through the concurrent scaling executor.
[0154] When expanding the physical machine package, the multi-component deployment system executes the startup script to deploy the package on the physical machine of the private cloud cluster and check the execution status; when deploying the program image, the multi-component deployment system starts the image and checks the container running status of the private cloud cluster.
[0155] Figure 9 Schematic diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure are shown.
[0156] like Figure 9 As shown, depending on the application deployment type, the component scaling-down and recycling system performs container resource recycling and physical machine resource recycling. When the application being recycled is a physical machine in a private cloud cluster, the component scaling-down and recycling system adds the physical machine to the standby pool and calls PXE to reinstall the system; when the application image (container) in a private cloud cluster is being recycled, the component scaling-down and recycling system destroys the container and reclaims the resources to the resource pool.
[0157] In some embodiments, before scaling down, the component scaling down and recycling system needs to check whether business data already exists in the scaling down server or container; and, the component scaling down and recycling system needs to check whether other services that applications depend on are deployed on the scaling down server or container. If business data or dependent services exist, the component scaling down and recycling system exits the scaling down process.
[0158] In some embodiments, in order to remove deployed services on a physical machine, the component recycling system can initiate a PXE installation process, reinstall the operating system and format the data disk of the scaled-down physical machine, and then put the scaled-down resources into a standby pool; or after destroying the container, put the scaled-down resources into a resource pool.
[0159] Figure 10 Block diagrams illustrating some embodiments of the cluster resource control apparatus of this disclosure are shown.
[0160] like Figure 10 As shown, the cluster resource control device 10 includes a determination unit 101, an addition unit 102, a generation unit 103, and an execution unit 104.
[0161] When the resource to be controlled is a resource to be expanded, the determining unit 101 determines the binding relationship between the resource to be expanded and the corresponding application.
[0162] The addition unit 102 adds the initialized resources to be expanded to the resource pool of the corresponding application according to the binding relationship.
[0163] In some embodiments, the adding unit 102 passes the relevant information of the resource to be expanded to the pre-script of the corresponding application, executes the pre-script, and completes the initialization of the resource to be expanded.
[0164] The generation unit 103 generates the data packet to be executed for the application to be processed based on the deployment type of the application to be processed.
[0165] In some embodiments, when the deployment type is package deployment, the resource to be expanded is a physical machine, and the generation unit 103 generates a program package of the application to be processed as an executable data package; when the deployment type is image deployment, the resource to be expanded is a container image, the generation unit 103 generates a program package of the application to be processed, and generates an executable data package based on the program package and the running image of the application to be processed.
[0166] The execution unit 104 deploys the data packets to be executed on the corresponding resources to be expanded in the resource pool of the application to be processed.
[0167] In some embodiments, when the resource to be expanded is a physical machine, the execution unit 104 sends the data packet to be executed to the physical machine for execution; when the resource to be expanded is a container image, the execution unit 104 sends the data packet to be executed to an idle physical machine in the corresponding resource pool for execution.
[0168] In some embodiments, the execution unit 104 obtains the relevant information of the resource to be expanded, and sends the relevant information of the resource to be expanded to the third-party program of the application to be processed through the deployment interface configured for the application to be processed, so that the third-party program can deploy the data packet to be executed on the resource to be expanded for execution according to its own deployment method.
[0169] In some embodiments, the execution unit 104 executes a post-script of the application to be processed. The post-script is used for at least one of the following processes: returning the scaling result to the cluster management node; creating the corresponding volume for the corresponding scaling resources; and cleaning up the garbage generated by the scaling process.
[0170] In some embodiments, the cluster resource control device 10 further includes an establishment unit 105, used to establish SSH connections with each resource to be expanded. The SSH connections are used to execute relevant scripts of the corresponding applications. The establishment unit 105 can only establish one SSH connection with each resource to be expanded at a time.
[0171] In some embodiments, the establishment unit 105 retains the corresponding SSH connection for a preset time period after the execution of the data packet to be executed by the corresponding application is completed.
[0172] In some embodiments, the cluster resource control device 10 further includes a scaling-down unit 106, which is used to determine whether there is important data in the resource to be scaled down and whether there are services that depend on the resource to be scaled down when the resource to be controlled is a resource to be scaled down; if there is no important data and no services that depend on the resource to be scaled down, the scaling-down unit 106 removes the resource to be scaled down from the cluster.
[0173] In some embodiments, the scaling-down unit transmits relevant information about the resource to be scaled down through a configured scaling-down interface, so as to remove the resource to be scaled down from the cluster.
[0174] In some embodiments, the scaling-down unit 106 obtains the scaling-down result by polling the configured query interface.
[0175] In some embodiments, after the shrinking unit 106 removes the resource to be shrunk from the cluster, if the resource to be shrunk is a physical machine, the adding unit 102 adds the resource to the resource pool; if the resource to be shrunk is a container image, the shrinking unit 106 destroys the container image, and the adding unit adds the resource to the resource pool.
[0176] In some embodiments, when the resource to be scaled up is a physical machine, the execution unit 104 executes a post-script for the scaling up process. The post-script is used to start the installation process to reinstall the operating system on the resource to be scaled up.
[0177] In the above embodiments, based on the binding relationship between the resources to be expanded and the applications, the application's data packets are deployed and executed on the corresponding resources to be expanded. This allows for resource expansion applicable to applications with different business needs, thereby improving the applicability of resource expansion.
[0178] Figure 11 Block diagrams illustrating other embodiments of the cluster resource control apparatus of this disclosure are shown.
[0179] like Figure 11 As shown, the cluster resource control device 11 of this embodiment includes: a memory 111 and a processor 112 coupled to the memory 111. The processor 112 is configured to execute the cluster resource control method in any embodiment of this disclosure based on instructions stored in the memory 111.
[0180] The memory 111 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory stores, for example, the operating system, application programs, bootloaders, databases, and other programs.
[0181] Figure 12 Block diagrams illustrating further embodiments of the cluster resource control apparatus of this disclosure are shown.
[0182] like Figure 12 As shown, the cluster resource control device 12 of this embodiment includes: a memory 1210 and a processor 1220 coupled to the memory 1210. The processor 1220 is configured to execute the cluster resource control method of any of the foregoing embodiments based on instructions stored in the memory 1210.
[0183] The memory 1210 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory stores, for example, the operating system, application programs, bootloaders, and other programs.
[0184] The cluster resource control device 12 may further include an input / output interface 1230, a network interface 1240, and a storage interface 1250. These interfaces 1230, 1240, and 1250, as well as the memory 1210 and processor 1220, can be connected, for example, via a bus 1260. The input / output interface 1230 provides a connection interface for input / output devices such as monitors, mice, keyboards, and touchscreens. The network interface 1240 provides a connection interface for various networked devices. The storage interface 1250 provides a connection interface for external storage devices such as SD cards and USB flash drives.
[0185] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media containing computer-usable program code.
[0186] Figure 13 Block diagrams illustrating some embodiments of the control system for cluster resources of this disclosure are shown.
[0187] like Figure 13 As shown, the cloud computing system 13 includes a cluster resource control device 131, which is used to execute the cluster resource control method in any of the above embodiments.
[0188] The methods for controlling cluster resources, the apparatus for controlling cluster resources, the cloud computing system, and the computer-readable storage medium according to this disclosure have been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.
[0189] The methods and systems of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.
[0190] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.
Claims
1. A method for controlling cluster resources, comprising: When the resource to be controlled is a resource to be expanded, determine the binding relationship between the resource to be expanded and the corresponding application, wherein the resource to be expanded is a physical machine or a container image; Based on the binding relationship, a scaling method matching the corresponding application is determined, including a standardized scaling method and a custom scaling method; The scaling process is performed according to the scaling method matched with the corresponding application, including: when the scaling method matched with the corresponding application is the standardized scaling method, adding the initialized resources to be scaled to the resource pool of the corresponding application; generating the data packet to be executed for the application to be processed according to the deployment type of the application to be processed; and deploying the data packet to be executed on the corresponding resources to be scaled in the resource pool of the application to be processed for execution. When the expansion method matching the corresponding application is a custom expansion method, the initialized resource to be expanded is added to the resource pool of the corresponding application; relevant information of the resource to be expanded is obtained; through the deployment interface configured for the application to be processed, the relevant information of the resource to be expanded is sent to the third-party program of the application to be processed, so that the third-party program can deploy the data packet to be executed on the resource to be expanded for execution according to its own deployment method. The step of generating the executable data package for the application to be processed includes: when the deployment type is package deployment, the resource to be expanded is a physical machine, generating the application package as the executable data package; when the deployment type is image deployment, the resource to be expanded is a container image, generating the application package, and generating the executable data package based on the application package and the running image of the application to be processed.
2. The control method according to claim 1, wherein, Before adding the initialized resources to be expanded to the resource pool of the corresponding application, the method includes: The relevant information of the resources to be expanded is passed to the pre-script of the corresponding application; Execute the aforementioned pre-script to complete the initialization of the resources to be expanded.
3. The control method according to claim 1, wherein, The step of deploying the data packet to be executed on the corresponding resources to be expanded in the resource pool of the application to be processed includes: If the resource to be expanded is a physical machine, the data packet to be executed is sent to the physical machine for execution; If the resource to be expanded is a container image, the data packet to be executed is sent to an idle physical machine in the corresponding resource pool for execution.
4. The control method according to claim 1 further includes: Execute the post-script of the application to be processed, the post-script being used for at least one of the following processes: Return the scaling result to the cluster management node; Create corresponding volumes for the respective resources to be expanded; Clean up the waste generated during the expansion and treatment.
5. The control method according to claim 1, further comprising: Establish a Secure Shell SSH connection with each resource to be expanded to execute the relevant scripts of the corresponding application. Only one SSH connection can be established with each resource to be expanded at the same time.
6. The control method according to claim 5 further includes: After the execution of the data packet to be executed by the corresponding application is completed, the corresponding SSH connection is retained for a preset time period.
7. The control method according to any one of claims 1-6, further comprising: When the resource to be controlled is a resource to be scaled down, determine whether the resource to be scaled down contains important data and whether there are services that depend on the resource to be scaled down. If there is no important data and no services depend on the resource to be scaled down, remove the resource to be scaled down from the cluster.
8. The control method according to claim 7, wherein, The step of removing the resource to be scaled down from the cluster includes: The configured scaling-down interface transmits the relevant information of the resource to be scaled down, so as to remove the resource to be scaled down from the cluster. Also includes: The scaling-down results are obtained by polling the configured query interface.
9. The control method according to claim 7, further comprising, after removing the resource to be scaled down from the cluster: If the resource to be scaled up is a physical machine, add the resource to the resource pool. If the resource to be scaled up is a container image, destroy the container image and add the resource to the resource pool.
10. The control method according to claim 9, further comprising: When the resource to be scaled up is a physical machine, a post-processing script for scaling up is executed. This post-processing script is used to initiate the installation process to reinstall the operating system on the resource to be scaled up.
11. A control device for cluster resources, comprising: A module for performing the cluster resource control method as described in any one of claims 1 to 10.
12. A control device for cluster resources, comprising: Memory; and A processor coupled to the memory, the processor being configured to execute the cluster resource control method of any one of claims 1-10 based on instructions stored in the memory.
13. A cloud computing system, comprising: A cluster resource control device for executing the cluster resource control method according to any one of claims 1-10.
14. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for controlling cluster resources according to any one of claims 1-10.