[0055] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.
[0056] In related technologies, when applying for resources from the business in the cluster management platform to create a new container, it is first necessary to monitor and manage the resource margin of the entire cluster to ensure that there are sufficient resources in the appropriate physical machine. Complete the creation of the new container. Taking the HULK scheduling platform as an example, the kubernetes application (abbreviated as: K8s, which is an open source, used to manage containerized applications on multiple physical machines in the cloud platform) is usually used to uniformly manage resources in the cluster. During the operation of the cluster, the K8s application monitors the resource margin of the cluster. It is possible to pull all resource data monitored within the preset time every preset time (for example, one hour), and calculate the resource margin of the cluster based on the pulled resource data, and save it in the database in. When querying the resource margin, you can view the saved resource margin data from the database. However, the above method of pulling the cluster data in full takes a long time, and the calculation of the resource margin is performed at intervals of a period of time, which lacks real-time performance, which leads to a poor success rate of the entire cluster expansion and affects the commitment to the business And the stability of the system.
[0057] The inventor noticed this problem and proposed a method for determining the resource margin, which is specifically as follows:
[0058] The execution subject of the cluster resource management method provided by the embodiments of the present disclosure may be a cluster management device or a functional module and/or functional entity in the cluster management device or a physical machine in the cluster that can implement the cluster resource management method. The specifics can be determined according to actual use requirements, and the embodiments of the present disclosure do not limit it.
[0059] Taking the cluster management device as the execution subject of the cluster resource management method as an example, the cluster resource management method provided by the embodiment of the present disclosure will be exemplarily described below. In the embodiments of the present disclosure, the cluster resource management method provided above can be applied to the cluster scheduling field, and specifically can be applied to various public cloud and private cloud cloud products in the cluster scheduling field in scenarios involving cluster resource scheduling.
[0060] figure 1 It is a flowchart of a method for determining resource margin according to an exemplary embodiment, such as figure 1 As shown, applied to a cluster management device, the cluster management device is used to manage a cluster composed of multiple physical machines, and the method includes the following steps:
[0061] In step 101, monitor target management events in the cluster.
[0062] Wherein, the target management event is a management event for a target physical machine or a container in the target physical machine, the cluster includes at least one physical machine, and the target physical machine is any physical machine in the cluster.
[0063] For example, a management event on a physical machine or a container in a physical machine usually results in an increase or decrease in the resources of the entire cluster. For example, management events for a physical machine can include: physical machine addition, deletion, modification, and error events; events for a container in a physical machine can include: container creation, waiting, Modify events, delete time and error events, etc. Generally, the resources in the cluster can be managed uniformly by the kubernetes (abbreviation: K8s) platform. The K8s platform is an open source, used to manage containerized applications on multiple physical machines in the cloud platform. In the embodiment of the present disclosure, the cluster management device can monitor the events of the cluster by monitoring all events in the K8s platform.
[0064] In step 102, according to the monitored target management event and the historical resource reserve of the cluster, the target resource reserve of the cluster after the management event is executed is determined.
[0065] For example, in the embodiment of the present disclosure, the resource margin of each physical machine in the cluster can be learned from the historical resource margin of the cluster, and when a physical machine is monitored and joined to the scheduling of the cluster, you can The total amount of resources added to the cluster of the central processing unit (CPU), memory, data disk, and mirror disk of the physical machine, so as to obtain the target resource margin. And, in the case of monitoring the creation event of the container, if the container is scheduled to a certain physical machine in the cluster, the resource margin of this physical machine is subtracted from the amount of resources occupied by the container to obtain this physical machine The remaining resource margin. Correspondingly, if the delete event of the container is monitored, the resource margin of the physical machine where the container is located is added to the resource amount occupied by the container, and then the target resource margin is calculated.
[0066] Taking HLUK as an example, the cluster management method can include: the cluster management device can set the k8s subscription forwarding center to monitor all events of the k8s to learn all the events of the cluster. And these monitored events are sent to the resource margin service through the message form of the message queue for unified processing. After the resource margin service receives the event, it can use the management module to subscribe to the event of the cluster monitored by the forwarding center through k8s, and the current state of the cluster stored in the cache, calculate the resource margin of the cluster, and calculate the result (That is, the target resource margin in the above embodiment) is stored in the database. External calls can obtain the resource margin by directly obtaining the data in the database. After the resource margin service receives the event, the management module (usually called: carter-manager) can save the information (which can be an identifier) and current state of the physical machine and container to the cache (usually called in k8s) : Cellar).
[0067] For example, the management module in the resource margin service can also obtain the status information of the cluster, and update the status information of the cluster according to the status of the cluster and the events of the cluster monitored by the k8s subscription forwarding center. State information is stored in the cache. The management module in the resource margin service can also save the target resource margin in the historical database after calculating the target resource margin each time for subsequent analysis of the trend of the resource margin. The service module in the resource margin service is mainly used to provide the resource margin of the cluster when the resource of the cluster needs to be called externally.
[0068] For example, for a physical machine, the physical machine is scheduled to join the machine, and the resource margin management platform perceives this change and judges whether the data of the physical machine's CPU, memory, data disk, and mirror disk can be added to the resources of the cluster Total. If the physical machine has a modification event, the resource margin management platform perceives this change, compares whether the data of the physical machine's CPU, memory, data disk, and mirror disk has changed, and then determines the resource margin after the change. If the physical machine has a delete event, the resources of the deleted physical machine will be removed from the total resources. Similarly, for any container, after a business request applies for the container, it enters the container creation stage. When the container is created, the resource margin service perceives this change (that is, receives the event of the k8s subscription forwarding center to create the container), and judges that the container has Not scheduled to a physical machine. If it has been scheduled, the resources of this physical machine are subtracted from the resources occupied by this container. If it has not been scheduled to any physical machine, save this state and wait for the container modification event. Once the physical machine where the container is located in the received event is not empty, the resources of this physical machine are subtracted from the resources occupied by the container . Correspondingly, once the delete event is received, the resources of this physical machine are added to the resources occupied by this container.
[0069] In summary, the technical solution provided by the embodiments of the present disclosure monitors target management events in a cluster, where the target management event is a management event for a target physical machine or a container in the target physical machine. Contains at least one physical machine, the target physical machine is any physical machine in the cluster; according to the monitored target management event and the historical resource reserve of the cluster, determine the target resource reserve of the cluster after the management event is executed the amount. After monitoring the management event of the cluster, it can immediately determine the impact of the management event on the amount of resources of the cluster, and then determine the total amount of resources of the cluster, so as to improve the efficiency and real-time performance of resource margin monitoring and analysis.
[0070] figure 2 is based on figure 1 Shows a flowchart of a method for monitoring and managing events, such as figure 2 As shown, this step 101 includes:
[0071] In step 1011, the event message directed to the target physical machine or the container is monitored.
[0072] In step 1012, according to the state information of the target physical machine or the state information of the container, it is determined whether the event corresponding to the event message is an executable event that can change the state of the target physical machine or the state of the container.
[0073] For example, in the above step 1012, the correctness of the state circulation of the cluster can be ensured by the finite state machine, that is, it can be judged whether to execute the received event according to the received message for executing an event and the status information of the cluster. For example, if the physical machine does not exist, but a delete event is received, it is considered that the delete event is received before the creation event. The retransmission of this message is rejected, and the creation message is received first before processing the deletion. Specifically, in the case that there is no physical machine in the cluster, and the message of deleting the physical machine is received, it can be considered that the delete event is received before the creation event, and the message of deleting the physical machine can be rejected until After the message of creating the physical machine is received, the event of creating the physical machine is executed, and the message of deleting the event of the physical machine is received again, the event of deleting the physical machine is executed.
[0074] For example, usually the state information of the cluster is stored in the cache. In the embodiment of the present disclosure, the cluster management device can obtain the state information of the cluster from the cache. In K8s, the state information of the physical machine is usually characterized by the state of the node, and the state of the container can be characterized by the state of the pod. Among them, node and pod are both components in K8s. Among them, pod is an abstract concept, it contains one or more container groups composed of containers, as well as the resources shared by these containers. These resources include: shared storage, the use of a unique cluster IP address, and configuration information on how to run the container , Such as image version and container port. The pod model is similar to a logical host with a specific application. It can contain different application containers that are relatively tightly coupled. The containers in the same pod share an Internet Protocol (IP) address and port segment for collaborative work and scheduling. And run in a shared environment on the same node. Among them, pod must be running on node. A node is a working machine in K8s, which can be a physical machine or a virtual machine. A node can run multiple pods. According to the resource situation of each node, pods can be deployed on the nodes in the cluster.
[0075] In step 1013, if it is determined that the management event corresponding to the event message is the executable event, it is determined that the management event corresponding to the event message is the target management event.
[0076] image 3 is based on figure 2 Shows a flow chart of a method for determining cluster resource margin, such as image 3 As shown, this step 102 includes:
[0077] In step 1021, it is determined whether the target management event is a resource management event.
[0078] Wherein, the resource management event is the first management event of adding, modifying or deleting the target physical machine in the cluster, or the second management event of creating, modifying or deleting a container on the target physical machine.
[0079] For example, after the received resource management event is executed, the finite state machine is used to control the increase or decrease of the cluster resource margin to obtain the target resource margin, so that the resource margin can be accurately obtained in real time.
[0080] In step 1022, if it is determined that the target management event is the resource management event, the target resource reserve is determined according to the target management event and the historical resource reserve of the cluster.
[0081] Exemplarily, the step 1022 includes: determining the resource change amount of the cluster after the target management event is executed, the resource change amount being the resource increase amount or the resource decrease amount; according to the historical resource margin and the resource change amount, determining The target resource margin.
[0082] Optionally, in the embodiments of the present disclosure, a container creation event is taken as an example to provide an optional implementation manner, and the implementation manner includes the following steps:
[0083] a. The cluster management device receives the first service request.
[0084] Wherein, the first service request is used to request the creation of a first container;
[0085] b. The cluster management device obtains the historical resource margin of the cluster.
[0086] c. The cluster management device allocates resources to the target physical machine in the cluster for the first container according to the historical resource margin of the cluster.
[0087] d. Monitor cluster events.
[0088] e. After monitoring the event of creating the first container on the target physical machine, calculate the target resource surplus of the cluster based on the historical resource surplus of the cluster and the resource amount of the target physical machine occupied by the first container the amount.
[0089] In the above embodiment, in the case of monitoring that the first container is created on the target physical machine, the cluster may be calculated based on the historical resource margin of the cluster and the resource amount of the target physical machine occupied by the first container. The target resource margin of the cluster can be calculated in real time and accurately every time a cluster event is monitored, so that the resource margin of the cluster can be obtained efficiently, real-time and accurately.
[0090] Figure 4 is based on figure 1 Shows a flowchart of another method for determining resource margin, such as Figure 4 As shown, after step 101, the method further includes:
[0091] In step 103, if it is determined that the management event corresponding to the event message is the executable event, the status information of the target physical machine or the status information of the container is updated.
[0092] For example, in the embodiment of the present disclosure, the status information of the original cluster (which can be the status information of the cluster acquired last time, that is, the status information of the cluster acquired before the event of the cluster is monitored this time) can be combined with the monitoring For the management events of the cluster, update the state information of the original cluster to obtain the state information of the cluster at the current point in time, and then save the updated state information in the cache. Among them, the cluster management device may save the current status information of the cluster in the cache, so that the cluster status information can be updated after the next management event of the cluster is monitored.
[0093] Figure 5 is based on Figure 4 The flow chart of another method for determining resource margin is shown, such as Figure 5 As shown, after step 102, the method further includes:
[0094] In step 104, the historical resource reserve record of the cluster before executing the target management event is updated by the resource reserve record generated according to the target resource reserve, so as to determine according to the updated historical resource reserve record The trend of the cluster's resource margin.
[0095] For example, the historical resource margin record is stored in the historical database of the cluster, and the historical database and the resource margin database may be different databases, wherein a database with a cost lower than the resource margin database can be used as the historical database to save the history Resource margin records to facilitate subsequent use of historical resource margin records for analysis. As an optional implementation, or, after the cluster management device saves the target resource margin in the historical database, in the cluster resource management method provided by the embodiment of the present disclosure, the cluster management device may also be based on the current time of the cluster Click multiple historical resource margin records before clicking, analyze the change trend of the resource margin of the cluster, and then manage the physical machines in the cluster. For example, when the change trend of the resource margin is more and more surplus, you can Reduce the number of physical machines in the cluster; when the trend of the resource margin is becoming more and more scarce, you can increase the number of physical machines in the cluster, so that the cluster can be managed according to the trend of the resource margin in the cluster through data analysis Resources, adjust the cluster structure more reasonably.
[0096] In summary, the technical solutions provided by the embodiments of the present disclosure can monitor target management events in a cluster, where the target management event is a management event for a target physical machine or a container in the target physical machine. The cluster includes at least one physical machine, and the target physical machine is any physical machine in the cluster; according to the monitored target management event and the historical resource margin of the cluster, determine the target resource of the cluster after the management event is executed margin. After monitoring the management event of the cluster and ensuring the correctness of the event, determine the impact of the management event on the resource amount of the cluster, and then determine the resource margin of the cluster, and improve the efficiency and real-time performance of resource margin monitoring And precision.
[0097] Image 6 It is a block diagram of a device for determining resource margin according to an exemplary embodiment, such as Image 6 As shown, the device 600 includes:
[0098] The event monitoring module 610 is used to monitor target management events in the cluster, where the target management event is a management event for a target physical machine or a container in the target physical machine, the cluster includes at least one physical machine, and the target physical machine The machine is any physical machine in the cluster;
[0099] The resource margin determining module 620 is configured to determine the target resource margin of the cluster after the management event is executed according to the monitored target management event and the historical resource margin of the cluster.
[0100] Optionally, the event monitoring module 610 is used to:
[0101] Monitor event messages for the target physical machine or the container;
[0102] According to the state information of the target physical machine or the state information of the container, determine whether the event corresponding to the event message is an executable event that can change the state of the target physical machine or the state of the container;
[0103] In the case of determining that the management event corresponding to the event message is the executable event, it is determined that the management event corresponding to the event message is the target management event.
[0104] Optionally, the resource margin determining module 620 is configured to:
[0105] Determine whether the target management event is a resource management event. The resource management event is the first management event of adding, modifying, or deleting the target physical machine in the cluster, or the first management event of creating, modifying, or deleting a container on the target physical machine. 2. Management events;
[0106] If it is determined that the target management event is the resource management event, the target resource reserve is determined according to the target management event and the historical resource reserve of the cluster.
[0107] Optionally, the resource margin determining module 620 is configured to:
[0108] Determine the resource change amount of the cluster after the target management event is executed, and the resource change amount is the resource increase or the resource decrease;
[0109] According to the historical resource margin and the resource change amount, the target resource margin is determined.
[0110] Figure 7 is based on Image 6 The block diagram of another device for determining resource margin is shown, such as Figure 7 As shown, the device 600 further includes:
[0111] The status update module 630 is configured to update the status information of the target physical machine or the status information of the container when it is determined that the management event corresponding to the event message is the executable event.
[0112] Figure 8 is based on Figure 7 A block diagram of another device for determining resource margin is shown, such as Figure 8 As shown, the device 600 further includes:
[0113] The record update module 640 is configured to update the historical resource reserve record of the cluster before the target management event is executed by the resource reserve record generated according to the target resource reserve, so as to update the historical resource reserve record according to the updated historical resource reserve record , To determine the trend of the cluster’s resource margin.
[0114] In summary, the technical solutions provided by the embodiments of the present disclosure can monitor target management events in a cluster, where the target management event is a management event for a target physical machine or a container in the target physical machine. The cluster includes at least one physical machine, and the target physical machine is any physical machine in the cluster; according to the monitored target management event and the historical resource margin of the cluster, determine the target resource of the cluster after the management event is executed margin. After monitoring the management event of the cluster and ensuring the correctness of the event, determine the impact of the management event on the resource amount of the cluster, and then determine the resource margin of the cluster, and improve the efficiency and real-time performance of resource margin monitoring And precision.
[0115] For example, Picture 9 It is a block diagram showing an electronic device 900 according to an exemplary embodiment. For example, the electronic device 900 may be provided as a server. Reference Picture 9 The server 900 includes a processor 901, the number of which may be one or more, and a memory 902, configured to store a computer program executable by the processor 901. The computer program stored in the memory 902 may include one or more modules each corresponding to a set of instructions. In addition, the processor 901 may be configured to execute the computer program to execute the above-mentioned method for determining the resource margin.
[0116] In addition, the server 900 may also include a power supply component 903 and a communication component 904. The power supply component 903 may be configured to perform power management of the server 900, and the communication component 904 may be configured to implement communication with the server 900, for example, wired or wireless communication. . In addition, the server 900 may further include an input/output (I/O) interface 905. The server 900 can operate based on an operating system stored in the storage 902, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM and so on.
[0117] In another exemplary embodiment, there is also provided a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the above-mentioned method for determining a resource margin. For example, the computer-readable storage medium may be the aforementioned memory 902 including program instructions, and the aforementioned program instructions may be executed by the processor 901 of the server 900 to complete the aforementioned method for determining the resource margin.
[0118] The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details in the above-mentioned embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure. These simple modifications all belong to the protection scope of the present disclosure.
[0119] In addition, it should be noted that the various specific technical features described in the above-mentioned specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure provides various possible The combination method will not be explained separately.
