Server cluster resource scheduling method and system based on harris eagle algorithm

Abstract

The application relates to a server cluster resource scheduling method and system based on a Harris hawk algorithm, which comprises the following steps: S1, collecting node resource data in real time; S2, determining a current resource scheduling strategy according to a resource allocation algorithm and the node resource data; S3, deploying a Pod according to the current resource scheduling strategy; and S4, monitoring load data of each working node and determining a dynamically adjusted resource scheduling strategy according to the load data. After the application program is deployed, the load data of the working node is monitored, if the load of the working node is too high, step S1 is re-executed to dynamically adjust the resource scheduling strategy. Meanwhile, the resource allocation algorithm fully considers the utilization rates of CPU, memory, disk and network, and improves the Harris hawk algorithm to determine the resource scheduling strategy adjusted each time.

Description

Technical Field

[0001] This application relates to the field of server cluster technology, and in particular to a server cluster resource scheduling method and system based on the Harris Eagle algorithm. Background Technology

[0002] In a server cluster, different dependencies of an application are typically deployed in multiple containers, and the master node of the server cluster assigns worker nodes to each container for computation.

[0003] For example, in the Kubernetes container management platform, the default scheduler divides the application Pod deployment process into two phases: filtering and scoring. The filtering phase identifies worker nodes that meet the calculated metrics, while the scoring phase uses a selection strategy to score the remaining nodes, choosing the worker node with the highest score as the running location for the application Pod.

[0004] However, the default scheduler only allocates resources and schedules Pods during the initial deployment. After the application is running, resource usage changes as the workload on the nodes changes, and the resources allocated at deployment are insufficient to meet the growing needs of the Pods.

[0005] To enable dynamic adjustment of node resources in a server cluster as Pods change, this application provides a server cluster resource scheduling method and system based on the Harris Eagle algorithm. Summary of the Invention

[0006] To overcome the problems existing in related technologies, the first aspect of this application provides a server cluster resource scheduling method based on the Harris Eagle algorithm, comprising:

[0007] S1. Real-time collection of node resource data; the node resource data includes CPU utilization, memory utilization, disk utilization, and network utilization.

[0008] S2. Determine the current resource scheduling strategy based on the resource allocation algorithm and the node resource data;

[0009] S3. Deploy the pod according to the current resource scheduling strategy;

[0010] S4. Monitor the load data of each working node and determine the dynamically adjusted resource scheduling strategy based on the load data.

[0011] In one implementation, the node resource data is represented by a matrix, and the nodes...

[0012] The resource matrix is ​​as follows:

[0013]

[0014] Among them, u i,j It is the utilization rate of the j-th resource on worker node i.

[0015] In one implementation, S2 specifically includes: the resource allocation algorithm is an improvement on the Harris Eagle algorithm.

[0016] In one implementation, S2 specifically includes:

[0017] S201. Initialize the eagle flock; wherein, the position of an individual eagle is defined by the node resource matrix, and the individual eagle is used to simulate a worker node;

[0018] S202, Iterative optimization of the eagle flock;

[0019] S203, Update the location of the individual eagle:

[0020] S204. Determine whether to end the iteration based on the preset number of iterations or convergence conditions.

[0021] In one implementation, in S203, Cauchy perturbation is used to optimize individuals to enhance global search capability and convergence speed.

[0022] In one implementation, step S202 specifically includes:

[0023] The fitness value of each individual eagle is calculated using a fitness calculation function, and the eagle flock is iteratively optimized based on the fitness value.

[0024] The fitness calculation function is:

[0025]

[0026] Among them, X i w represents the fitness value of an individual eagle. j Let j be the weight parameter of the j-th resource. α is the variance of the resource utilization rate at node i, and α is the adjustment coefficient.

[0027] In one implementation, the variance calculation formula for the resource utilization rate of each node i is...

[0028] The formula is:

[0029]

[0030] Where M is the number of resource types, u i,j It refers to the use of the j-th resource on node i. It is the average utilization rate of all resources at node i.

[0031] In one implementation, The calculation formula is as follows:

[0032]

[0033] Where M is the number of resource types, u i,j It refers to the use of the j-th resource on node i. It is the average utilization rate of all resources at node i.

[0034] In one implementation, a set of hawks is initialized in the search space, where each hawk represents a possible node resource allocation scheme, and the dimension of each hawk corresponds to a different resource metric. The population is initialized using a Tent chaotic sequence to ensure population diversity.

[0035] The second aspect of this application provides a server cluster resource scheduling system based on the Harris Eagle algorithm, used to execute the resource scheduling method and update the resource scheduling strategy of the server cluster.

[0036] The technical solution provided in this application may include the following beneficial effects:

[0037] When deploying the application to a server cluster, this application obtains the resource data of each worker node and allocates worker nodes to the application's Pods using a resource allocation algorithm. After deployment, the load data of the worker nodes is monitored. If the load of a worker node is too high, step S1 is re-executed to adjust the resource scheduling strategy; if the load of the worker node is normal, the current resource scheduling strategy is maintained.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0039] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0040] Figure 1 This is a flowchart illustrating the server cluster resource scheduling method shown in the embodiments of this application;

[0041] Figure 2 This is another schematic diagram of the server cluster resource scheduling method shown in the embodiments of this application. Detailed Implementation

[0042] Preferred embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0043] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0044] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0045] Example 1

[0046] In existing server cluster resource scheduling and allocation algorithms, Pods are only allocated and scheduled during initial deployment. After the application starts running, as the workload on the nodes changes, the resources allocated during deployment are insufficient to meet the growing needs of the Pods.

[0047] To enable dynamic adjustment of node resources in a server cluster as Pods change, this application provides a server cluster resource scheduling method based on the Harris Eagle algorithm, such as... Figure 1 As shown, it includes the following steps:

[0048] S1. Real-time collection of node resource data;

[0049] S2. Determine the current resource scheduling strategy based on the resource allocation algorithm and the node resource data;

[0050] S3. Deploy the pod according to the current resource scheduling strategy;

[0051] S4. Monitor the load data of each working node, and determine the dynamically adjusted resource scheduling strategy based on the load data.

[0052] Specifically, the resource data includes CPU utilization, memory utilization, disk utilization, and network utilization.

[0053] Specifically, the resource scheduling strategy is the worker node address and computing resources allocated to each Pod.

[0054] In this embodiment, when deploying the application to the server cluster, resource data for each worker node is obtained, and worker nodes are allocated to the application's Pods using a resource allocation algorithm. After deployment, the load data of the worker nodes is monitored. If the load of a worker node is too high, step S1 is re-executed to adjust the resource scheduling strategy; if the load of a worker node is normal, the current resource scheduling strategy is maintained.

[0055] Example 2

[0056] To clearly illustrate the specific implementation of the resource allocation algorithm based on the server cluster resource scheduling method in Embodiment 1, this application also provides a server cluster resource scheduling method based on the Harris Eagle algorithm, including steps S1 to S4 as described in Embodiment 1, the process of which is as follows: Figure 1 As shown.

[0057] Furthermore, in step S1, the application is deployed on a Kubernetes cluster. The Prometheus monitoring system is responsible for collecting node resource data.

[0058] In this embodiment, the collected node resource data is encoded into a node resource matrix. The node resource matrix is:

[0059]

[0060] Among them, u i,j It is the utilization rate of the j-th resource on node i.

[0061] In this embodiment of the application, each column of the node resource matrix represents the resource utilization rate of a working node, and each row in the m-th column represents the utilization rate of a resource type.

[0062] Kubernetes clusters monitor resource usage, such as CPU utilization, memory utilization, disk I / O utilization, and network I / O utilization, by deploying exporters on each node.

[0063] Prometheus connects to the metrics data collection interface via RPC and uses Promql queries to retrieve node resource data from the Prometheus database.

[0064] Furthermore, the resource allocation algorithm in step S2 is an improvement upon the Harris Eagle algorithm. Specifically, each individual eagle has multiple dimensions.

[0065] Specifically, the dimension of each individual eagle i is represented as follows:

[0066] D i =M

[0067] Where M is the number of resource types (e.g., CPU, memory, disk I / O, network I / O).

[0068] The Harris Eagle Algorithm is a population optimization algorithm that simulates the predation behavior of Harris Eagles in nature. When solving population optimization problems using the Harris Eagle Algorithm, the positions of individual eagles, the prey, and the prey's energy are first defined.

[0069] In this embodiment of the application, the position of the individual eagle is the m-th column of the node resource matrix, which is the resource utilization rate of the i-th working node.

[0070] During group optimization, the time required to capture prey is calculated by continuously changing the positions of individual eagles, ultimately determining the optimal allocation scheme for eagle positions. In this embodiment, changing the position of an individual eagle is equivalent to changing the resource utilization rate of each column in the node resource matrix. Once the optimal allocation scheme is determined, the optimal node resource matrix can also be determined. Specifically, step S2 is as follows: Figure 2 As shown, it includes:

[0071] S201, Initialize the eagle flock;

[0072] Specifically, the eagle flock is initialized using a Tent chaotic sequence to ensure flock diversity.

[0073] S202, Iterative optimization of the eagle flock;

[0074] Specifically, the fitness value of each individual eagle is calculated based on node resource data. The fitness function comprehensively considers the weights of various indicators and actual data to dynamically adjust the resource allocation scheme.

[0075] S203, Update the location of the individual eagle:

[0076] Specifically, the eagle updates its position based on its historical best position and global best position in order to move in the search space and find the optimal solution.

[0077] In this embodiment of the application, the individual eagle x i Position representation: x i =[u i,1 ,...u i,j ].

[0078] Where i is the node currently under consideration, u i,j It is the utilization rate of the j-th resource on worker node i.

[0079] In this embodiment of the application, the energy formula for the prey is:

[0080]

[0081] Among them, E i This represents the prey energy of worker node i, where M is the number of resource types (e.g., CPU, memory, disk I / O, network I / O), and u i,j It is the utilization rate of the j-th resource on worker node i, w j It is the weight of the j-th resource, reflecting the importance of that resource in the prey energy calculation.

[0082] S204. Determine whether to end the iteration based on the preset number of iterations or convergence conditions.

[0083] In S203, Cauchy perturbation is used to optimize individuals in order to enhance global search capability and convergence speed.

[0084] In step S202, the fitness value of each individual eagle is calculated using a fitness calculation function. The fitness calculation function is:

[0085]

[0086] Among them, X i w represents the fitness value of an individual eagle. j Let j be the weight parameter of the j-th resource. α is the variance of the resource utilization rate at node i, and α is the adjustment coefficient.

[0087] In this embodiment, the fitness value of an individual eagle represents the resource utilization score of node i. The higher the fitness value, the higher the resource utilization score, indicating that the worker node is more suitable for scheduling new Pods. j This reflects the importance of the resource in scheduling decisions. Variance This reflects the uniformity of resource use; the smaller the variance, the more uniform the resource use. α is used to adjust for the effect of variance on the fitness value.

[0088] Furthermore, The calculation formula is

[0089]

[0090] The calculation formula is:

[0091]

[0092] Where M is the number of resource types, u i,j It refers to the use of the j-th resource on node i. It is the average utilization rate of all resources at node i.

[0093] In step S4, performance data after resource adjustments is collected periodically, including metrics such as node resource utilization and response time. This performance data is then analyzed in real time through the Prometheus monitoring system to monitor the effectiveness of resource balancing and performance improvement. If insufficient resource balancing or poor performance is detected, the system will review historical performance data and make adjustments using the improved Harris Eagle algorithm.

[0094] During the iteration process, the eagle updates its position based on its historical best position and global best position, while simultaneously optimizing using Cauchy perturbation and Tent chaotic sequences. This embodiment comprehensively considers actual usage data of CPU, memory, disk, and network, enabling dynamic adjustment of the resource allocation scheme. Furthermore, it adjusts the weight parameters in the resource allocation algorithm based on the analysis results, optimizing overall performance and re-evaluating resources and selecting nodes.

[0095] In this embodiment, it is proposed to optimize the population using Cauchy perturbation and Tent chaotic sequences, which can enhance global search capability and convergence speed. Tent chaotic sequences can improve the algorithm's search capability and avoid premature convergence.

[0096] The resource allocation method shown in this application embodiment utilizes the Harris Eagle algorithm to represent the resource utilization rate of i nodes as the dimension of an individual eagle.

[0097] During the node selection process, the resource allocation algorithm fully considers the utilization of CPU, memory, disk and network, and designs a fitness calculation function.

[0098] Once the fitness of the eagle flock meets the preset requirements or reaches the preset number of iterations, a new resource allocation strategy is obtained and fed back to the Kubernetes cluster for actual resource allocation and scheduling.

[0099] Furthermore, this embodiment of the application also monitors the resource utilization of each worker node by combining the Prometheus monitoring system. When the computing requirements of the Pod change, step S1 is re-executed to perform eagle swarm iteration, thereby achieving dynamic resource balancing and optimal scheduling of the Kubernetes cluster, improving resource utilization efficiency, and ensuring the performance and stability of the application.

[0100] Example 3

[0101] A server cluster resource scheduling system based on the Harris Eagle algorithm is used to update the resource scheduling strategy of the server cluster using the resource scheduling methods described in Embodiments 1 and 2.

[0102] The scheme of this application has been described in detail above with reference to the accompanying drawings.

[0103] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a particular embodiment can be found in the relevant descriptions of other embodiments. Those skilled in the art should also understand that the actions and modules described in the specification are not necessarily essential to this application.

[0104] Furthermore, it is understood that the steps in the method of this application embodiment can be adjusted, merged, or deleted in order according to actual needs, and the modules in the device of this application embodiment can be merged, divided, or deleted according to actual needs.

[0105] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0106] Alternatively, this application may be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) that, when executed by a processor of an electronic device (or electronic device, server, etc.), causes the processor to perform some or all of the steps of the methods described above according to this application.

[0107] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the present application can be implemented as electronic hardware, computer software, or a combination of both.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A server cluster resource scheduling method based on the Harris Eagle algorithm, characterized in that, include: S1. Real-time collection of node resource data; The node resource data includes CPU utilization, memory utilization, disk utilization, and network utilization. S2. Determine the current resource scheduling strategy based on the resource allocation algorithm and the node resource data. The resource allocation algorithm is an improvement on the Harris Eagle algorithm. S201. Initialize the eagle flock; where the position of an individual eagle is defined by the node resource matrix, and the individual eagle is used to simulate a worker node; S202, Iterative optimization of the eagle flock; S203. Update the position of the individual eagles and optimize them using Cauchy perturbation to enhance global search capability and convergence speed. S204. Determine whether to end the iteration based on the preset number of iterations or convergence conditions; S3. Deploy the Pod according to the current resource scheduling policy; S4. Monitor the load data of each working node, and determine the dynamically adjusted resource scheduling strategy based on the load data; The node resource data is represented by a matrix, and the node resource matrix is ​​as follows: in, It is the utilization rate of the j-th resource on worker node i.

2. The server cluster resource scheduling method based on the Harris Eagle algorithm according to claim 1, characterized in that, Step S202 specifically includes: The eagle flock is iteratively optimized based on its fitness value. The The variance of the utilization rate of each resource, The adjustment coefficient is M, where M is the number of resource types. This refers to the use of the j-th resource on worker node i. It is the average resource utilization of all worker node i.

3. A server cluster resource scheduling method based on the Harris Eagle algorithm according to claim 2, characterized in that, The working node Variance of resource utilization The calculation formula is: 。 4. A server cluster resource scheduling method based on the Harris Hawk algorithm as described in claim 3. The calculation formula is as follows: 。 5. A server cluster resource scheduling method based on the Harris Eagle algorithm according to claim 1, characterized in that, A set of hawks is initialized in the search space, where each hawk represents a possible node resource allocation scheme. The dimension of each hawk corresponds to a different resource index. The population is initialized using a Tent chaotic sequence to ensure population diversity.

6. A server cluster resource scheduling system based on the Harris Eagle algorithm, characterized in that, The resource scheduling strategy of the server cluster is used to execute the resource scheduling method described in claims 1 to 5.

Images