Cluster resource scheduling method, device, server and storage medium

Abstract

The present disclosure relates to a cluster resource scheduling method, device, server and storage medium, the method comprising: receiving current over-allocated resources sent by a server node in a cluster, the current over-allocated resources are determined by the server node based on the used resources and total allocatable resources of the currently running normal job, including unallocated resources and allocated but currently unused resources, the normal job is a job running based on allocated resources; scheduling a non-running job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the non-running job, the job execution parameter value is the execution order value corresponding to the job. The present disclosure can make full use of the unused resources in the application resources of the normal job to run the job with a lower job execution parameter value, solve the problem of resource waste caused by the difference between the application value and the usage value of the job, and improve the resource utilization rate in the cluster.

Description

Technical Field

[0001] The present disclosure relates to the technical field of resource scheduling, and in particular to a cluster resource scheduling method, device, server and storage medium. Background Art

[0002] In the related technology, the distributed resource management and scheduling system is basically allocated and managed based on the application quota. For example, the Hadoop ecosystem allocates resources based on the configuration resources of the machine. In order to ensure the stability of the system, resources will be reserved for the operating system. The configuration resources will be less than the physical resources of the machine. The resource allocation is strictly allocated according to the total configuration of all machines in the cluster. Because the application resources will basically not be fully used during the execution of the job, even if the cluster resource allocation is full, it is difficult to achieve a high level of machine utilization, and the physical server resources cannot be fully utilized; the k8s cloud ecological resource management system provides burstable and besteffort resources to complement each other, but there is still a problem of cluster physical resource waste caused by the large difference between the application value and the usage value.

[0003] In the related art, resources are allocated and managed according to application quotas, and the usage value often fails to reach the application value, which causes a phenomenon in which logical resources are tight and physical machine resources are idle, thereby wasting a large amount of physical server resources. Summary of the invention

[0004] The present disclosure provides a cluster resource scheduling method, device, server and storage medium to at least solve the problem of physical resource waste caused by the difference between application value and usage value in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a cluster resource scheduling method is provided, which is applied to a scheduling center server in a cluster, and includes:

[0006] Receive current over-allocated resources sent by a server node in the cluster, where the current over-allocated resources are determined by the server node based on the used resources and the total allocatable resources of the currently running normal job, including unallocated resources and allocated but currently unused resources, where the normal job is a job running based on the allocated resources;

[0007] Schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the unrunning job, and the job execution parameter value is a value of an execution order corresponding to the job.

[0008] Optionally, also include:

[0009] Get historical resource usage data for normal operations;

[0010] Determine the periodic change pattern of the consumption resources of normal operation over time according to the historical resource usage data;

[0011] According to the periodic variation rule, the maximum over-allocation value allowed for the server node is periodically adjusted.

[0012] According to a second aspect of an embodiment of the present disclosure, a cluster resource scheduling method is provided, which is applied to a server node in a cluster, and includes:

[0013] Obtaining used resources of a currently running normal job, where the normal job is a job running based on allocated resources;

[0014] Determine current over-allocated resources according to the used resources and the total allocatable resources, wherein the current over-allocated resources include unallocated resources and allocated but currently unused resources;

[0015] Sending the current over-allocated resources to a scheduling center server, the scheduling center server being used to schedule unrunning jobs whose job execution parameter values ​​are lower than those of normal jobs to the current over-allocated resources, wherein the job execution parameter values ​​are values ​​corresponding to the execution order of the jobs;

[0016] The unrunning job assigned by the scheduling center server is received, the unrunning job is used as an over-allocated job, and the over-allocated job is run using the current over-allocated resources.

[0017] Optionally, determining the current over-allocated resources according to the used resources and the allocatable total resources includes:

[0018] Determining an over-allocation coefficient according to the used resources and the total allocatable resources;

[0019] Determine the number of times that historical over-allocated resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number;

[0020] The current over-allocation resource is determined according to the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient.

[0021] Optionally, the total allocatable resources also include a buffer zone;

[0022] The determining of the over-allocation coefficient according to the used resources and the total allocatable resources includes:

[0023] If the used resources are not 0, determining an over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources;

[0024] If the used resource is 0, the over-allocation coefficient is determined to be a preset over-allocation coefficient.

[0025] Optionally, determining the over-allocation coefficient according to the used resources, the buffer zone, and the total allocatable resources includes:

[0026] The ratio of the sum of the used resources and the buffer to the total allocatable resources is determined, and the difference between 1 and the ratio is used as the over-allocation coefficient.

[0027] Optionally, after using the current over-allocated resources to run the over-allocated job, the method further includes:

[0028] If a burst resource request of the normal operation is received, the buffer is used to process the burst resource request.

[0029] Optionally, when using the buffer to process the burst resource request, the method further includes:

[0030] Obtaining the running time of the over-allocated jobs, and sorting the over-allocated jobs in order of the running time from shortest to longest;

[0031] According to the order of the over-allocated jobs, the current over-allocated resources used by the over-allocated jobs are recovered in sequence until the recovered current over-allocated resources meet the resources requested by the burst resource request;

[0032] The recovered current over-allocated resources are used to process the burst resource request and release the buffer.

[0033] Optionally, determining the over-allocation attenuation coefficient according to the number of times includes:

[0034] According to the times, the over-allocation attenuation coefficient is determined according to the following formula:

[0035]

[0036] Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

[0037] Optionally, obtaining the used resources of the currently running normal job includes:

[0038] Get the actual resources used by normal operations and the resource quotas applied for normal operations;

[0039] The maximum value of the preset ratio of the applied resource quota and the actually used resources is used as the used resources for normal operation.

[0040] Optionally, also include:

[0041] Setting a memory overflow score of the normal job as a first score, and setting a memory overflow score of the over-allocated job running based on the current over-allocated resources as a second score, wherein the second score is greater than the first score;

[0042] If memory overflow occurs, the over-allocated job is stopped according to the first score and the second score, and the current over-allocated resources occupied by the over-allocated job are released.

[0043] According to a third aspect of an embodiment of the present disclosure, a cluster resource scheduling device is provided, which is applied to a scheduling center server in a cluster, including:

[0044] An over-allocated resource receiving module is configured to receive current over-allocated resources sent by a server node in the cluster, wherein the current over-allocated resources are determined by the server node based on the used resources and the total allocatable resources of the currently running normal job, including unallocated resources and allocated but currently unused resources, wherein the normal job is a job running based on the allocated resources;

[0045] The over-allocated resource scheduling module is configured to schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the unrunning job, and the job execution parameter value is the value of the execution order corresponding to the job.

[0046] Optionally, the cluster resource scheduling device further includes:

[0047] A historical resource data acquisition module is configured to acquire historical resource usage data of normal operations;

[0048] A resource rule determination module is configured to determine a periodic change rule of consumption resources of a normal operation over time according to the historical resource usage data;

[0049] The allowed over-allocation value adjustment module is configured to periodically adjust the allowed over-allocation maximum value of the server node according to the periodic variation rule.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a cluster resource scheduling device is provided, which is applied to a server node in a cluster, including:

[0051] A used resource acquisition module is configured to acquire used resources of a currently running normal job, where the normal job is a job running based on allocated resources;

[0052] An over-allocated resource determination module is configured to determine current over-allocated resources according to the used resources and the total allocatable resources, wherein the current over-allocated resources include unallocated resources and allocated but currently unused resources;

[0053] An over-allocated resource sending module is configured to send the current over-allocated resources to a scheduling center server, and the scheduling center server is used to schedule the unrunning jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources;

[0054] The over-allocated job running module is configured to receive the unrunning job assigned by the scheduling center server, use the unrunning job as the over-allocated job, and use the current over-allocated resources to run the over-allocated job.

[0055] Optionally, the over-allocated resource determination module includes:

[0056] an over-allocation coefficient determining unit, configured to determine an over-allocation coefficient according to the used resources and the total allocatable resources;

[0057] An over-allocation attenuation coefficient determination unit is configured to determine the number of times the historical over-allocation resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number;

[0058] The over-allocation resource determination unit is configured to determine the current over-allocation resource according to the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient.

[0059] Optionally, the total allocatable resources also include a buffer zone;

[0060] The over-allocation coefficient determination unit comprises:

[0061] A first over-allocation coefficient determination subunit is configured to determine an over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources if the used resources are not 0;

[0062] The second over-allocation coefficient determination subunit is configured to determine the over-allocation coefficient as a preset over-allocation coefficient if the used resource is 0.

[0063] Optionally, the first overmatching coefficient determination subunit is specifically configured as follows:

[0064] If the used resources are not 0, the ratio of the sum of the used resources and the buffer to the total allocatable resources is determined, and the difference between 1 and the ratio is used as the over-allocation coefficient.

[0065] Optionally, the cluster resource scheduling device further includes:

[0066] The burst request processing module is configured to use the buffer to process the burst resource request if a burst resource request of the normal operation is received.

[0067] Optionally, the cluster resource scheduling device further includes:

[0068] The over-allocated job sorting module is configured to obtain the running time of the over-allocated jobs and sort the over-allocated jobs in order of the running time from shortest to longest;

[0069] An over-allocated resource recovery module is configured to recover the current over-allocated resources used by the over-allocated jobs in sequence according to the order of the over-allocated jobs, until the recovered current over-allocated resources meet the resources requested by the burst resource request;

[0070] The buffer releasing module is configured to process the burst resource request using the recovered current over-allocated resources and release the buffer.

[0071] Optionally, the over-allocation attenuation coefficient determining unit is specifically configured as follows:

[0072] Determine the number of times that historical over-allocated resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number of times according to the following formula:

[0073]

[0074] Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

[0075] Optionally, the used resource acquisition module includes:

[0076] A resource acquisition unit is configured to acquire actually used resources of a normal operation and acquire an applied resource quota of a normal operation;

[0077] The used resource determination unit is configured to use the maximum value of the preset ratio of the applied resource quota and the actually used resources as the used resources for normal operation.

[0078] Optionally, the cluster resource scheduling device further includes:

[0079] a memory overflow score setting module, configured to set the memory overflow score of the normal job as a first score, and set the memory overflow score of the over-allocated job running based on the current over-allocated resources as a second score, wherein the second score is greater than the first score;

[0080] The memory overflow processing module is configured to stop running the over-allocated job and release the current over-allocated resources occupied by the over-allocated job according to the first score and the second score if memory overflow occurs.

[0081] According to a fifth aspect of an embodiment of the present disclosure, a server is provided, including:

[0082] processor;

[0083] a memory for storing instructions executable by the processor;

[0084] The processor is configured to execute the instruction to implement the cluster resource scheduling method as described in the first aspect or the cluster resource scheduling method as described in the second aspect.

[0085] According to a sixth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of a server, the server can execute the cluster resource scheduling method described in the first aspect or the cluster resource scheduling method described in the second aspect.

[0086] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, which includes a readable program code. When the readable program code is executed by a processor of an electronic device, the electronic device can execute the cluster resource scheduling method as described in the first aspect or the second aspect.

[0087] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0088] The scheduling center server of the disclosed embodiment receives the current over-allocated resources sent by the server nodes in the cluster, and schedules the non-running jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources of the server nodes, so that the server nodes use the current over-allocated resources to run the non-running jobs. Since the current over-allocated resources include unallocated resources and allocated but currently unused resources, the unused resources in the application resources of the normal jobs can be fully utilized to run the jobs with lower job execution parameter values, thereby solving the problem of resource waste caused by the difference between the application value and the usage value of the job and improving the resource utilization rate within the cluster.

[0089] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0091] Figure 1 is a flow chart of a cluster resource scheduling method according to an exemplary embodiment;

[0092] Figure 2 is a flow chart of a cluster resource scheduling method according to an exemplary embodiment;

[0093] Figure 3is a flow chart of a cluster resource scheduling method according to an exemplary embodiment;

[0094] Figure 4 is a schematic diagram of machine resources of a server node in an embodiment of the present disclosure;

[0095] Figure 5 is a block diagram of a cluster resource scheduling device according to an exemplary embodiment;

[0096] Figure 6 is a block diagram of a cluster resource scheduling device according to an exemplary embodiment;

[0097] Figure 7 It is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0098] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0099] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0100] Figure 1 is a flow chart of a cluster resource scheduling method according to an exemplary embodiment, wherein the cluster includes a scheduling center server and a server node, and the cluster resource scheduling method is used in the scheduling center server in the cluster, such as Figure 1 As shown, the cluster resource scheduling method includes the following steps.

[0101] In step S11, current over-allocated resources sent by a server node in the cluster are received. The current over-allocated resources are determined by the server node based on the used resources and the total allocatable resources of the currently running normal job. The normal job is a job running based on the allocated resources.

[0102] Among them, when each job is configured, the user who configures the job can also configure the job execution parameter value corresponding to the job according to the importance of the job. The job execution parameter value specifies the execution order of the job. The job execution parameter value corresponding to the job with a lower importance is lower, and the execution order is later. Resources can be allocated to the jobs corresponding to the job execution parameter values ​​in the order from high to low. The job execution parameter value can be, for example, a priority value. Normal jobs are generally jobs with higher job execution parameter values. Normal jobs apply for resources based on the application resource quota and obtain the allocated resources corresponding to the application resource quota, and run based on the allocated resources. The resources may include memory and / or CPU. The currently over-allocated resources may include unallocated resources and allocated but currently unused resources. The total allocable resources include unallocated resources, used resources of normal jobs, and allocated but currently unused resources.

[0103] The scheduling center server allocates resources to jobs in the order of job execution parameter values ​​from high to low, so that jobs with higher job execution parameter values ​​are run based on the allocated resources, and thus these jobs are treated as normal jobs. Since the requested resources for each job are generally more than the used resources, that is, the actual used resources of the job during runtime generally do not reach the requested resources, the allocated but currently unused resources can be used as over-allocated resources to run other jobs. That is, each server node in the cluster determines the resources that can be over-allocated based on the used resources of the normal jobs currently running and the total allocable resources of the server nodes, as the current over-allocated resources, and sends the current over-allocated resources to the scheduling center server, so that the scheduling center server receives the current over-allocated resources sent by the server nodes in the cluster.

[0104] In step S12, the non-running job whose job execution parameter value is lower than that of the normal job is scheduled to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the non-running job, and the job execution parameter value is the value of the execution order corresponding to the job.

[0105] The scheduling center server takes the sum of the current over-allocated resources of each server node in the cluster as the cluster over-allocated resources, and allocates and schedules the non-running jobs whose job execution parameter values ​​are lower than those of normal jobs based on the cluster over-allocated resources. That is, these non-running jobs are scheduled to the current over-allocated resources of each server node, so that the server nodes can use the current over-allocated resources to run the non-running jobs, thereby filling the idle resources of the server nodes.

[0106] The unrunning job can be called an over-allocated job after it is run using the current over-allocated resources. Because the job execution parameters of the over-allocated job are lower, when resource competition occurs, the over-allocated resources used by the over-allocated job can be recovered first, and the impact caused by resource competition can be reduced.

[0107] The cluster resource scheduling method provided in this exemplary embodiment can perform over-allocation between offline and offline, and can also perform over-allocation between offline and online. It can save an average of 30% of hardware resources, and the proportion of triggering resource competition and resource recovery is no more than 0.002%. It can be seen that the cluster resource scheduling method provided in this exemplary embodiment can significantly improve the utilization rate of hardware resources.

[0108] The cluster resource scheduling method provided in this embodiment receives the current over-allocated resources sent by the server nodes in the cluster, and schedules the non-running jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources of the server nodes, so that the server nodes use the current over-allocated resources to run the non-running jobs. Since the current over-allocated resources include unallocated resources and allocated but currently unused resources, the unused resources in the application resources of the normal jobs can be fully utilized to run the jobs with lower job execution parameter values, thereby solving the problem of resource waste caused by the difference between the application value and the usage value of the job and improving the resource utilization rate in the cluster.

[0109] Based on the above technical solution, the method may also optionally include: obtaining historical resource usage data of normal operations; determining the periodic change pattern of resource consumption of normal operations over time based on the historical resource usage data; and periodically adjusting the maximum allowable over-allocation value of the server node based on the periodic change pattern.

[0110] In many cases, the resources consumed by machines change periodically over time. Therefore, we can study the historical resource usage data of normal operations and determine the periodic change pattern of resource consumption of normal operations over time. That is, draw the resource operation curve of normal operations and make periodic adjustments to the maximum allowable over-allocation value according to the periodic change pattern. For example, if normal operations consume more resources in a certain time period, the maximum over-allocation value can be set to a smaller value. If normal operations consume less resources in a certain time period, the maximum allowable over-allocation value can be set to a larger value. This can make the determined over-allocation resources smoother and more stable.

[0111] Figure 2 is a flow chart of a cluster resource scheduling method according to an exemplary embodiment, wherein the cluster includes a scheduling center server and a server node, and the cluster resource scheduling method is used in the server node in the cluster, such as Figure 2 As shown, the cluster resource scheduling method includes the following steps.

[0112] In step S21, the used resources of the currently running normal job are obtained, where the normal job is a job running based on the allocated resources.

[0113] Among them, when each job is configured, the user who configures the job can also configure the job execution parameter value corresponding to the job according to the importance of the job. The job execution parameter value specifies the execution order of the job. The job with lower importance corresponds to a lower job execution parameter value and the execution order is later. Resources can be allocated to the jobs corresponding to the job execution parameter values ​​in order from high to low. The job execution parameter value can be, for example, a priority value. Normal jobs apply for resources based on the resource quota application and obtain the allocated resources corresponding to the resource quota application, and run based on the allocated resources. The resources may include memory and / or CPU.

[0114] The server node may monitor the normal jobs currently running on itself and obtain the used resources of the normal jobs.

[0115] An agent module may be provided in the server node, through which the used resources of the currently running normal job are obtained, and the current over-allocated resources are determined, and the current over-allocated resources are sent to the scheduling center server.

[0116] In step S22, current over-allocated resources are determined according to the used resources and the total allocatable resources, where the current over-allocated resources include unallocated resources and allocated but currently unused resources.

[0117] The dispatch center server preferentially allocates resources to jobs with higher job execution parameter values, so that these jobs run based on the allocated resources, and these jobs are treated as normal jobs. Since the requested resources for each job are generally more than the used resources, that is, the actual resources used by the job during runtime generally do not reach the quota of the requested resources, the allocated but currently unused resources can be used as over-allocated resources to run other jobs. That is, the server node can use the difference between the total allocatable resources and the used resources as the current idle resources based on the used resources and the total allocatable resources of the normal jobs currently running on itself, and determine the resources that can be over-allocated based on the idle resources as the current over-allocated resources. Among them, the current over-allocated resources may include memory and / or CPU.

[0118] In step S23, the current over-allocated resources are sent to a scheduling center server, which is used to schedule unrunning jobs whose job execution parameter values ​​are lower than those of normal jobs to the current over-allocated resources, wherein the job execution parameter values ​​are values ​​of the execution order corresponding to the jobs.

[0119] After determining the current over-allocated resources, the current over-allocated resources are sent to the scheduling center server. The scheduling center server can use the sum of the current over-allocated resources of each server node in the cluster as the cluster over-allocated resources, and allocate and schedule the non-running jobs whose job execution parameter values ​​are lower than those of normal jobs based on the cluster over-allocated resources, and schedule these non-running jobs to the current over-allocated resources of each server node.

[0120] In step S24, the unrunning job assigned by the scheduling center server is received, the unrunning job is used as an over-allocated job, and the over-allocated job is run using the current over-allocated resources.

[0121] The server node receives the unrunning job assigned by the scheduling center server, takes the unrunning job as an over-allocated job, and uses the current over-allocated resources to run the over-allocated job, thereby filling the idle resources of the server node.

[0122] The cluster resource scheduling method provided by this exemplary embodiment obtains the used resources of the currently running normal job, determines the current over-allocated resources based on the used resources and the total allocatable resources, and sends the current over-allocated resources to the scheduling center server. The scheduling center server schedules the non-running job with a job execution parameter value lower than the normal job in the current over-allocated resources, receives the non-running job allocated by the scheduling center server, regards the non-running job as an over-allocated job, and uses the current over-allocated resources to run the over-allocated job. Since the current over-allocated resources include unallocated resources and allocated but currently unused resources, the unused resources in the application resources of the normal job can be fully utilized to run the job with a lower job execution parameter value, thereby solving the problem of resource waste caused by the difference between the application value and the usage value of the job, and improving the resource utilization rate within the cluster.

[0123] Based on the above technical solution, the acquisition of the used resources of the currently running normal job may optionally include: obtaining the actual used resources of the normal job and obtaining the applied resource quota of the normal job; and taking the maximum value of the preset ratio of the applied resource quota and the actually used resources as the used resources of the normal job.

[0124] Considering that the newly started normal job has not yet fully entered the working mode, resource protection will be performed in the first 3 minutes. The maximum value of the preset ratio of the applied resource quota and the actual used resources can be used as the used resources of the currently running normal job. That is, if the preset ratio of the applied resource quota is greater than the actual used resources, the preset ratio of the applied resource quota is used as the used resources of the normal job. If the preset ratio of the applied resource quota is less than the actual used resources, the actual used resources are used as the used resources of the normal job. This can reduce frequent resource competition caused by the incomplete startup of the normal job. The preset ratio can be 50%, for example.

[0125] Based on the above technical solution, it may also optionally include: setting the memory overflow score of the normal job as the first score, and setting the memory overflow score of the over-allocated job running based on the current over-allocated resources as the second score, wherein the second score is greater than the first score; if a memory overflow occurs, stopping the running of the over-allocated job and releasing the current over-allocated resources occupied by the over-allocated job according to the first score and the second score.

[0126] The memory overflow score (oom_score) of the normal job is set lower than the memory overflow score of the over-allocated job, so that when the server node has a memory overflow, the operating system of the server node can preferentially kill the over-allocated job with a high memory overflow score according to the memory overflow score, that is, stop running the over-allocated job and release the current over-allocated resources occupied by the over-allocated job to ensure the stable execution of the normal job. For example, the first score can be set to 100 and the second score can be set to 400.

[0127] Figure 3 is a flow chart of a cluster resource scheduling method according to an exemplary embodiment, wherein the cluster includes a scheduling center server and a server node, and the cluster resource scheduling method is used in the server node in the cluster, such as Figure 3 As shown, the cluster resource scheduling method includes the following steps.

[0128] In step S31, the used resources of the currently running normal job are obtained, where the normal job is a job running based on the allocated resources.

[0129] In step S32, an over-allocation coefficient is determined according to the used resources and the total allocatable resources.

[0130] The over-allocation coefficient is a parameter for determining the current over-allocation resources. The ratio of used resources to total allocatable resources can be taken as the used ratio, and the difference between 1 and the used ratio can be taken as the over-allocation coefficient.

[0131] In an exemplary embodiment, the allocatable total resources also include a buffer zone; determining the over-allocation coefficient based on the used resources and the allocatable total resources includes: if the used resources are not 0, determining the over-allocation coefficient based on the used resources, the buffer zone and the allocatable total resources; if the used resources are 0, determining the over-allocation coefficient as a preset over-allocation coefficient.

[0132] The preset over-allocation coefficient may be determined according to demand, and may generally be less than 50%, for example, 30%.

[0133] Figure 4 is a schematic diagram of machine resources of a server node in an embodiment of the present disclosure, such as Figure 4As shown, the machine resources in the server node include allocated resources, unallocated resources and operating system resources. The allocated resources and unallocated resources constitute the total allocable resources. The allocated resources include the used resources and buffers of normal jobs. The currently unused resources and unallocated resources in the allocated resources can be used as over-allocated resources. The buffer can provide buffers for the currently running normal jobs, that is, if there are sudden resource requests without resource avoidance, the buffer can be used to process them. When determining the current over-allocated resources, the determination is made based on the used resources, buffers and the total allocable resources, that is, the determination is made based on the over-allocated resources.

[0134] When a normal job is running on a server node, that is, when the used resources are not 0, the over-allocation coefficient is determined based on the used resources, buffer, and total allocatable resources. If more resources are used, the over-allocation coefficient is smaller, and if less resources are used, the over-allocation coefficient is larger. If the used resources are not 0, the over-allocation coefficient is determined to be the preset over-allocation coefficient. The preset over-allocation coefficient is smaller, which can prevent a large number of jobs from being instantly dispatched to the same server node to trigger resource competition and then recycling, and can achieve smooth loading of over-allocated jobs.

[0135] The determining of the over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources includes: determining the ratio of the sum of the used resources and the buffer zone to the total allocatable resources, and taking the difference between 1 and the ratio as the over-allocation coefficient.

[0136] Used resources and buffers are resources that cannot be used as over-allocated resources. Therefore, the ratio of the sum of used resources and buffers to the total allocatable resources is determined as the used ratio, and the difference between 1 and the used ratio is used as the over-allocation coefficient. This ensures that the over-allocation coefficient is smaller when the used resources are more, and larger when the used resources are less.

[0137] In step S33, the number of times the historical over-allocated resources are recovered within a preset time is determined, and the over-allocation attenuation coefficient is determined according to the number.

[0138] Among them, the recovery of over-allocated resources is carried out when resource competition occurs. That is, when there is a sudden resource request for a normal job, the over-allocated resources used by the over-allocated job must be recovered to ensure the normal operation of the normal job.

[0139] Determine the number of times historical over-allocated resources are recovered within a preset time, and when the number is large, the determined over-allocation attenuation coefficient is large, so that the over-allocation attenuation coefficient can be used to ensure the stability of the server node, so that the over-allocated resources can be recovered quickly and restored slowly.

[0140] In an exemplary embodiment, determining the over-allocation attenuation coefficient according to the number of times includes: determining the over-allocation attenuation coefficient according to the number of times according to the following formula:

[0141]

[0142] Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

[0143] The preset time can be set according to demand, for example, it can be 10 minutes, and t is the time from the last recovery of historical over-allocated resources to the current time, which is measured in minutes.

[0144] In the above formula of over-allocation attenuation coefficient, min(1,n*α1) is used to constrain the recovery of over-allocated resources, that is, when there is a burst resource request for normal operation, the over-allocated resources are recovered, and (1-t*α2) is used to constrain the recovery of over-allocated resources, that is, when the used resources used by normal operation are reduced, the over-allocated resources can be slowly increased. By making the first coefficient greater than the second coefficient, for example, α1 is 0.25 and α2 is 0.05, fast recovery and slow recovery of over-allocated resources can be achieved. Fast recovery can ensure the operation of normal operation, and slow recovery can slowly increase over-allocated resources to reserve resources for normal operation, so as to avoid scheduling a large number of operations to use over-allocated resources and trigger resource competition.

[0145] In step S34, the current over-allocation resource is determined according to the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient.

[0146] The maximum value of over-allocation allowed is a preset value, which can be adjusted by the dispatch center server according to the change pattern of resource consumption of normal operations over time. The change pattern can be learned by a machine learning model based on historical resource usage data of normal operations.

[0147] The product of the over-allocation coefficient, the maximum over-allocation value allowed, and the over-allocation attenuation coefficient is used as the current over-allocation resource.

[0148] In step S35, the current over-allocated resources are sent to a scheduling center server, and the scheduling center server is used to schedule the unrunning jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources.

[0149] In step S36, the unrunning job assigned by the scheduling center server is received, the unrunning job is used as an over-allocated job, and the over-allocated job is run using the current over-allocated resources.

[0150] In step S37, if a burst resource request of the normal operation is received, the buffer is used to process the burst resource request.

[0151] If a burst resource request from a normal job is received after the over-allocated job is run using the current over-allocated resources, the buffer is used first to process the burst resource request from the normal job, thereby reducing the risk of failure of the normal job.

[0152] In step S38, the running time of the over-allocated jobs is obtained, and the over-allocated jobs are sorted in order of the running time from shortest to longest.

[0153] When receiving a burst resource request from a normal job, since the job execution parameter value of the normal job is higher than that of the over-allocated job, it is necessary to avoid resources for the normal job, that is, it is necessary to reclaim the current over-allocated resources used by the over-allocated job. When reclaiming the current over-allocated resources, the current over-allocated resources used by the over-allocated job with the shortest running time are preferentially reclaimed to reduce the impact of resource recycling. Therefore, it is necessary to obtain the running time of the currently running over-allocated job and sort the over-allocated jobs in order of running time from shortest to longest.

[0154] In step S39, according to the order of the over-allocated jobs, the current over-allocated resources used by the over-allocated jobs are recovered in sequence until the recovered current over-allocated resources meet the resources requested by the burst resource request.

[0155] According to the order of the over-allocated jobs, the current over-allocated resources used by the over-allocated jobs are recovered in sequence, that is, the current over-allocated resources used by the over-allocated jobs with shorter running time are recovered first, and the recovery is stopped when the recovered current over-allocated resources meet the resources requested by the burst resource request.

[0156] When resource competition occurs, over-allocated resources will be gradually recovered until the over-allocated resources recovered by the server node meet the resources requested by the burst resource request. The recovery strategy will sort the over-allocated jobs from short to long according to the running time of the over-allocated jobs, and give priority to recovering over-allocated jobs with shorter execution time, which will have a smaller impact. After the recovery is completed, the over-allocation attenuation coefficient will be increased according to the number of resource recovery times to reduce the over-allocated resource value of the server node, thereby informing the scheduling center server that the current server node resources are already tight and do not schedule tasks for execution again.

[0157] In step S310, the recovered current over-allocated resources are used to process the burst resource request and release the buffer.

[0158] When the recovered current over-allocation resources meet the resources requested by the burst resource request, the burst resource request is processed using the recovered current over-allocation resources to release the buffer.

[0159] The cluster resource scheduling method provided by this exemplary embodiment determines the over-allocation coefficient based on the used resources and the total allocatable resources, determines the number of times the historical over-allocated resources are recovered within a preset time, and determines the over-allocation attenuation coefficient based on the number, determines the current over-allocated resources based on the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient, and controls the quota of the current over-allocated resources by the over-allocation coefficient and the over-allocation attenuation coefficient. When more resources are used, the over-allocation coefficient is smaller, so that the value of the current over-allocated resources is smaller; when less resources are used, the over-allocation coefficient is larger, so that the value of the current over-allocated resources is larger; when the number of times the historical over-allocated resources are recovered within the preset time is large, the over-allocation attenuation coefficient is small, so that the value of the current over-allocated resources is small, that is, when the number of times the over-allocated resources are recovered is large, the value of the over-allocated resources of the server node can be reduced to ensure the normal operation of operations on the server node.

[0160] Figure 5 is a block diagram of a cluster resource scheduling device according to an exemplary embodiment, the device is applied to a scheduling center server in a cluster, and refers to Figure 5 The device includes an over-allocated resource receiving module 51 and an over-allocated resource scheduling module 52.

[0161] The over-allocated resource receiving module 51 is configured to receive current over-allocated resources sent by a server node in the cluster, wherein the current over-allocated resources are determined by the server node based on the used resources and the total allocatable resources of the currently running normal job, including unallocated resources and allocated but currently unused resources, wherein the normal job is a job running based on the allocated resources;

[0162] The over-allocated resource scheduling module 52 is configured to schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the unrunning job, and the job execution parameter value is a value of the execution order corresponding to the job.

[0163] Optionally, the cluster resource scheduling device further includes:

[0164] A historical resource data acquisition module is configured to acquire historical resource usage data of normal operations;

[0165] A resource rule determination module is configured to determine a periodic change rule of consumption resources of a normal operation over time according to the historical resource usage data;

[0166] The allowed over-allocation value adjustment module is configured to periodically adjust the allowed over-allocation maximum value of the server node according to the periodic variation rule.

[0167] The cluster resource scheduling device provided by this exemplary embodiment receives the current over-allocated resources sent by the server nodes in the cluster through the over-allocated resource receiving module. The over-allocated resource scheduling module schedules the non-running jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources of the server nodes, so that the server nodes use the current over-allocated resources to run the non-running jobs. Since the current over-allocated resources include unallocated resources and allocated but currently unused resources, the unused resources in the application resources of the normal jobs can be fully utilized to run jobs with lower priority, thereby solving the problem of resource waste caused by the difference between the application value and the usage value of the job and improving the resource utilization rate in the cluster.

[0168] Figure 6 is a block diagram of a cluster resource scheduling device according to an exemplary embodiment, the device is applied to a server node in a cluster, and refers to Figure 6 The device includes a used resource acquisition module 61, an over-allocation resource determination module 62, an over-allocation resource sending module 63 and an over-allocation job running module 64.

[0169] The used resource acquisition module 61 is configured to acquire used resources of a currently running normal job, where the normal job is a job running based on allocated resources;

[0170] The over-allocated resource determination module 62 is configured to determine current over-allocated resources according to the used resources and the total allocatable resources, wherein the current over-allocated resources include unallocated resources and allocated but currently unused resources;

[0171] The over-allocated resource sending module 63 is configured to send the current over-allocated resources to a scheduling center server, and the scheduling center server is used to schedule the unrunning jobs whose job execution parameter values ​​are lower than those of the normal jobs to the current over-allocated resources, wherein the job execution parameter values ​​are values ​​corresponding to the execution order of the jobs;

[0172] The over-allocated job running module 64 is configured to receive unrunning jobs assigned by the scheduling center server, use the unrunning jobs as over-allocated jobs, and use the current over-allocated resources to run the over-allocated jobs.

[0173] Optionally, the over-allocated resource determination module includes:

[0174] an over-allocation coefficient determining unit, configured to determine an over-allocation coefficient according to the used resources and the total allocatable resources;

[0175] An over-allocation attenuation coefficient determination unit is configured to determine the number of times the historical over-allocation resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number;

[0176] The over-allocation resource determination unit is configured to determine the current over-allocation resource according to the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient.

[0177] Optionally, the total allocatable resources also include a buffer zone;

[0178] The over-allocation coefficient determination unit comprises:

[0179] A first over-allocation coefficient determination subunit is configured to determine an over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources if the used resources are not 0;

[0180] The second over-allocation coefficient determination subunit is configured to determine the over-allocation coefficient as a preset over-allocation coefficient if the used resource is 0.

[0181] Optionally, the first overmatching coefficient determination subunit is specifically configured as follows:

[0182] If the used resources are not 0, the ratio of the sum of the used resources and the buffer to the total allocatable resources is determined, and the difference between 1 and the ratio is used as the over-allocation coefficient.

[0183] Optionally, the cluster resource scheduling device further includes:

[0184] The burst request processing module is configured to use the buffer to process the burst resource request if a burst resource request of the normal operation is received.

[0185] Optionally, the cluster resource scheduling device further includes:

[0186] The over-allocated job sorting module is configured to obtain the running time of the over-allocated jobs and sort the over-allocated jobs in order of the running time from shortest to longest;

[0187] An over-allocated resource recovery module is configured to recover the current over-allocated resources used by the over-allocated jobs in sequence according to the order of the over-allocated jobs, until the recovered current over-allocated resources meet the resources requested by the burst resource request;

[0188] The buffer releasing module is configured to process the burst resource request using the recovered current over-allocated resources and release the buffer.

[0189] Optionally, the over-allocation attenuation coefficient determining unit is specifically configured as follows:

[0190] Determine the number of times that historical over-allocated resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number of times according to the following formula:

[0191]

[0192] Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

[0193] Optionally, the used resource acquisition module includes:

[0194] A resource acquisition unit is configured to acquire actually used resources of a normal operation and acquire an applied resource quota of a normal operation;

[0195] The used resource determination unit is configured to use the maximum value of the preset ratio of the applied resource quota and the actually used resources as the used resources for normal operation.

[0196] Optionally, the cluster resource scheduling device further includes:

[0197] a memory overflow score setting module, configured to set the memory overflow score of the normal job as a first score, and set the memory overflow score of the over-allocated job running based on the current over-allocated resources as a second score, wherein the second score is greater than the first score;

[0198] The memory overflow processing module is configured to stop running the over-allocated job and release the current over-allocated resources occupied by the over-allocated job according to the first score and the second score if memory overflow occurs.

[0199] The cluster resource scheduling device provided by this exemplary embodiment obtains the used resources of the currently running normal job through the used resource acquisition module, the over-allocated resource determination module determines the current over-allocated resources according to the used resources and the total allocatable resources, the over-allocated resource sending module sends the current over-allocated resources to the scheduling center server, the scheduling center server schedules the non-running job with a job execution parameter value lower than the normal job in the current over-allocated resources, the over-allocated job running module receives the non-running job allocated by the scheduling center server, regards the non-running job as an over-allocated job, and uses the current over-allocated resources to run the over-allocated job. Since the current over-allocated resources include unallocated resources and allocated but currently unused resources, the unused resources in the application resources of the normal job can be fully utilized to run the job with a lower job execution parameter value, thereby solving the problem of resource waste caused by the difference between the application value and the usage value of the job and improving the resource utilization rate within the cluster.

[0200] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0201] Figure 7is a block diagram of a server according to an exemplary embodiment. Figure 7 The server 700 includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions, such as an application, that can be executed by the processing component 722. The application stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform the above-mentioned cluster resource scheduling method.

[0202] The server 700 may also include a power supply component 726 configured to perform power management of the server 700, a wired or wireless network interface 750 configured to connect the server 700 to a network, and an input / output (I / O) interface 758. The server 700 may operate based on an operating system stored in the memory 732, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0203] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 732 including instructions, and the instructions can be executed by the processing component 722 of the server 700 to complete the above cluster resource scheduling method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0204] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0205] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A cluster resource scheduling method, characterized in that: Applied to the dispatch center server in the cluster, including: Receive current over-allocated resources sent by server nodes in the cluster, wherein the current over-allocated resources are determined by the cluster server nodes according to the used resources and the total allocatable resources of the currently running normal job, and the current over-allocated resources are determined according to the over-allocation coefficient, the maximum over-allocation value allowed, and the over-allocation attenuation coefficient; the over-allocation coefficient is used to determine a parameter of the current over-allocated resources; the maximum over-allocation value allowed is a preset value; the over-allocation attenuation coefficient is determined by the cluster server nodes according to the number of times the historical over-allocated resources are recovered within a preset time, and the over-allocation attenuation coefficient is determined according to the number; the current over-allocated resources include unallocated resources and allocated but currently unused resources, and the normal job is a job running based on allocated resources; Schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the unrunning job, and the job execution parameter value is a value of an execution order corresponding to the job.

2. The cluster resource scheduling method according to claim 1, characterized in that: Also includes: Get historical resource usage data for normal operations; Determine the periodic change pattern of the consumption resources of normal operation over time according to the historical resource usage data; According to the periodic variation rule, the maximum over-allocation value allowed for the server node is periodically adjusted.

3. A cluster resource scheduling method, characterized in that: Applied to server nodes in the cluster, including: Obtaining used resources of a currently running normal job, where the normal job is a job running based on allocated resources; Determine an over-allocation coefficient based on the used resources and the total allocatable resources; determine the number of times the historical over-allocation resources are recovered within a preset time, and determine the over-allocation attenuation coefficient based on the number; determine the current over-allocation resources based on the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient; the current over-allocation resources include unallocated resources and allocated but currently unused resources; Sending the current over-allocated resources to a scheduling center server, the scheduling center server being used to schedule unrunning jobs whose job execution parameter values ​​are lower than those of normal jobs to the current over-allocated resources, wherein the job execution parameter values ​​are values ​​corresponding to the execution order of the jobs; The unrunning job assigned by the scheduling center server is received, the unrunning job is used as an over-allocated job, and the over-allocated job is run using the current over-allocated resources.

4. The cluster resource scheduling method according to claim 3, characterized in that: The total allocatable resources also include a buffer zone; The determining of the over-allocation coefficient according to the used resources and the total allocatable resources includes: If the used resources are not 0, determining an over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources; If the used resource is 0, the over-allocation coefficient is determined to be a preset over-allocation coefficient.

5. The cluster resource scheduling method according to claim 4, characterized in that: The determining of the over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources includes: The ratio of the sum of the used resources and the buffer to the total allocatable resources is determined, and the difference between 1 and the ratio is used as the over-allocation coefficient.

6. The cluster resource scheduling method according to claim 4, characterized in that: After using the current over-allocated resources to run the over-allocated job, the method further includes: If a burst resource request of the normal operation is received, the buffer is used to process the burst resource request.

7. The cluster resource scheduling method according to claim 6, characterized in that: When the buffer is used to process the burst resource request, the method further includes: Obtaining the running time of the over-allocated jobs, and sorting the over-allocated jobs in order of the running time from shortest to longest; According to the order of the over-allocated jobs, the current over-allocated resources used by the over-allocated jobs are recovered in sequence until the recovered current over-allocated resources meet the resources requested by the burst resource request; The recovered current over-allocated resources are used to process the burst resource request and release the buffer.

8. The cluster resource scheduling method according to claim 5, characterized in that: The determining of the over-allocation attenuation coefficient according to the number of times includes: According to the times, the over-allocation attenuation coefficient is determined according to the following formula: Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

9. The cluster resource scheduling method according to any one of claims 3 to 8, characterized in that: The obtaining of the used resources of the currently running normal job includes: Get the actual resources used by normal operations and the resource quotas applied for normal operations; The maximum value of the preset ratio of the applied resource quota and the actually used resources is used as the used resources for normal operation.

10. The cluster resource scheduling method according to any one of claims 3 to 8, characterized in that: Also includes: Setting a memory overflow score of the normal job as a first score, and setting a memory overflow score of the over-allocated job running based on the current over-allocated resources as a second score, wherein the second score is greater than the first score; If memory overflow occurs, the over-allocated job is stopped according to the first score and the second score, and the current over-allocated resources occupied by the over-allocated job are released.

11. A cluster resource scheduling device, characterized in that: Applied to the dispatch center server in the cluster, including: An over-allocated resource receiving module is configured to receive current over-allocated resources sent by a server node in a cluster, wherein the current over-allocated resources are determined by the cluster server node based on the used resources and the total allocatable resources of the currently running normal job, and the current over-allocated resources are determined based on the over-allocation coefficient, the maximum over-allocation value allowed, and the over-allocation attenuation coefficient; the over-allocation coefficient is used to determine a parameter of the current over-allocated resource; the maximum over-allocation value allowed is a preset value; the over-allocation attenuation coefficient is determined by the cluster server node to determine the number of times the historical over-allocated resources are recovered within a preset time, and the over-allocation attenuation coefficient is determined based on the number of times; the current over-allocated resources include unallocated resources and allocated but currently unused resources, and the normal job is a job running based on allocated resources; The over-allocated resource scheduling module is configured to schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources of the server node, so that the server node uses the current over-allocated resources to run the unrunning job, and the job execution parameter value is the value of the execution order corresponding to the job.

12. The cluster resource scheduling device according to claim 11, characterized in that: The cluster resource scheduling device also includes: A historical resource data acquisition module is configured to acquire historical resource usage data of normal operations; A resource rule determination module is configured to determine a periodic change rule of consumption resources of a normal operation over time according to the historical resource usage data; The allowed over-allocation value adjustment module is configured to periodically adjust the allowed over-allocation maximum value of the server node according to the periodic variation rule.

13. A cluster resource scheduling device, characterized in that: Applied to server nodes in the cluster, including: A used resource acquisition module is configured to acquire used resources of a currently running normal job, where the normal job is a job running based on allocated resources; The over-allocation resource determination module is configured to determine an over-allocation coefficient according to the used resources and the total allocatable resources; determine the number of times the historical over-allocation resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number; determine the current over-allocation resources according to the over-allocation coefficient, the maximum allowed over-allocation value and the over-allocation attenuation coefficient; the current over-allocation resources include unallocated resources and allocated but currently unused resources; an over-allocated resource sending module, configured to send the current over-allocated resources to a scheduling center server, the scheduling center server being used to schedule an unrunning job whose job execution parameter value is lower than that of the normal job to the current over-allocated resources, wherein the job execution parameter value is a value corresponding to an execution order of the job; The over-allocated job running module is configured to receive the unrunning job assigned by the scheduling center server, use the unrunning job as the over-allocated job, and use the current over-allocated resources to run the over-allocated job.

14. The cluster resource scheduling device according to claim 13, characterized in that: The total allocatable resources also include a buffer zone; The over-allocation coefficient determination unit comprises: A first over-allocation coefficient determination subunit is configured to determine an over-allocation coefficient according to the used resources, the buffer zone and the total allocatable resources if the used resources are not 0; The second over-allocation coefficient determination subunit is configured to determine the over-allocation coefficient as a preset over-allocation coefficient if the used resource is 0.

15. The cluster resource scheduling device according to claim 14, characterized in that: The first overmatching coefficient determination subunit is specifically configured as follows: If the used resources are not 0, the ratio of the sum of the used resources and the buffer to the total allocatable resources is determined, and the difference between 1 and the ratio is used as the over-allocation coefficient.

16. The cluster resource scheduling device according to claim 14, characterized in that: The cluster resource scheduling device also includes: The burst request processing module is configured to use the buffer to process the burst resource request if a burst resource request of the normal operation is received.

17. The cluster resource scheduling device according to claim 16, characterized in that: The cluster resource scheduling device also includes: The over-allocated job sorting module is configured to obtain the running time of the over-allocated jobs and sort the over-allocated jobs in order of the running time from shortest to longest; An over-allocated resource recovery module is configured to recover the current over-allocated resources used by the over-allocated jobs in sequence according to the order of the over-allocated jobs, until the recovered current over-allocated resources meet the resources requested by the burst resource request; The buffer releasing module is configured to process the burst resource request using the recovered current over-allocated resources and release the buffer.

18. The cluster resource scheduling device according to claim 14, characterized in that: The over-allocation attenuation coefficient determination unit is specifically configured as follows: Determine the number of times that historical over-allocated resources are recovered within a preset time, and determine the over-allocation attenuation coefficient according to the number of times according to the following formula: Among them, β is the over-allocation attenuation coefficient, n is the number of times the historical over-allocation resources are recovered within the preset time, t is the length of time from the last recovery of the historical over-allocation resources to the current time, α1 is the first coefficient, α2 is the second coefficient, and the first coefficient is greater than the second coefficient.

19. The cluster resource scheduling device according to any one of claims 13 to 18, characterized in that: The used resource acquisition module includes: A resource acquisition unit is configured to acquire actually used resources of a normal operation and acquire an applied resource quota of a normal operation; The used resource determination unit is configured to use the maximum value of the preset ratio of the applied resource quota and the actually used resources as the used resources for normal operation.

20. The cluster resource scheduling device according to any one of claims 13 to 18, characterized in that: The cluster resource scheduling device also includes: a memory overflow score setting module, configured to set the memory overflow score of the normal job as a first score, and set the memory overflow score of the over-allocated job running based on the current over-allocated resources as a second score, wherein the second score is greater than the first score; The memory overflow processing module is configured to stop running the over-allocated job and release the current over-allocated resources occupied by the over-allocated job according to the first score and the second score if memory overflow occurs.

21. A server, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instruction to implement the cluster resource scheduling method according to claim 1 or 2 or the cluster resource scheduling method according to any one of claims 3 to 10.

22. A storage medium, when instructions in the storage medium are executed by a processor of a server, enables the server to execute the cluster resource scheduling method according to claim 1 or 2 or the cluster resource scheduling method according to any one of claims 3 to 10.

Images