Task scheduling method, system and device based on resource coordination allowance and medium

Abstract

The application discloses a task scheduling method and system based on resource coordination quota, a device and a medium. The method comprises the following steps: screening a plurality of high-pressure resource utilization rates greater than a preset resource borrowing threshold from a plurality of historical resource utilization rates of each preset node, and determining a plurality of high-pressure continuous resource utilization rates; a preset node with a number of high-pressure continuous resource utilization rates greater than a preset number is regarded as a resource borrowing node; predicting the resource coordination quota of the resource borrowing node based on the current available resource amount and the current task amount of the resource borrowing node; determining a resource lending node based on the current idle resource amount of each preset node; increasing the resource domination authority range of the resource borrowing node and reducing the resource domination authority range of the resource lending node according to the resource coordination quota; and scheduling tasks based on the corresponding resource domination authority range of each preset node, which can improve the timeliness of task scheduling, the flexibility of resource allocation, and the stability of task processing.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a task scheduling method, system, device, and medium based on resource collaboration quota. Background Technology

[0002] With the rapid development of cloud computing, big data, and artificial intelligence technologies, the scale of distributed computing clusters is constantly expanding, and more and more data processing tasks need to be executed in parallel in cluster environments composed of a large number of computing nodes. In distributed systems, task scheduling is a key factor affecting overall performance and resource utilization. How to achieve efficient hardware resource coordination among nodes to realize effective scheduling of data processing tasks has become a technical problem that urgently needs to be solved in this field.

[0003] In existing technologies, task scheduling primarily involves pre-allocating fixed hardware resources to each computing node in the cluster before system startup or task processing begins. When a task is scheduled to a node, that node executes the corresponding computation based on the pre-allocated resource quota, or assigns newly received tasks to the node with the lowest current load, or migrates some running tasks to other nodes when a node's load is too high, thereby distributing the load pressure. However, the task load on different nodes changes dynamically over time, and existing technologies require task scheduling when nodes are already overloaded, resulting in inflexible resource configuration, untimely task scheduling, and poor task processing stability. Summary of the Invention

[0004] The purpose of this application is to provide a task scheduling method, system, device, and medium based on resource collaboration quotas. This addresses the problems of inflexible resource allocation, untimely task scheduling, and poor task processing stability in existing technologies. By predicting the resource collaboration quota of a resource borrowing node based on its current available resources and current task load, and using the remaining node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node, resource collaboration can be performed in advance based on the actual task load and current available resources of each node before node overload, improving the timeliness of task scheduling. By increasing the current available resources of the resource borrowing node according to the resource collaboration quota and decreasing the current idle resources of the resource lending node according to the resource collaboration quota for task scheduling, the transfer of resource ownership can be achieved based on the resource borrowing and lending mechanism without involving hardware relocation. This changes the upper limit of resources that the corresponding node can control, improving the flexibility of resource allocation and the stability of task processing.

[0005] In a first aspect, embodiments of this application provide a task scheduling method based on resource collaboration quotas, the method comprising: Obtain the current available resources, current task volume, current resource utilization rate of each preset node, as well as multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaboration quota, and the initial value of the collaboration quota is 0. Select multiple high-voltage resource utilization rates from the multiple historical resource utilization rates of each preset node that are greater than the preset resource borrowing threshold, and determine multiple high-voltage continuous resource utilization rates from the multiple high-voltage resource utilization rates according to the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are used as resource borrowing nodes. The resource collaboration limit of the resource borrowing node is predicted based on the current available resources and current task volume of the resource borrowing node. The current idle resource quantity of each preset node is determined based on the current available resource quantity of each preset node and multiple historical resource utilization rates, and the preset node with the largest current idle resource quantity is taken as the resource lending node corresponding to the resource borrowing node. The resource borrowing node's collaboration limit is increased according to the resource collaboration limit to expand its resource control authority. The resource lending node's collaboration limit is decreased according to the resource collaboration limit to reduce its resource control authority. Each preset node schedules tasks based on its corresponding resource control authority.

[0006] Furthermore, the currently available resources include the preset resources; Before selecting the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node, the method also includes: Filter out the resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes; The resource borrowing threshold of the resource borrowing node is determined based on the first preset ratio and the preset resource amount of the resource borrowing node. Based on the second preset ratio and the preset resource amount of each remaining node, the resource lending threshold of each remaining node is determined, and the sum of the resource lending thresholds of all remaining nodes is calculated to obtain the current lending threshold. If the resource collaboration limit is greater than the current available lending threshold, terminate the operation of increasing the current available resource amount of the resource borrowing node.

[0007] Furthermore, the collaboration limit of resource borrowing nodes is increased according to the resource collaboration limit to expand the scope of resource allocation authority of resource borrowing nodes, and the collaboration limit of resource lending nodes is decreased according to the resource collaboration limit to reduce the scope of resource allocation authority of resource lending nodes, including: The sum of the resource collaboration quota and the collaboration quota of the resource borrowing node is calculated to obtain the first collaboration quota update result. The sum of the preset resource amount of the resource borrowing node and the first collaboration quota update result is calculated to obtain the increase result of the current available resource amount of the resource borrowing node. The increase result of the current available resource amount is the resource allocation authority range of the resource borrowing node. The negative of the resource collaboration quota is calculated and summed with the collaboration quota of the resource lending node to obtain the second collaboration quota update result. The preset resource amount of the resource lending node is calculated and summed with the second collaboration quota update result to obtain the reduction result of the current idle resource amount of the resource lending node. The reduction result of the current idle resource amount of the resource lending node is the resource allocation authority range of the resource lending node.

[0008] Furthermore, the number of resource lending nodes is multiple; The preset node with the largest current idle resource volume is selected as the resource lending node corresponding to the resource borrowing node, including: Sort the current available resources of each remaining node in descending order; The current idle resource amount of each remaining node is accumulated according to the sorting result until the accumulated calculation result is greater than or equal to the resource sharing quota. The multiple remaining nodes corresponding to the accumulated calculation result are then used as multiple resource lending nodes corresponding to the resource borrowing node.

[0009] Furthermore, the currently available resources include the preset resources; After filtering through multiple historical resource utilization rates of each preset node that exceed a preset resource borrowing threshold, the method further includes: Filter out multiple low-voltage resource utilization rates from multiple historical resource utilization rates of each preset node that are less than or equal to a preset resource return threshold, and identify the timestamp corresponding to each low-voltage resource utilization rate; Based on each timestamp, determine multiple low-voltage continuous resource utilization rates among the multiple low-voltage resource utilization rates of each preset node. If the number of multiple low-voltage continuous resource utilization rates is greater than the preset number, the corresponding preset node will be used as the resource return node. The difference between the current available resources of the resource return node and the preset resources is calculated to obtain the pending return coordination quota of the resource return node. The resource receiving node corresponding to the pending return coordination quota among multiple preset nodes is determined so as to increase the current available resources of the resource receiving node according to the pending return coordination quota.

[0010] Furthermore, there are multiple resource borrowing nodes; Identify the resource receiving node corresponding to the collaborative amount to be repaid from multiple preset nodes, including: Compare the size relationships of multiple resource collaboration limits corresponding to multiple resource borrowing nodes, and select the resource borrowing node with the largest resource collaboration limit as the resource receiving node corresponding to the collaboration limit to be repaid. Alternatively, identify the original resource lending node in the resource borrowing record of the obtained resource return node, and use the original resource lending node as the resource receiving node corresponding to the collaborative amount to be returned.

[0011] Secondly, embodiments of this application provide a task scheduling system based on resource collaboration quotas, the system comprising: The data acquisition module is used to acquire the current available resources, current task volume, current resource utilization rate of each preset node, as well as multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaborative quota, and the initial value of the collaborative quota is 0. The borrowing node determination module is used to filter multiple high-voltage resource utilization rates that are greater than a preset resource borrowing threshold from multiple historical resource utilization rates of each preset node, and to determine multiple high-voltage continuous resource utilization rates from multiple high-voltage resource utilization rates based on the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are used as resource borrowing nodes. The resource collaboration quota prediction module is used to predict the resource collaboration quota of the resource borrowing node based on the current available resources and current task volume of the resource borrowing node. The lending node determination module is used to determine the current idle resource quantity of each preset node based on the current available resource quantity of each preset node and multiple historical resource utilization rates, and to take the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node. The task scheduling module is used to increase the collaboration limit of resource borrowing nodes according to the resource collaboration limit, so as to increase the resource control authority range of resource borrowing nodes, and to decrease the collaboration limit of resource lending nodes according to the resource collaboration limit, so as to decrease the resource control authority range of resource lending nodes. Each preset node performs task scheduling based on the corresponding resource control authority range.

[0012] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0013] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0014] Fifthly, embodiments of this application also provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the method described in the first aspect.

[0015] In this embodiment, the current available resources, current task volume, and current resource utilization rate of each preset node are obtained, along with multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaborative quota, with an initial value of 0. Multiple high-voltage resource utilization rates greater than a preset resource borrowing threshold are selected from the multiple historical resource utilization rates of each preset node. Multiple high-voltage continuous resource utilization rates are determined based on the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are designated as resource borrowing nodes. Based on the current... The resource collaboration limit of the resource borrowing node is predicted based on the available resource quantity and the current task quantity; the current idle resource quantity of each preset node is determined based on the current available resource quantity of each preset node and multiple historical resource utilization rates, and the preset node with the largest current idle resource quantity is designated as the resource lending node corresponding to the resource borrowing node; the collaboration limit of the resource borrowing node is increased according to the resource collaboration limit to increase the resource allocation authority range of the resource borrowing node, and the collaboration limit of the resource lending node is decreased according to the resource collaboration limit to decrease the resource allocation authority range of the resource lending node. Each preset node performs task scheduling based on the corresponding resource allocation authority range. The above-described task scheduling method based on resource collaboration quota solves the problems of inflexible resource allocation, untimely task scheduling, and poor task processing stability in existing technologies. By predicting the resource collaboration quota of the resource borrowing node based on its current available resources and current task load, and using the remaining node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node, resource collaboration can be performed in advance based on the actual task load and current available resources of each node before node overload, improving the timeliness of task scheduling. By increasing the current available resources of the resource borrowing node according to the resource collaboration quota and decreasing the current idle resources of the resource lending node according to the resource collaboration quota for task scheduling, the transfer of resource ownership can be achieved based on the resource borrowing and lending mechanism without involving hardware movement, thereby changing the upper limit of the resources that the corresponding node can control, improving the flexibility of resource allocation and the stability of task processing. Attached Figure Description

[0016] Figure 1 This is a flowchart of a task scheduling method based on resource collaboration quota provided in an embodiment of this application; Figure 2 This is a flowchart of another task scheduling method based on resource collaboration quota provided in the embodiments of this application; Figure 3 This is a flowchart of a method for collaboratively updating available resource quantities provided in an embodiment of this application; Figure 4 This is a structural block diagram of a task scheduling system based on resource collaboration quota provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a task scheduling device based on resource collaboration quota provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0018] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] First, this solution can be used in scenarios involving task processing within distributed systems, particularly in multi-node distributed systems where task scheduling is crucial to ensure overall system performance and resource utilization. By predicting the resource coordination limit of the resource-borrowing node based on its current available resources and task load, and designating the remaining node with the largest available idle resources as the corresponding resource-lending node, resource coordination can be preemptively implemented based on the actual task load and available resources of each node before node overload, improving the timeliness of task scheduling. Increasing the available resources of the resource-borrowing node according to the resource coordination limit, and decreasing the available idle resources of the resource-lending node according to the same limit, allows for task scheduling by the resource-borrowing node. This achieves the transfer of resource ownership without involving hardware relocation, based on the resource borrowing and lending mechanism, thereby changing the upper limit of resources that the corresponding node can control, improving the flexibility of resource allocation and the stability of task processing. Based on these use cases, it is understandable that the implementing entity of this solution can be a server capable of data processing, resource coordination, and task scheduling.

[0021] The task scheduling method, system, device, and medium based on resource collaboration quota provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0022] Figure 1 This is a flowchart illustrating a task scheduling method based on resource sharing quotas provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following: S101, obtain the current available resources, current task volume, current resource utilization rate of each preset node, and multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaboration quota, and the initial value of the collaboration quota is 0.

[0023] In this context, a preset node can be a pre-registered and independent computing unit within a distributed cluster. Each node has its own independent hardware resource quota and can be uniformly scheduled and managed by the system. The currently available resources can be the total amount of hardware resources that a preset node can actually control at the current moment, such as the number of CPU cores, bytes of memory, or GPU computing power. The currently available resources equal the sum of the node's preset resources (fixed resources allocated at startup) and its current collaborative quota (virtual resource changes generated through borrowing or lending). This value directly determines the maximum resource limit that the node can currently handle. The current task load can be the total number of algorithm tasks currently being executed or queued for execution on a preset node, or the cumulative total resource requirements of those tasks. This indicator reflects the node's immediate load pressure. The current resource utilization rate is the percentage of hardware resources used by a preset node at the current moment relative to its currently available resources, calculated as: (Currently used resources / Current available resources) × 100%. The current resource utilization rate directly reflects the node's current resource scarcity. Historical resource utilization can be the resource utilization rate recorded and reported by each preset node before the current time according to a preset period. The preset resource borrowing threshold can be the minimum resource utilization rate that a node can trigger resource borrowing. When multiple consecutive historical resource utilization rates of a node exceed the preset resource borrowing threshold, it indicates that the node is under continuous high pressure and is eligible for resource borrowing.

[0024] In one embodiment, the specific method for obtaining the current available resources, current task load, current resource utilization rate, and multiple historical resource utilization rates collected according to a preset period for each preset node, and comparing the relationship between the multiple historical resource utilization rates of each preset node and a preset resource borrowing threshold, can be as follows: Obtain the real-time available resources, task load, and resource utilization rate continuously reported by each preset node according to a preset period through an interaction interface. Determine the current available resources, current task load, current resource utilization rate, and multiple historical resource utilization rates of each preset node at the current moment based on the timestamp corresponding to the current time and the available resources, task load, and resource utilization rate. Then, obtain the comparison result of the relationship between the multiple historical resource utilization rates of each preset node and the preset resource borrowing threshold by comparing whether the average of the multiple historical resource utilization rates of each preset node is greater than the preset resource borrowing threshold, or by comparing whether the corresponding historical resource utilization rates of each preset node are greater than the preset resource borrowing threshold.

[0025] In one embodiment, the currently available resource amount includes a preset resource amount. After comparing the relationship between the multiple historical resource utilization rates of each preset node and the preset resource borrowing threshold, the method further includes: filtering multiple low-voltage resource utilization rates among the multiple historical resource utilization rates of each preset node that are less than or equal to the preset resource return threshold, and identifying the timestamp corresponding to each low-voltage resource utilization rate; determining multiple low-voltage continuous resource utilization rates among the multiple low-voltage resource utilization rates of each preset node based on each timestamp; if the number of multiple low-voltage continuous resource utilization rates is greater than a preset number, designating the corresponding preset node as a resource return node; calculating the difference between the current available resource amount of the resource return node and the preset resource amount to obtain the pending return collaborative quota of the resource return node, and determining the resource receiving node among the multiple preset nodes corresponding to the pending return collaborative quota, so as to increase the current available resource amount of the resource receiving node according to the pending return collaborative quota.

[0026] The preset resource amount can be a fixed amount of hardware resources pre-allocated to each preset node when the system starts or before the task begins. It is the baseline resource configuration of the node and does not change with borrowing or lending operations in the resource coordination mechanism. The preset resource return threshold can be the maximum resource utilization rate of the preset node when it needs to return coordinated resources. Low-voltage resource utilization rate can be a preset node whose historical resource utilization rate is lower than the preset resource return threshold. Low-voltage continuous resource utilization rate can be multiple consecutive low-voltage resource utilization rates on the time axis. Resource return node can be a node among multiple preset nodes whose historical resource utilization rate has been consistently lower than the preset resource return threshold. The amount of coordinated resources to be returned can be the amount of resources that the resource return node currently needs to return to the cluster. This value is equal to the difference between the node's current available resources and its preset resource amount. The resource receiving node can be a node among multiple preset nodes used to receive resources returned by the resource return node.

[0027] In one embodiment, the specific implementation of filtering multiple low-voltage resource utilization rates that are less than or equal to a preset resource return threshold from multiple historical resource utilization rates of each preset node, and identifying the timestamp corresponding to each low-voltage resource utilization rate, can be as follows: Each historical resource utilization rate of each preset node is compared with the preset resource return threshold. All historical resource utilization rates with values ​​lower than the preset resource return threshold are filtered from all historical resource utilization rates of that node, and these historical resource utilization rates are taken as low-voltage resource utilization rates. Simultaneously, the timestamp corresponding to each low-voltage resource utilization rate is extracted. The specific implementation of determining multiple consecutive low-voltage resource utilization rates from multiple low-voltage resource utilization rates of each preset node based on each timestamp, and identifying the corresponding preset node as a resource return node when the number of consecutive low-voltage resource utilization rates is greater than a preset number, can be as follows: Based on the timestamps corresponding to each low-voltage resource utilization rate, each low-voltage resource utilization rate is sorted in chronological order. The sorted low-voltage resource utilization rate sequence is traversed to identify multiple consecutive low-voltage resource utilization rates that are consecutively adjacent on the time axis, and the number of consecutive low-voltage resource utilization rates is counted. The process compares the utilization rates of multiple low-voltage continuous resources with a preset quantity, identifying preset nodes whose utilization rates exceed the preset quantity as resource return nodes. The difference between the current available resources of the resource return node and the preset resource quantity is calculated to obtain the pending return coordination quota for the resource return node. The resource receiving node corresponding to the pending return coordination quota is then identified among the preset nodes. The specific implementation of increasing the current available resources of the resource receiving node according to the pending return coordination quota can be as follows: Calculate the difference between the current available resources and the preset resource quantity, and use this difference as the pending return coordination quota for the resource return node. When the difference is greater than zero, it indicates that the node still holds borrowed resources and needs to return the portion exceeding the preset resource quantity to the cluster; when the difference is less than or equal to zero, it indicates that the node currently has no resources to return, and no return operation is performed. Resource borrowing nodes among the preset nodes are identified, designated as resource receiving nodes, and the pending return coordination quota is added to the current available resources of the resource receiving nodes.

[0028] This solution identifies nodes with consistently low historical resource utilization rates as resource return nodes and calculates the outstanding collaborative amount for each node before returning the resources. This ensures the automatic recycling and redistribution of idle resources, avoids resource waste caused by borrowing nodes occupying excess resources for extended periods, and further improves resource utilization and collaboration.

[0029] In one embodiment, there are multiple resource borrowing nodes; determining the resource receiving node corresponding to the collaborative amount to be repaid among multiple preset nodes includes: comparing the size relationship of multiple resource collaborative amounts corresponding to multiple resource borrowing nodes, and taking the resource borrowing node with the largest resource collaborative amount as the resource receiving node corresponding to the collaborative amount to be repaid; or, identifying the original resource lending node in the resource borrowing record of the obtained resource return node, and taking the original resource lending node as the resource receiving node corresponding to the collaborative amount to be repaid.

[0030] In one embodiment, the specific implementation of comparing the size relationship of multiple resource collaboration quotas corresponding to multiple resource borrowing nodes and selecting the resource borrowing node with the largest resource collaboration quota as the resource receiving node corresponding to the collaboration quota to be repaid can be as follows: When there are multiple resource borrowing nodes in the cluster that are determined by the system to be under high load, since these nodes each have different resource collaboration quotas (i.e., each needs to borrow different amounts of resources), the size relationship of multiple resource collaboration quotas corresponding to multiple resource borrowing nodes can be compared, and the borrowing demand of multiple resource borrowing nodes can be sorted according to the size relationship. The resource borrowing node with the largest resource collaboration quota is determined as the node with the largest borrowing demand and is selected as the resource receiving node corresponding to the collaboration quota to be repaid. The specific implementation of identifying the original resource lending node in the resource borrowing record of the obtained resource return node and using the original resource lending node as the resource receiving node corresponding to the collaborative quota to be returned can be as follows: query the historical resource borrowing record associated with the resource return node, extract the node identifier of the original resource lending node from the record, that is, from which computing node the resource previously borrowed by the node came, determine the original resource lending node as the resource receiving node corresponding to the collaborative quota to be returned, and directly return the resource to be returned to the original resource lending node to restore the amount of available resources that the node previously lost due to borrowing resources.

[0031] This solution compares the resource collaboration limits of multiple resource borrowing nodes and identifies the resource borrowing node with the largest resource collaboration limit as the resource receiving node. Alternatively, it queries the historical borrowing records of resource returning nodes to identify the original resource lending node as the resource receiving node. This ensures that returned resources are allocated to the most needed node or returned to the original lending node, thereby improving the rationality of resource collaboration and the efficiency of task scheduling.

[0032] S102, filter out multiple high-voltage resource utilization rates from multiple historical resource utilization rates of each preset node that are greater than the preset resource borrowing threshold, and determine multiple high-voltage continuous resource utilization rates from the multiple high-voltage resource utilization rates according to the timestamps corresponding to each high-voltage resource utilization rate, and select preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than the preset number as resource borrowing nodes.

[0033] Among these, the high-voltage resource utilization rate can be the resource utilization rate among multiple historical resource utilization rates of each preset node that is greater than a preset resource borrowing threshold. The high-voltage continuous resource utilization rate can be multiple consecutive high-voltage resource utilization rates that form an adjacent sequence on the timeline among multiple historical resource utilization rates of each preset node. The preset quantity can be a pre-set minimum number of high-voltage continuous resource utilization rates that a preset node must have under continuous high-voltage conditions. The resource borrowing node can be a node among multiple preset nodes whose historical resource utilization rate is consistently greater than a preset resource borrowing threshold.

[0034] In one embodiment, the specific method for filtering multiple high-voltage resource utilization rates that are greater than a preset resource borrowing threshold from multiple historical resource utilization rates of each preset node can be as follows: Determine the relationship between each historical resource utilization rate of each preset node and the preset resource borrowing threshold based on the comparison results; filter historical resource utilization rates that are greater than the preset resource borrowing threshold to obtain multiple high-voltage resource utilization rates for each preset node. The specific method for determining multiple continuous high-voltage resource utilization rates from the multiple high-voltage resource utilization rates based on the timestamps corresponding to each high-voltage resource utilization rate, and for selecting preset nodes with a number of multiple continuous high-voltage resource utilization rates greater than a preset number as resource borrowing nodes, can be as follows: Since each preset node carries a resource utilization timestamp corresponding to each resource utilization rate when reporting resource utilization, the timestamp corresponding to each high-voltage resource utilization rate can be extracted, and the multiple high-voltage resource utilization rates corresponding to each preset node can be sorted according to the order of the timestamps. Based on the sorting result, determine the high-voltage resource utilization rates that are consecutively adjacent on the timeline to obtain multiple continuous high-voltage resource utilization rates for each preset node. The number of high-voltage continuous resource utilization rates for each preset node is counted and compared with the preset number. If the number of high-voltage continuous resource utilization rates for multiple nodes is greater than the preset number, the corresponding preset node is designated as the resource borrowing node.

[0035] This solution filters historical records with resource utilization rates exceeding a preset borrowing threshold and identifies continuous high-voltage resource utilization sequences by combining timestamps. When the number of continuous high-voltage records exceeds a preset number, the corresponding node is identified as a resource borrowing node. This ensures that resource borrowing requests are initiated only by nodes that have been under high load for a long time, avoiding false triggers caused by momentary jitter, and further improving the accuracy of resource borrowing determination and the reliability of scheduling decisions.

[0036] S103, predict the resource collaboration limit of the resource borrowing node based on the current available resources and current task volume of the resource borrowing node.

[0037] The resource sharing quota can represent the amount of resources a preset node needs to borrow or lend from its currently available resources. The resource sharing quota includes positive and negative values. A positive value indicates that the node needs to borrow resources from the cluster, and a negative value indicates that the node needs to borrow resources from the cluster.

[0038] In one embodiment, the specific method for predicting the resource collaboration limit of a resource borrowing node based on its current available resources and current task volume can be as follows: Based on the comparison results of size relationships, if the utilization rate of multiple historical resources of a certain preset node is greater than a preset resource borrowing threshold, it is determined that the current available resources of that node are about to fall below the resource volume required by the pending tasks under its load. In this case, the node needs more available resources to schedule tasks, and therefore, the node is identified as a resource borrowing node. The specific method for predicting the resource collaboration limit of a resource borrowing node based on its current available resources and current task volume can be as follows: Based on the task processing records of the resource borrowing node, the average resource consumption coefficient per unit of task volume is determined, and the product of the current task volume and the average resource consumption coefficient is calculated. The difference between the current available resources and the product is also calculated, and this difference is used as the predicted resource collaboration limit of the resource borrowing node.

[0039] S104. Based on the current available resources of each preset node and multiple historical resource utilization rates, determine the current idle resources of each preset node, and designate the preset node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node.

[0040] The current amount of free resources can be the total amount of unused hardware resources of a certain remaining node at the current moment. This value reflects the upper limit of resources that the node can currently borrow from other nodes. The resource lending node can be a preset node among multiple remaining nodes used to provide free hardware resources to the resource borrowing node. This node has relatively sufficient free hardware resources at the current moment and has the ability and qualification to temporarily lend resources to other nodes.

[0041] In one embodiment, determining the current idle resource quantity of each preset node based on its current available resource quantity and multiple historical resource utilization rates can be achieved by: filtering out resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes; obtaining the current available resource quantity and current resource utilization rate of each remaining node; calculating the product of the current available resource quantity and the current resource utilization rate; and calculating the difference between the current available resource quantity and the product to obtain the current idle resource quantity of each remaining node. Optionally, to smooth short-term fluctuations and improve the accuracy of idle resource quantity estimation, multiple historical resource utilization rates of each remaining node over the most recent several preset periods can be obtained; the average or weighted average of these historical resource utilization rates can be calculated as a reference resource utilization rate; the difference between 1 and the reference resource utilization rate can be calculated; and the difference can be multiplied by the current available resource quantity to obtain the current idle resource quantity of each remaining node.

[0042] In one embodiment, the specific implementation of using the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node can be as follows: sort the current idle resource quantities of all remaining nodes in descending order of numerical value, and determine the remaining node with the largest current idle resource quantity in the sorting result as the resource lending node corresponding to that resource borrowing node.

[0043] In one embodiment, there are multiple resource lending nodes; the remaining node with the largest current idle resource amount in the sorting result is used as the resource lending node corresponding to the resource borrowing node, including: sorting the current idle resource amount of each remaining node in descending order; accumulating the current idle resource amount of each remaining node according to the sorting result until the accumulated calculation result is greater than or equal to the resource sharing quota, and using the multiple remaining nodes corresponding to the accumulated calculation result as multiple resource lending nodes corresponding to the resource borrowing node.

[0044] In one embodiment, the current free resource amounts of each remaining node are sorted in descending order. The current free resource amounts of each remaining node are then accumulated according to the sorting result until the accumulated result is greater than or equal to the resource sharing quota. The multiple remaining nodes corresponding to the accumulated result are then designated as multiple resource lending nodes corresponding to the resource borrowing node. A specific implementation of this approach is as follows: when the current free resource amount of a single remaining node is insufficient to meet the resource sharing quota required by the resource borrowing node, multiple nodes among the remaining nodes can be selected as resource lending nodes. The current free resource amounts of each remaining node are sorted in descending order. Starting from the remaining node with the largest current free resource amount, the current free resource amounts of each remaining node are traversed sequentially. The current free resource amounts of the traversed remaining nodes are accumulated, and after each accumulation operation, the accumulated result is compared with the resource sharing quota until the value of the accumulated result first reaches or exceeds the resource sharing quota. All remaining nodes that have been accumulated are collectively determined as multiple resource lending nodes corresponding to the resource borrowing node.

[0045] This scheme accumulates the idle resources of each remaining node in descending order of their current available resources until the accumulated amount reaches or exceeds the resource sharing limit required by the borrowing node. All remaining nodes that have participated in the accumulation are then identified as multiple resource lending nodes corresponding to the borrowing node. This allows for the aggregation of redundant resources from multiple nodes to complete the borrowing response when a single node's idle resources are insufficient, avoiding borrowing failures due to single node resource limits and further improving the success rate of resource borrowing and the overall utilization rate of cluster resources.

[0046] S105, increase the collaboration limit of the resource borrowing node according to the resource collaboration limit to increase the resource control authority range of the resource borrowing node, and decrease the collaboration limit of the resource lending node according to the resource collaboration limit to reduce the resource control authority range of the resource lending node. Each preset node performs task scheduling based on the corresponding resource control authority range.

[0047] In one embodiment, the resource borrowing node's collaboration quota is increased according to the resource collaboration quota to increase its resource control authority range, and the resource lending node's collaboration quota is decreased according to the resource collaboration quota to decrease its resource control authority range. The specific implementation of task scheduling based on the corresponding resource control authority range for each preset node can be as follows: calculate the difference between the current idle resource amount of the resource lending node and the resource collaboration quota to obtain the final idle resource amount of the resource lending node, and transfer the virtual resource authority of the reduced portion of the resource from the resource lending node to the resource borrowing node. That is, the final available resource amount of the resource borrowing node at this time is the sum of the node's current available resource amount and the resource collaboration quota. The resource borrowing node can use the final available resource amount as the maximum resource limit for the current tasks it can handle for task processing. In other words, the resource borrowing node can simultaneously use the original resources corresponding to the current available resource amount and the collaborative resources corresponding to the resource collaboration quota from the resource lending node to jointly schedule its current pending tasks, and immediately schedule tasks that were originally in a queue waiting state due to insufficient resources to the execution state, thereby reducing the task queuing length and task execution delay of the node.

[0048] The technical solution provided in this application embodiment obtains the current available resource quantity, current task quantity, current resource utilization rate of each preset node, and multiple historical resource utilization rates collected according to a preset period. The current available resource quantity includes the collaborative quota, and the initial value of the collaborative quota is 0. Multiple high-voltage resource utilization rates greater than a preset resource borrowing threshold are selected from the multiple historical resource utilization rates of each preset node. Multiple high-voltage continuous resource utilization rates are determined based on the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are designated as resource borrowing nodes. Based on the resource borrowing nodes... The resource collaboration limit of the resource borrowing node is predicted based on the current available resources and current task volume; the current idle resources of each preset node are determined based on the current available resources and multiple historical resource utilization rates, and the preset node with the largest current idle resources is designated as the resource lending node corresponding to the resource borrowing node; the collaboration limit of the resource borrowing node is increased according to the resource collaboration limit to increase the resource allocation authority range of the resource borrowing node, and the collaboration limit of the resource lending node is decreased according to the resource collaboration limit to decrease the resource allocation authority range of the resource lending node. Each preset node performs task scheduling based on its corresponding resource allocation authority range. The above-described task scheduling method based on resource collaboration quota solves the problems of inflexible resource allocation, untimely task scheduling, and poor task processing stability in existing technologies. By predicting the resource collaboration quota of the resource borrowing node based on its current available resources and current task load, and using the remaining node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node, resource collaboration can be performed in advance based on the actual task load and current available resources of each node before node overload, improving the timeliness of task scheduling. By increasing the current available resources of the resource borrowing node according to the resource collaboration quota and decreasing the current idle resources of the resource lending node according to the resource collaboration quota for task scheduling, the transfer of resource ownership can be achieved based on the resource borrowing and lending mechanism without involving hardware movement, thereby changing the upper limit of the resources that the corresponding node can control, improving the flexibility of resource allocation and the stability of task processing.

[0049] Figure 2 This is a flowchart of another task scheduling method based on resource sharing quota provided in an embodiment of this application. For example... Figure 2 As shown, the specific steps include the following: S201, obtain the current available resources, current task volume, current resource utilization rate of each preset node, as well as multiple historical resource utilization rates collected according to a preset period, and compare the relationship between the multiple historical resource utilization rates of each preset node and the preset resource borrowing threshold.

[0050] S202, determine the resource borrowing node among multiple preset nodes based on the comparison results of size relationship, and predict the resource collaboration quota of the resource borrowing node based on the current available resources and current task volume of the resource borrowing node.

[0051] S203, filter out resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes, and determine the current idle resource amount of each remaining node based on the multiple resource utilization rates and the current available resource amount. The current available resource amount includes the preset resource amount.

[0052] S204, filter out the resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes.

[0053] In one embodiment, the specific implementation of filtering out resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes can be: filtering out resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes.

[0054] S205, determine the resource borrowing threshold of the resource borrowing node based on the first preset ratio and the preset resource amount of the resource borrowing node.

[0055] The first preset ratio can be a pre-defined coefficient used to control the maximum amount of resources that a resource-borrowing node can borrow; in this scheme, the value is primarily set to 300%. The resource borrowing threshold can be the maximum amount of resources that a resource-borrowing node is allowed to borrow in the resource coordination mechanism. When the total amount of resources borrowed by a resource-borrowing node reaches or exceeds this threshold, the node will be prohibited from borrowing more resources to prevent a single node from excessively occupying shared resources in the cluster and affecting the normal operation of other nodes.

[0056] In one embodiment, the specific implementation of determining the resource borrowing threshold of a resource borrowing node based on a first preset ratio and a preset resource amount of the resource borrowing node can be: calculating the product of the first preset ratio and the preset resource amount of the resource borrowing node, and using the product as the resource borrowing threshold of the resource borrowing node.

[0057] S206, Based on the second preset ratio and the preset resource amount of each remaining node, determine the resource lending threshold of each remaining node, and calculate the sum of the resource lending thresholds of all remaining nodes to obtain the current lending threshold.

[0058] The second preset ratio can be a pre-set coefficient used to control the maximum amount of resources that each remaining node can borrow from the cluster; in this scheme, it is mainly set to 70%. The resource lending threshold can be the maximum amount of resources that each remaining node can borrow from the cluster. The current lendable threshold can be the maximum total amount of resources that all nodes in the current cluster with resource lending qualifications can provide to the resource borrowing node within their respective allowed lending limits. This threshold is used to compare with the resource coordination quota required by the resource borrowing node to determine whether the current overall idle resource supply capacity of the cluster can meet the resource borrowing needs of that node.

[0059] In one embodiment, the specific implementation of determining the resource lending threshold of each remaining node based on the second preset ratio and the preset resource amount of each remaining node, and calculating the sum of the resource lending thresholds of all remaining nodes to obtain the current lending threshold can be as follows: calculate the product of the second preset ratio and the preset resource amount of each remaining node respectively, use the product as the resource lending threshold of each remaining node, and calculate the sum of all resource lending thresholds, and use the sum as the current lending threshold.

[0060] S207, if the resource collaboration limit is greater than the current available lending threshold, terminate the operation of increasing the current available resource amount of the resource borrowing node.

[0061] In one embodiment, the specific implementation of terminating the increase of the current available resource amount of the resource borrowing node when the resource collaboration limit is greater than the current available lending threshold can be as follows: compare the resource collaboration limit of the resource borrowing node with the current available lending threshold. If the resource collaboration limit is greater than the current available lending threshold, it means that the amount of resources requested by the resource borrowing node this time has exceeded the maximum total amount of resources that all remaining nodes in the cluster can provide within their respective allowed lending limits. At this time, even if all the available lending resources of all remaining nodes are pooled together, it is impossible to fill the resource gap of the resource borrowing node. Therefore, the increase of the current available resource amount of the resource borrowing node is directly terminated.

[0062] S208: Sort the current free resources of each remaining node in descending order, and select the remaining node with the largest current free resources in the sorting result as the resource lending node corresponding to the resource borrowing node.

[0063] S209, increase the current available resources of the resource borrowing node according to the resource coordination quota, and decrease the current idle resources of the resource lending node according to the resource coordination quota, so that the resource borrowing node can perform task scheduling.

[0064] The technical solution provided in this application determines the resource borrowing limit of a node based on a first preset ratio and the preset resource amount of the resource borrowing node. At the same time, it determines the resource lending limit of each node based on a second preset ratio and the preset resource amount of each remaining node, and sums them to obtain the total amount of resources that the cluster can currently lend. When the resource collaboration amount required by the resource borrowing node exceeds its own borrowing limit or exceeds the total amount that the cluster can currently lend, the resource addition operation for that node is immediately terminated. This can avoid a single node over-borrowing resources, which would lead to its own load imbalance, and the overall cluster resources being over-borrowed, which would affect the normal operation of other nodes, thus ensuring the security and stability of resource collaboration.

[0065] Figure 3 This is a flowchart of a method for collaboratively updating available resource quantities provided in an embodiment of this application. Figure 3 As shown, the currently available resources also include the collaboration quota, which has an initial value of 0. The collaboration quota includes the following steps: S301, calculate the sum of the resource collaboration quota and the collaboration quota of the resource borrowing node to obtain the first collaboration quota update result, and calculate the sum of the preset resource amount of the resource borrowing node and the first collaboration quota update result to obtain the increase result of the current available resource amount of the resource borrowing node. The increase result of the current available resource amount is the resource control authority range of the resource borrowing node.

[0066] The collaboration quota can be the current effective resource collaboration quota value of a preset node (i.e., the cumulative amount of resources borrowed or lent by the node). If the collaboration quota is positive, it means that the node has currently net borrowed the corresponding amount of resources from the cluster; if the collaboration quota is negative, it means that the node has currently net borrowed the corresponding amount of resources from the cluster; if the collaboration quota is zero, it means that the node is not currently participating in resource collaboration. The collaboration quota is dynamically updated with each resource borrowing or returning operation of the corresponding node. The first collaboration quota update result can be the latest value of the cumulative borrowed resources of the resource borrowing node after completing this resource borrowing operation.

[0067] In one embodiment, the specific implementation of calculating the sum of the resource coordination limit and the coordination limit of the resource borrowing node to obtain the first coordination limit update result, and calculating the sum of the preset resource amount of the resource borrowing node and the first coordination limit update result to obtain the increase in the current available resource amount of the resource borrowing node, can be as follows: The resource coordination limit is denoted as ΔR, which is a positive number representing the amount of resources to be borrowed this time; the current coordination limit of the resource borrowing node is denoted as (Qcoop, in); and the preset resource amount of the resource borrowing node is denoted as (Rinit, in). The resource coordination limit and the current coordination limit of the resource borrowing node are summed: (Qcoop, in_new) = (Qcoop, in) + ΔR. The summation result is used as the first coordination limit update result. This update result represents the total amount of resources borrowed by the resource borrowing node after completing this resource borrowing. The preset resource amount of the resource borrowing node is denoted as (Rinit,in). The sum of the preset resource amount and the update result of the first collaborative quota is calculated as: (Rtotal,in) = (Rinit,in) + (Qcoop,in_new). The sum is taken as the increase in the current available resource amount of the resource borrowing node. The increase in the current available resource amount represents all the resources that the resource borrowing node can allocate for task scheduling and processing. That is, without involving the physical migration of hardware resources, the virtual ownership of resources is increased only by adjusting the value of the collaborative quota.

[0068] S302, calculate the negative of the resource collaboration quota and sum it with the collaboration quota of the resource lending node to obtain the second collaboration quota update result, and calculate the sum of the preset resource amount of the resource lending node and the second collaboration quota update result to obtain the reduction result of the current idle resource amount of the resource lending node. The reduction result of the current idle resource amount of the resource lending node is the resource control authority range of the resource lending node.

[0069] The second collaborative quota update result can be a new collaborative quota value obtained by summing the negative of the current collaborative quota with the current collaborative quota of the resource lending node. This update result represents the latest value (indicated by a negative value) of the cumulative amount of resources lent by the resource lending node after completing this resource lending operation.

[0070] In one embodiment, the sum of the negative of the resource collaboration quota and the collaboration quota of the resource lending node is used to obtain the second collaboration quota update result. The sum of the preset resource amount of the resource lending node and the second collaboration quota update result is then used to obtain the decrease in the current idle resource amount of the resource lending node. A specific implementation of this can be as follows: the current collaboration quota of the resource lending node is denoted as (Qcoop, out), the preset resource amount of the resource lending node is denoted as (Rinit, out), and the resource collaboration quota is denoted as ΔR (this value is positive). The negative of this resource collaboration quota is then calculated to obtain... ΔR represents the amount of resources the resource lending node needs to reduce in this instance. An update calculation is performed on the resource lending node's collaboration quota, summing the negative of the current collaboration quota with the existing collaboration quota of the resource lending node: (Qcoop, out_new) = (Qcoop, out) + ( ΔR), and the summation result is used as the second collaborative quota update result. This update result represents the cumulative amount of resources borrowed by the resource lending node after completing this resource lending. The preset resource amount of the resource lending node is summed with the second collaborative quota update result: (Rtotal, out) = (Rinit, out) + (Qcoop, out_new), and the summation result is used as the decrease in the current idle resource amount of the resource lending node. The decrease in the current idle resource amount of the resource lending node represents all the resources that the resource lending node can allocate for task scheduling within its resource allocation authority scope. That is, without involving the physical migration of hardware resources, the transfer of virtual ownership of resources is achieved only by adjusting the value of the collaborative quota.

[0071] The technical solution provided in this application embodiment increases the available resource amount of the node by adding the current resource collaboration amount to the original collaboration amount of the resource borrowing node and summing it with its preset resource amount. Conversely, it decreases the available resource amount of the node by adding the opposite of the current resource collaboration amount to the original collaboration amount of the resource lending node and summing it with its preset resource amount. This achieves real-time transfer of virtual ownership of resources, ensures the consistency and accuracy of resource status of all nodes in the cluster, and improves the stability of task scheduling and processing.

[0072] Figure 4 This is a structural block diagram of a task scheduling system based on resource sharing quota provided in an embodiment of this application. For example... Figure 4 As shown, it specifically includes the following: The data acquisition module 401 is used to acquire the current available resources, current task volume, current resource utilization rate of each preset node, and multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaboration quota, and the initial value of the collaboration quota is 0. The borrowing node determination module 402 is used to filter multiple high-voltage resource utilization rates that are greater than the preset resource borrowing threshold among multiple historical resource utilization rates of each preset node, and to determine multiple high-voltage continuous resource utilization rates among multiple high-voltage resource utilization rates according to the timestamps corresponding to each high-voltage resource utilization rate, and to take preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number as resource borrowing nodes. The resource collaboration quota prediction module 403 is used to predict the resource collaboration quota of the resource borrowing node based on the current available resources and current task volume of the resource borrowing node. The lending node determination module 404 is used to determine the current idle resource quantity of each preset node based on the current available resource quantity of each preset node and multiple historical resource utilization rates, and to take the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node. The task scheduling module 405 is used to increase the collaboration quota of the resource borrowing node according to the resource collaboration quota, so as to increase the resource control authority range of the resource borrowing node, and to decrease the collaboration quota of the resource lending node according to the resource collaboration quota, so as to decrease the resource control authority range of the resource lending node. Each preset node performs task scheduling based on the corresponding resource control authority range.

[0073] The technical solution provided in this application includes a data acquisition module for acquiring the current available resource quantity, current task quantity, current resource utilization rate, and multiple historical resource utilization rates collected according to a preset period for each preset node. The current available resource quantity includes a collaborative quota, with an initial value of 0. A borrowing node determination module is used to filter multiple high-voltage resource utilization rates from the multiple historical resource utilization rates of each preset node that are greater than a preset resource borrowing threshold, and to determine multiple high-voltage continuous resource utilization rates from the multiple high-voltage resource utilization rates based on the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are designated as resource borrowing nodes. A resource collaborative quota prediction module is used to... The system predicts the resource collaboration limit of the resource borrowing node based on its current available resources and current task volume. The lending node determination module determines the current idle resources of each preset node based on its current available resources and multiple historical resource utilization rates, and designates the preset node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node. The task scheduling module increases the collaboration limit of the resource borrowing node according to the resource collaboration limit, thereby increasing the resource allocation authority of the resource borrowing node, and decreases the collaboration limit of the resource lending node according to the resource collaboration limit, thereby reducing the resource allocation authority of the resource lending node. Each preset node performs task scheduling based on its corresponding resource allocation authority. The aforementioned task scheduling system based on resource collaboration quotas solves the problems of inflexible resource allocation, untimely task scheduling, and poor task processing stability in existing technologies. By predicting the resource collaboration quota of a resource borrowing node based on its current available resources and current task load, and designating the remaining node with the largest current idle resources as the resource lending node corresponding to the resource borrowing node, resource collaboration can be performed in advance based on the actual task load and current available resources of each node before node overload, improving the timeliness of task scheduling. By increasing the current available resources of the resource borrowing node according to the resource collaboration quota and decreasing the current idle resources of the resource lending node according to the resource collaboration quota for task scheduling, the transfer of resource ownership can be achieved based on the resource borrowing and lending mechanism without involving hardware relocation. This changes the upper limit of resources that the corresponding node can control, improving the flexibility of resource allocation and the stability of task processing.

[0074] Furthermore, the currently available resources include the preset resources; The lending node determination module 404 is also used for: Filter out the resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes; The resource borrowing threshold of the resource borrowing node is determined based on the first preset ratio and the preset resource amount of the resource borrowing node. Based on the second preset ratio and the preset resource amount of each remaining node, the resource lending threshold of each remaining node is determined, and the sum of the resource lending thresholds of all remaining nodes is calculated to obtain the current lending threshold. If the resource collaboration limit is greater than the current available lending threshold, terminate the operation of increasing the current available resource amount of the resource borrowing node.

[0075] Furthermore, the task scheduling module 405 is specifically used for: The sum of the resource collaboration quota and the collaboration quota of the resource borrowing node is calculated to obtain the first collaboration quota update result. The sum of the preset resource amount of the resource borrowing node and the first collaboration quota update result is calculated to obtain the increase result of the current available resource amount of the resource borrowing node. The increase result of the current available resource amount is the resource allocation authority range of the resource borrowing node. The negative of the resource collaboration quota is calculated and summed with the collaboration quota of the resource lending node to obtain the second collaboration quota update result. The preset resource amount of the resource lending node is calculated and summed with the second collaboration quota update result to obtain the reduction result of the current idle resource amount of the resource lending node. The reduction result of the current idle resource amount of the resource lending node is the resource allocation authority range of the resource lending node.

[0076] Furthermore, the number of resource lending nodes is multiple; The lending node determination module 404 is specifically used for: Sort the current available resources of each remaining node in descending order; The current idle resource amount of each remaining node is accumulated according to the sorting result until the accumulated calculation result is greater than or equal to the resource sharing quota. The multiple remaining nodes corresponding to the accumulated calculation result are then used as multiple resource lending nodes corresponding to the resource borrowing node.

[0077] Furthermore, the currently available resources include the preset resources; The system also includes: The low-voltage utilization rate filtering module is used to filter multiple low-voltage resource utilization rates that are less than or equal to a preset resource return threshold from multiple historical resource utilization rates of each preset node, and to identify the timestamp corresponding to each low-voltage resource utilization rate. The return node determination module is used to determine multiple low-voltage continuous resource utilization rates among multiple low-voltage resource utilization rates of each preset node based on each timestamp. If the number of multiple low-voltage continuous resource utilization rates is greater than the preset number, the corresponding preset node is used as the resource return node. The resource return module is used to calculate the difference between the current available resource amount of the resource return node and the preset resource amount, obtain the collaborative quota to be returned of the resource return node, and determine the resource receiving node corresponding to the collaborative quota to be returned among multiple preset nodes, so as to increase the current available resource amount of the resource receiving node according to the collaborative quota to be returned.

[0078] Furthermore, there are multiple resource borrowing nodes; The resource return module is specifically used for: Compare the size relationships of multiple resource collaboration limits corresponding to multiple resource borrowing nodes, and select the resource borrowing node with the largest resource collaboration limit as the resource receiving node corresponding to the collaboration limit to be repaid. Alternatively, identify the original resource lending node in the resource borrowing record of the obtained resource return node, and use the original resource lending node as the resource receiving node corresponding to the collaborative amount to be returned.

[0079] This application also provides an electronic device, which can integrate a resource-coordination-based task scheduling device provided in this application. Figure 5 This is a schematic diagram of the structure of a task scheduling device based on resource collaboration quota provided in an embodiment of this application, with reference to... Figure 5 The task scheduling device based on resource sharing quota includes: an input device 53, an output device 54, a memory 52, and one or more processors 51; the memory 52 is used to store one or more programs; when one or more programs are executed by one or more processors 51, the one or more processors 51 implement the task scheduling method based on resource sharing quota as provided in the above embodiments. The input device 53, output device 54, memory 52, and processors 51 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0080] The memory 52, as a computing device readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the task scheduling method based on resource sharing quota provided in any embodiment of this application. The memory 52 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on device usage. Furthermore, the memory 52 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 52 may further include memory remotely located relative to the processor 51, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0081] Input device 53 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 54 may include display devices such as a display screen.

[0082] The processor 51 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 52, thereby realizing the above-mentioned task scheduling method based on resource coordination quota.

[0083] The task scheduling device, equipment, and computer based on resource sharing quota provided above can be used to execute the task scheduling method based on resource sharing quota provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0084] This application embodiment also provides a storage medium for storing computer-executable instructions. When executed by a computer processor, these computer-executable instructions are used to execute the task scheduling method based on resource coordination quota provided in the above embodiment. The task scheduling method based on resource coordination quota includes: obtaining the current available resource quantity, current task quantity, current resource utilization rate of each preset node, and multiple historical resource utilization rates collected according to a preset period. The current available resource quantity includes a coordination quota, and the initial value of the coordination quota is 0; filtering multiple high-voltage resource utilization rates from the multiple historical resource utilization rates of each preset node that are greater than a preset resource borrowing threshold, and determining multiple high-voltage continuous resource utilization rates from the multiple high-voltage resource utilization rates based on the timestamps corresponding to each high-voltage resource utilization rate; and scheduling multiple high-voltage continuous resource utilization rates. Preset nodes with a high-voltage continuous resource utilization rate greater than a preset number are designated as resource borrowing nodes. The resource coordination limit of the resource borrowing nodes is predicted based on their current available resources and current task load. The current idle resources of each preset node are determined based on their current available resources and multiple historical resource utilization rates, and the preset node with the largest current idle resources is designated as the resource lending node corresponding to the resource borrowing node. The coordination limit of the resource borrowing nodes is increased according to the resource coordination limit to expand their resource allocation authority, and the coordination limit of the resource lending nodes is decreased according to the resource coordination limit to reduce their resource allocation authority. Each preset node performs task scheduling based on its corresponding resource allocation authority.

[0085] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROMs, floppy disks, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. Furthermore, storage media may reside in a first computer system in which a program is executed, or may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). Storage media may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0086] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the task scheduling method based on resource sharing quota as described above, but can also execute related operations in the task scheduling method based on resource sharing quota provided in any embodiment of this application.

[0087] The task scheduling device, equipment, and storage medium based on resource sharing quota provided in the above embodiments can execute the task scheduling method based on resource sharing quota provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the task scheduling method based on resource sharing quota provided in any embodiment of this application.

[0088] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A task scheduling method based on resource collaboration quota, characterized in that, The method includes: The system obtains the current available resources, current task volume, current resource utilization rate of each preset node, as well as multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaborative quota, the initial value of which is 0. The collaborative quota includes the cumulative borrowed or lent resources of each preset node at the current moment. Multiple high-voltage resource utilization rates that are greater than a preset resource borrowing threshold are selected from the multiple historical resource utilization rates of each preset node. Multiple high-voltage continuous resource utilization rates are determined from the multiple high-voltage resource utilization rates according to the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are selected as resource borrowing nodes. The preset resource borrowing threshold includes the minimum resource utilization rate at which each preset node can trigger resource borrowing. The high-voltage continuous resource utilization rate includes multiple consecutive adjacent high-voltage resource utilization rates of each preset node. The resource collaboration limit of the resource borrowing node is predicted based on the current available resources and current task volume of the resource borrowing node, which includes: determining the average resource consumption coefficient per unit task volume according to the task processing record of the resource borrowing node, calculating the product of the current task volume and the average resource consumption coefficient, and calculating the difference between the current available resources and the product, and using the difference as the prediction result of the resource collaboration limit of the resource borrowing node. The resource collaboration limit includes the amount of resources that each preset node needs to borrow or lend from the current available resources. The current idle resource amount of each preset node is determined based on the current available resource amount of each preset node and multiple historical resource utilization rates, and the preset node with the largest current idle resource amount is taken as the resource lending node corresponding to the resource borrowing node. The resource borrowing node's collaboration limit is increased according to the resource collaboration limit to increase its resource control authority range, and the resource lending node's collaboration limit is decreased according to the resource collaboration limit to decrease its resource control authority range. Each preset node performs task scheduling based on its corresponding resource control authority range.

2. The task scheduling method based on resource collaboration quota according to claim 1, characterized in that, The currently available resource quantity includes the preset resource quantity; Before selecting the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node, the method further includes: Filter out the resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes; The resource borrowing threshold of the resource borrowing node is determined based on the first preset ratio and the preset resource amount of the resource borrowing node. The resource borrowing threshold includes the maximum amount of resources that the resource borrowing node is allowed to borrow in the resource coordination mechanism. Based on the second preset ratio and the preset resource amount of each remaining node, the resource lending threshold of each remaining node is determined, and the sum of the resource lending thresholds of all remaining nodes is calculated to obtain the current lending threshold. If the resource collaboration limit is greater than the current available lending threshold, the operation to increase the current available resource amount of the resource borrowing node is terminated.

3. The task scheduling method based on resource collaboration quota according to claim 2, characterized in that, The step of increasing the collaboration limit of the resource borrowing node according to the resource collaboration limit, thereby increasing the resource allocation authority of the resource borrowing node, and decreasing the collaboration limit of the resource lending node according to the resource collaboration limit, thereby decreasing the resource allocation authority of the resource lending node, includes: The resource collaboration quota is calculated by summing the resource collaboration quota with the resource borrowing node's collaboration quota to obtain the first collaboration quota update result. The preset resource amount of the resource borrowing node is calculated by summing the first collaboration quota update result with the first collaboration quota update result to obtain the increase result of the current available resource amount of the resource borrowing node. The increase result of the current available resource amount is the resource allocation authority range of the resource borrowing node. The negative of the resource collaboration quota is calculated and summed with the collaboration quota of the resource lending node to obtain the second collaboration quota update result. The preset resource amount of the resource lending node is calculated and summed with the second collaboration quota update result to obtain the reduction result of the current idle resource amount of the resource lending node. The reduction result of the current idle resource amount of the resource lending node is the resource allocation authority range of the resource lending node.

4. The task scheduling method based on resource collaboration quota according to claim 1, characterized in that, The number of resource lending nodes is multiple; The step of selecting the preset node with the largest current idle resource quantity as the resource lending node corresponding to the resource borrowing node includes: Sort the current available resources of each remaining node in descending order; The current idle resource amount of each remaining node is accumulated according to the sorting result until the accumulated calculation result is greater than or equal to the resource coordination quota, and the multiple remaining nodes corresponding to the accumulated calculation result are regarded as multiple resource lending nodes corresponding to the resource borrowing node.

5. The task scheduling method based on resource collaboration quota according to claim 1, characterized in that, The current available resources include the preset resources; After filtering multiple high-voltage resource utilization rates from multiple historical resource utilization rates of each preset node that are greater than a preset resource borrowing threshold, the method further includes: Filter out multiple low-voltage resource utilization rates from the multiple historical resource utilization rates of each preset node that are less than or equal to a preset resource return threshold, and identify the timestamp corresponding to each low-voltage resource utilization rate; Based on each timestamp, determine multiple low-voltage continuous resource utilization rates among the multiple low-voltage resource utilization rates of each preset node. If the number of multiple low-voltage continuous resource utilization rates is greater than a preset number, the corresponding preset node is used as a resource return node. The difference between the current available resource amount of the resource return node and the preset resource amount is calculated to obtain the pending return coordination quota of the resource return node. The resource receiving node corresponding to the pending return coordination quota among multiple preset nodes is determined so as to increase the current available resource amount of the resource receiving node according to the pending return coordination quota.

6. The task scheduling method based on resource collaboration quota according to claim 5, characterized in that, The number of resource borrowing nodes is multiple; The step of determining the resource receiving node corresponding to the collaborative amount to be repaid from among multiple preset nodes includes: Compare the size relationships of multiple resource collaboration limits corresponding to multiple resource borrowing nodes, and select the resource borrowing node with the largest resource collaboration limit as the resource receiving node corresponding to the collaboration limit to be repaid. Alternatively, the original resource lending node in the resource borrowing record of the obtained resource return node can be identified, and the original resource lending node can be used as the resource receiving node corresponding to the collaborative amount to be returned.

7. A task scheduling system based on resource collaboration quotas, characterized in that, The system includes: The data acquisition module is used to acquire the current available resources, current task volume, current resource utilization rate of each preset node, and multiple historical resource utilization rates collected according to a preset period. The current available resources include the collaborative quota, the initial value of which is 0, and the collaborative quota includes the cumulative borrowed resources or cumulative borrowed resources of each preset node at the current moment. The borrowing node determination module is used to filter multiple high-voltage resource utilization rates that are greater than a preset resource borrowing threshold from multiple historical resource utilization rates of each preset node, and to determine multiple high-voltage continuous resource utilization rates from the multiple high-voltage resource utilization rates according to the timestamps corresponding to each high-voltage resource utilization rate. Preset nodes with a number of multiple high-voltage continuous resource utilization rates greater than a preset number are designated as resource borrowing nodes. The preset resource borrowing threshold includes the minimum resource utilization rate at which each preset node can trigger resource borrowing, and the high-voltage continuous resource utilization rate includes multiple consecutive adjacent high-voltage resource utilization rates of each preset node. The resource collaboration quota prediction module is used to predict the resource collaboration quota of the resource borrowing node based on the current available resources and current task volume of the resource borrowing node. The module includes: determining the average resource consumption coefficient per unit of task volume based on the task processing records of the resource borrowing node; calculating the product of the current task volume and the average resource consumption coefficient; calculating the difference between the current available resources and the product; and using the difference as the prediction result of the resource collaboration quota of the resource borrowing node. The resource collaboration quota includes the amount of resources that each preset node needs to borrow or lend from the current available resources. The lending node determination module is used to determine the current idle resource amount of each preset node based on the current available resource amount of each preset node and multiple historical resource utilization rates, and to take the preset node with the largest current idle resource amount as the resource lending node corresponding to the resource borrowing node. The task scheduling module is used to increase the collaboration quota of the resource borrowing node according to the resource collaboration quota, so as to increase the resource control authority range of the resource borrowing node, and to decrease the collaboration quota of the resource lending node according to the resource collaboration quota, so as to decrease the resource control authority range of the resource lending node, and each preset node performs task scheduling based on the corresponding resource control authority range.

8. The task scheduling system based on resource collaboration quota according to claim 7, characterized in that, The currently available resource quantity includes the preset resource quantity; The module for determining the borrowed node is also used for: Filter out the resource borrowing nodes from multiple preset nodes to obtain multiple remaining nodes; The resource borrowing threshold of the resource borrowing node is determined based on the first preset ratio and the preset resource amount of the resource borrowing node. The resource borrowing threshold includes the maximum amount of resources that the resource borrowing node is allowed to borrow in the resource coordination mechanism. Based on the second preset ratio and the preset resource amount of each remaining node, the resource lending threshold of each remaining node is determined, and the sum of the resource lending thresholds of all remaining nodes is calculated to obtain the current lending threshold. If the resource collaboration limit is greater than the current available lending threshold, the operation to increase the current available resource amount of the resource borrowing node is terminated.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of a task scheduling method based on resource sharing quota as described in any one of claims 1-6.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of a task scheduling method based on resource sharing quota as described in any one of claims 1-6.

Images