Task concurrency scheduling methods, computer equipment and storage media, and program products
By dynamically adjusting the concurrency quota and health coefficient, the problem of resource imbalance caused by fixed concurrency quota is solved, and efficient utilization and stable operation of cluster resources are achieved.
Patent Information
- Application Number
- CN202611325617.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
In large-scale task scenarios, the fixed concurrency quota scheduling method leads to an uneven demand for computing, storage and other hardware resources among the working nodes in the cluster. This can easily cause node task blocking, execution timeouts or even node crashes, resulting in resource waste and making it difficult to ensure the stability of cluster operation.
Based on the current resource utilization rate of the target cluster and the health status indicators of the worker nodes, the global concurrency quota is dynamically adjusted. The target concurrency quota of the worker nodes is calculated based on the health coefficient, and task scheduling is performed based on the concurrency quota lease to ensure the balance and stability of resource utilization.
It improves the balance of hardware resource utilization in the target cluster, ensures the stability of cluster task execution, avoids node resource idleness and overload, and improves overall task processing efficiency.
Smart Images

Figure CN123114758A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of distributed scheduling technology, and in particular to a task concurrent scheduling method, computer equipment and storage medium, and program product. Background Technology
[0002] With the large-scale deployment of large language model agents in software development, data processing and other scenarios, multiple working nodes need to execute massive agent inference tasks in a distributed manner in parallel.
[0003] Existing technologies typically rely on fixed concurrency quota scheduling, which pre-allocates a fixed number of concurrent task execution permissions to each worker node. However, in large-scale task scenarios, the computing and storage requirements of each worker node in the cluster vary. Fixed concurrency quotas can easily lead to problems such as task blocking, execution timeouts, and even node crashes on some nodes. Furthermore, it can cause some nodes to have idle and wasted computing resources, resulting in an imbalance in the allocation of cluster hardware resources and making it difficult to guarantee the stability of the cluster's operation. Therefore, improving the balance of hardware resource utilization in the target cluster and enhancing the stability of cluster task execution have become urgent problems to be solved. Summary of the Invention
[0004] The main technical problem addressed by this application is to provide a task concurrency scheduling method, computer equipment and storage medium, and program product that can improve the balance of hardware resource utilization of the target cluster and help improve the stability of cluster task execution.
[0005] To address the aforementioned technical problems, the first aspect of this application provides a task concurrency scheduling method, comprising: adjusting the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, and obtaining the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster; obtaining the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes; issuing concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and scheduling the tasks to be executed on the worker nodes based on the concurrency quota leases of the worker nodes.
[0006] To address the aforementioned technical problems, a second aspect of this application provides a computer device comprising at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor executes the program instructions to implement the task concurrency scheduling method of the first aspect described above.
[0007] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the task concurrency scheduling method described in the first aspect.
[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer program product, comprising: a computer program / instruction, which, when executed by a processor, implements the task concurrency scheduling method described in the first aspect.
[0009] The above scheme adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed on the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution. Attached Figure Description
[0010] Figure 1 This is a flowchart illustrating an embodiment of the task concurrency scheduling method of this application; Figure 2 This is a schematic diagram of the framework of an embodiment of the utilization detection module in the task concurrency scheduling method of this application; Figure 3 This is a schematic diagram of the framework of an embodiment of the multi-level token bucket module in the task concurrency scheduling method of this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the task concurrency scheduling system of this application; Figure 5 This is a schematic diagram of the framework of an embodiment of the task concurrency scheduling device of this application; Figure 6 This is a schematic diagram of the framework of an embodiment of the computer device of this application; Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application; Figure 8 This is a schematic diagram of the framework of an embodiment of the computer program product of this application. Detailed Implementation
[0011] The embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0012] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.
[0013] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper indicates two or more objects.
[0014] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the task concurrent scheduling method of this application. It should be noted that the process operations in this embodiment can be executed by an electronic device with computing capabilities or related equipment containing electronic devices; no specific structure or type is limited herein. Specifically, this embodiment may include the following steps: Step S11: Based on the current resource utilization of the target cluster, adjust the global concurrency quota of the target cluster, and obtain the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster.
[0015] In one implementation scenario, multiple worker nodes in the target cluster execute agent inference tasks in a distributed manner in parallel. Specifically, the target cluster provides model service entry points to each worker node through a shared model inference gateway. The target cluster contains several worker nodes, each worker node runs several task containers, the contents of which execute agent inference tasks and call model services through the model inference gateway. The scheduling center of the target cluster maintains an agent inference task queue, organizes the agent inference tasks to be executed according to priority, and dispatches each task to be executed to each worker node for execution based on the concurrency limit.
[0016] In an implementation scenario, the current resource utilization rate of the target cluster represents the overall current load pressure level of the target cluster. Its value reflects the remaining redundancy of computing resources at the cluster level and the overall saturation of tasks to be executed.
[0017] In a specific implementation scenario, the current resource utilization of the target cluster is obtained by periodically probing the monitoring indicators of the gateway. Specifically, the monitoring indicators may include at least one of the following: cluster processor utilization, cluster memory utilization, cluster storage I / O throughput, and cluster network bandwidth utilization. The monitoring indicators obtained from the above probing are aggregated to obtain the current resource utilization of the target cluster.
[0018] In another specific implementation scenario, when multiple types of model services are deployed in the target cluster, there are differences in resource consumption for different request types (inference requests, training requests, fine-tuning requests, batch processing requests, real-time interaction requests, etc.). That is, under the same request volume, the resource consumption of different request types is significantly different. The request volume of each model service in the target cluster calculated by request type can be aggregated to obtain the current resource utilization rate.
[0019] In a specific implementation scenario, for all online services within the target cluster, model M={ , , ..., } (where n is the total number of models deployed in the cluster), perform differentiated data collection, count the real-time request volume of each service model according to request type, and define the request type set T={ , , ..., The request types include, but are not limited to, real-time inference, offline training, model fine-tuning, batch data processing, heartbeat detection, and API calls. The collected basic dataset is Q(i, j), where Q(i, j) represents the i-th model. Request type j The system measures the real-time request volume per unit time, with data collection granularity supporting adaptive adaptation to 1s / 10s / 1min. It dynamically fits and generates weight coefficients w(i,j) of the basic dataset Q(i,j) based on model computing power specifications and request resource consumption characteristics. The weight coefficients can be generated using the following formula: w(i,j)= × ...(1) In formula (1), The basic computing power coefficient of the model is determined by the model parameter scale, inference computing power consumption, and memory usage. The request type consumption coefficient is obtained by fitting the average CPU, GPU, memory, and IO resource utilization of various requests using historical operation and maintenance data, where w(i,j)∈(0,1). Based on the collected segmented request volume and dynamic weights, the comprehensive load value S of a single model is calculated. i The overall load value can be found using the following formula: = ...(2) In formula (2), This represents the current overall load state of the i-th model, integrating the resource consumption contribution of all types of requests for that model. It also retrieves the resource consumption allocated to the model by the target cluster. The proportion of computing resources to the total resources of the target cluster A global weighted aggregation of all single-model load values is performed to obtain the initial scalar of cluster utilization. Please refer to the following formula: = ...(3) Obtain the target cluster's maximum load threshold S_max, which is the theoretical maximum aggregate load value under full load conditions. S_max is calculated from the cluster's total computing power, the maximum concurrent request volume of each model, and the maximum overhead of each request type. Also obtain the idle offset correction coefficient σ, which corrects for quantization errors caused by cluster hardware idleness and request fluctuations. For example, its value ranges from 0.02 to 0.08. Based on the maximum load threshold S_max and the idle offset correction coefficient σ, adjust the initial scalar... After standardization correction, the cluster utilization rate U is obtained, which can be referred to in the following formula: U= σ)+σ……(4) The cluster utilization rate U output in formula (4) is a single floating-point scalar value with a fixed range of [0, 1]. U=0 indicates that the cluster is completely idle, U=1 indicates that the cluster is fully loaded, and 0<U<1 corresponds to different load utilization states, which can quantify the overall resource utilization level of the target cluster. The above scheme, by assigning differentiated weights to different models and different types of requests, and then through weighted aggregation and standardization correction, can more accurately reflect the actual load of the target cluster, providing more reliable data support for the subsequent dynamic adjustment of global concurrency quotas.
[0020] In one implementation scenario, after calculating the current resource utilization of the target cluster, the current resource utilization is written to the shared storage of the target cluster, such as Redis (Remote Dictionary Server), to monitor the global load dynamics in real time.
[0021] Please see Figure 2 , Figure 2This is a schematic diagram of the framework of an embodiment of the utilization detection module in the task concurrency scheduling method of this application. In one implementation scenario, the current resource utilization can be generated based on the utilization detection module of the target cluster. Specifically, the utilization detection module includes an indicator capture unit, a utilization aggregation unit, and a storage writing unit. The indicator capture unit is used to periodically initiate indicator capture requests to the indicator interface of the model inference gateway to obtain the basic performance indicators and request volume data of each model service, and synchronize the captured raw data to the utilization aggregation unit. After receiving the raw data, the utilization aggregation unit obtains the current resource utilization of the target cluster according to the aggregation logic. The storage writing unit then writes the calculated current resource utilization value into the cluster shared storage.
[0022] In one implementation scenario, the global concurrency quota of the target cluster is adjusted based on its current resource utilization. The global concurrency quota represents the maximum number of tasks that can be executed simultaneously within the same time period. Specifically, the global concurrency quota of the target cluster before this adjustment can be pre-determined by the cluster hardware configuration and historical best operating parameters, or it can be the calculation result after the last adjustment. When the current resource utilization of the target cluster is high, it indicates that the overall load pressure of the cluster is already high. Therefore, in this adjustment, the global concurrency quota can be appropriately reduced to limit the overall task concurrency scale and prevent the cluster load from further increasing and causing operational failures. When the current resource utilization of the target cluster is low, it indicates that the cluster still has sufficient computing resource redundancy. At this time, the global concurrency quota can be appropriately increased to increase the overall task concurrency scale, utilize idle computing resources, and improve the overall task processing efficiency of the cluster.
[0023] In a specific implementation scenario, a utilization threshold is obtained. Specifically, the utilization threshold can be determined based on the historical operating data of the target cluster. It is used to characterize the load threshold at which the target cluster can operate stably. The comparison result between the current resource utilization and the utilization threshold is obtained. If the current resource utilization is greater than the utilization threshold, the global concurrency quota of the target cluster is reduced. If the current resource utilization is not greater than the utilization threshold, the global concurrency quota of the target cluster is increased.
[0024] In a specific implementation scenario, in response to the current resource utilization exceeding a utilization threshold, the global concurrency quota of the target cluster is reduced based on a multiplicative subtraction factor. Specifically, the value of the multiplicative subtraction factor typically ranges from (0, 1), and its magnitude can be determined based on the current resource utilization. For example, the greater the current resource utilization exceeds the utilization threshold, the smaller the value of the multiplicative subtraction factor, and the larger the reduction magnitude. The global concurrency quota of the target cluster is multiplied by the multiplicative subtraction factor to achieve the reduction. This solution quickly adjusts the global concurrency scale through the multiplicative subtraction factor, and the adjustment magnitude adapts to load deviations, thereby improving the rationality of global concurrency quota adjustments.
[0025] In a specific implementation scenario, in response to the current resource utilization rate not exceeding the utilization threshold, the global concurrency quota of the target cluster is increased based on an additive increment step size. Specifically, the value of the additive increment step size is determined by the average resource requirement per task in the target cluster. This avoids a sudden increase in cluster load caused by an excessively large single increase. The upward adjustment operation is completed by adding the additive increment step size to the original global concurrency quota of the target cluster. If the current resource utilization rate of the target cluster is still far from the utilization threshold, the increment step size can be increased based on the difference, making full use of idle resources while maintaining the smoothness of the adjustment process.
[0026] In a specific implementation scenario, the utilization thresholds include a first warning threshold, a second warning threshold, and a third warning threshold. The first warning threshold is less than the second warning threshold, the third warning threshold is less than the first warning threshold, and the second warning threshold is less than 1. When the current resource utilization of the target cluster is greater than the first warning threshold but not greater than the second warning threshold, only a small reduction is made to the global concurrency quota to avoid task scheduling fluctuations. When the current resource utilization of the target cluster is greater than the second warning threshold, a large reduction is performed to quickly control the overall load of the cluster and prevent operational failures due to overload. When the current resource utilization of the target cluster is not greater than the third warning threshold, a large increase is performed to quickly restore idle computing power of the target cluster. When the current resource utilization of the target cluster is greater than the third warning threshold but not greater than the first warning threshold, a small increase is performed to slowly expand the concurrency scale and avoid rapid load increases affecting operational stability. The above solution, through multi-threshold layered adjustment, can adapt to the actual scheduling needs of different load ranges of the target cluster, taking into account both the efficiency of concurrent scale expansion and the stability of cluster operation. It avoids the waste of idle computing power under low load and can also control peaks and maintain stability in a timely manner during load increase, adapting to the scheduling needs of dynamic load changes in distributed clusters.
[0027] In one implementation scenario, the global concurrency quota is adjusted based on the current resource utilization of the target cluster and then clamped with an upper and / or lower limit. That is, the adjusted global concurrency quota shall not exceed the preset maximum global quota upper limit, nor shall it be lower than the preset minimum global quota lower limit. This is to avoid adjusting the concurrency quota to an unreasonable range under extreme load scenarios, prevent the extreme situation of no quota available for allocation leading to complete task blocking, or excessive quota causing complete cluster overload, and further ensure the robustness of the scheduling logic.
[0028] In one implementation scenario, the health coefficient of a worker node represents the health status of the current working state of the worker node. It can be used to assess the reliability of the worker node in handling different numbers of concurrent tasks. The higher the value, the more stable the worker node is in operation and the more concurrent tasks it can handle. Conversely, the lower the value, the worse the current stability of the worker node is or the abnormal operation. It can only handle a relatively low number of concurrent tasks or cannot execute tasks.
[0029] In a specific implementation scenario, the health status indicators of a worker node include: node memory baseline, node load indicators, processor idle rate, node crawling status, and node restart time. The node memory baseline represents the baseline level of memory usage under normal operating conditions of the worker node, which may include the average and peak memory utilization rate within a preset time window. The node load indicator represents the number of task requests that the worker node needs to process per unit time and the average response latency, reflecting the current processing pressure of the node. It may include the average request queue length and request processing timeout rate within a preset time window. The processor idle rate is the proportion of the worker node's currently unused processor computing power to the total computing power. A higher idle rate indicates that the node currently has sufficient computing power redundancy. The node crawling status indicates whether the crawling unit can normally obtain running data from the worker node. Crawling failure indicates that there is an anomaly in the node or network link. The node restart time represents the time since the last abnormal restart of the node.
[0030] In one implementation scenario, the working status category of a working node is determined based on its health status indicators. The working status category is used to distinguish whether the working node can provide stable computing services normally. For example, the working node can be divided into three status categories: normal operation, minor abnormality, and severe abnormality.
[0031] In a specific implementation scenario, corresponding normal ranges and abnormal thresholds are set for each health status indicator. If all indicators are within the normal range, the work node is determined to be in normal operation. If only a few indicators exceed the normal threshold but do not trigger the severe abnormality judgment condition, the work node is determined to be in a slightly abnormal state. If many indicators exceed the normal threshold, or any indicator triggers the severe abnormality threshold, the work node is determined to be in a severely abnormal state. After obtaining the status category of the work node, the health coefficient of the work node is obtained by fitting the deviation between the status category and the indicators of the current node. For example, the health coefficient range of the work node in normal operation is (0.8, 1], the health coefficient range of the work node in slightly abnormal state is (0.3, 0.8], and the health coefficient range of the work node in severely abnormal state is [0, 0.3]. The greater the deviation of the indicators, the smaller the health coefficient value. The health coefficient of the work node that is severely abnormal and completely unable to undertake tasks is directly set to 0.
[0032] In a specific implementation scenario, when the load-per-cpu (average load per CPU) health status metric of a worker node exceeds a first preset value and the CPU idle rate exceeds a second preset value, the worker node is in a D-state process, generating high load but not consuming CPU. The worker node's working state is determined to be a file system deadlock, and a health coefficient of 0 is assigned to the deadlocked worker node, effectively blocking it. This scheme, by assigning a health coefficient of 0, can promptly isolate worker nodes exhibiting deadlock anomalies, preventing the allocation of new tasks to these nodes, preventing the anomaly from spreading and affecting the task execution efficiency of the entire cluster, and further improving the reliability of global task scheduling.
[0033] In a specific implementation scenario, in response to the worker node's working status category being node restart, a dynamically increasing health coefficient is assigned to the worker node during the first working period after a successful restart. The maximum value of the dynamically increasing health coefficient is 1. The first working period can be set to 5 minutes, 10 minutes, or 30 minutes, etc., and the specific value can be configured based on the average execution time of the model inference task and the node startup time. The initial health coefficient after restart is set according to the type of exception before the node restart. If it is a regular routine restart, the initial health coefficient is set to the first initial coefficient; if it is a forced restart after an exception, the initial health coefficient is set to the second initial coefficient. Afterward, the health coefficient increases by a preset step size every unit scheduling cycle until it reaches the maximum value of 1. This scheme, by gradually increasing the health coefficient of the node that has just completed the restart and gradually assigning more tasks, can avoid performance fluctuations caused by the node taking on too many tasks in the initial stage of restart before it has fully entered a stable operating state. It helps the node smoothly recover its ability to take on tasks and reduces the impact of node restart on the overall cluster task scheduling stability.
[0034] In a specific implementation scenario, the dynamically increasing health coefficient allocated to the worker node can exhibit a linear growth trend or gradually increase according to an exponential growth trend. The specific growth curve can be pre-configured according to the task type and node recovery characteristics of the target cluster to adapt to the recovery rhythm requirements after node restart under different business scenarios.
[0035] In a specific implementation scenario, in response to a work status category where the running load exceeds the load threshold, a health coefficient is assigned to the worker node based on a preset ratio that decreases iteratively, with the minimum value of the iteratively decreased health coefficient being the preset minimum coefficient. This scheme, when a node's running load continuously exceeds the safety threshold, can gradually reduce the concurrent tasks handled by the node by progressively lowering the health coefficient. This provides sufficient load fallback buffer space for the node and prevents sudden load drops in a short period from causing significant fluctuations in the overall task scheduling of the cluster, achieving a smooth load transfer and gradually bringing the node's load back to a safe range. This avoids nodes triggering crashes due to prolonged overload and maintains the stability of global scheduling.
[0036] In a specific implementation scenario, if a worker node's workload consistently exceeds a load threshold within a preset time period and the worker node becomes aged, its working state is determined to be "load above the load threshold." The worker node's health coefficient is then reduced by a preset percentage over a preset time step. For example, the preset percentage might be set to 0.7, until the worker node's workload falls below the load threshold or its health coefficient drops to a preset minimum. This preset minimum coefficient can be dynamically set based on the worker node's aging level; the higher the aging level, the smaller the preset minimum coefficient, preventing aging nodes from consuming excessive task quotas and slowing down the overall cluster efficiency. This preset minimum coefficient can then be used as the default health coefficient for that worker node.
[0037] In a specific implementation scenario, after determining the working node's status category based on its health status indicators, if the working node's status category is determined to be a file system deadlock, a deadlock detection signal is output. Based on this signal, a node restart is performed on the deadlocked node. Specifically, a kernel-level forced restart is executed. After a successful restart, the process returns to assigning a health coefficient to the working node based on its status category. This solution automatically detects and triggers the restart operation for file system deadlock nodes, eliminating the need for manual troubleshooting and restarting by operations personnel. This reduces the negative impact of deadlock failures on cluster task scheduling efficiency and further improves the automation level of cluster fault handling.
[0038] In a specific implementation scenario, after a node restart fails, a health coefficient of 0 is directly assigned to the working node to complete the node blocking, and the abnormal alarm is pushed to the operation and maintenance management platform to notify the operation and maintenance personnel to intervene and handle it.
[0039] In a specific implementation scenario, after a worker node successfully restarts, a reset signal is generated to identify that the worker node has performed a node restart. In response to the reset signal, the working status category of the worker node is determined to be node restart. During the first working period after the worker node successfully restarts, a dynamically increasing health coefficient is assigned to the worker node, and the maximum value of the dynamically increasing health coefficient is 1. This enables the load to ramp up in a short time after the node restarts, preventing the restarted node from being judged as a file system deadlock again due to instantaneous peak values.
[0040] In a specific implementation scenario, if the health status indicator of a worker node cannot be obtained, it indicates that the worker node or its corresponding network link is unable to communicate and cannot achieve normal data interaction. Therefore, the worker node is determined to be disconnected and offline, and a health coefficient of 0 is assigned to the worker node.
[0041] In a specific implementation scenario, when the working status category of a worker node is offline, a connectivity probe can be performed on the offline worker node at a preset time interval. If the probe results are all connection failures after a preset number of consecutive probes, the operation and maintenance personnel are notified to check the worker node. If the probe results are successful, the health status indicators of the worker node are re-acquired, the working status category is re-determined and the corresponding health coefficient is assigned to restore the worker node's ability to undertake tasks.
[0042] Step S12: Based on the global concurrency quota and the health coefficient of the worker node, obtain the target concurrency quota for the worker node.
[0043] In one implementation scenario, the global concurrency quota is the upper limit of the total number of tasks that can be executed simultaneously by all worker nodes in the target cluster. Based on this upper limit, the quota is allocated according to the health coefficient ratio of all worker nodes. Nodes with better health status are allocated more concurrency quota, while nodes with poor health status are allocated less or no concurrency quota. Tasks are prioritized to worker nodes with stable operation and sufficient computing power, fully tapping the computing power potential of high-quality nodes, while controlling the task allocation scale of abnormal nodes to ensure the stability of the overall cluster operation.
[0044] In a specific implementation scenario, the target concurrency quota is obtained by multiplying the global concurrency quota by a health coefficient, thus coupling global load adaptation with the local health status of nodes. Specifically, for any worker node in the target cluster, a health weight is obtained based on the ratio of the worker node's health coefficient to the sum of the health coefficients of all worker nodes in the target cluster. The target concurrency quota for the worker node is then obtained by multiplying the global concurrency quota by the health weight.
[0045] In a specific implementation scenario, a first predicted quota is obtained based on the global concurrency quota and the health coefficient of the worker nodes. This first predicted quota represents the predicted available concurrency of the worker nodes under the current global concurrency quota of the target cluster. A second predicted quota is obtained based on the hardware data of the worker nodes. This second predicted quota represents the maximum concurrency that the worker nodes can handle based on their own hardware performance. Specifically, the hardware data of the worker nodes is the node memory baseline, and the second predicted quota is a static upper limit calculated based on the node memory baseline. The smaller value between the first and second predicted quotas is taken as the target concurrency quota. This scheme balances dynamic global load adaptation and single-node hardware capability constraints, further ensuring the stability of single-node operation.
[0046] In a specific implementation scenario, the second predicted quota is a static upper limit calculated based on the hardware data of the worker node, specifically including physical resource limitations such as the node's memory capacity and the number of CPU cores. This is converted into an upper limit for the number of concurrent tasks that can be executed. Specifically, the second predicted quota can be obtained by looking up a preset hardware configuration table, or by calculating it by dividing the memory size by the average memory usage per task using a formula.
[0047] Step S13: Based on the target concurrency quota of the worker node, issue a concurrency quota lease to the worker node, and schedule the tasks to be executed on the worker node based on the concurrency quota lease of the worker node.
[0048] In one implementation scenario, concurrency quota leases are issued based on node permission tokens. These tokens contain the target concurrency quota for worker nodes. Worker nodes apply for corresponding task execution quotas by holding these tokens. After a task is completed and the quota is released, a new quota can be applied for based on the token, enabling dynamic and cyclical reuse of concurrency quotas. When the health coefficient of a worker node is updated or the global concurrency quota is adjusted, causing a change in the target concurrency quota, the quota value in the node permission token is simultaneously refreshed. This ensures that the executable concurrency scale of the worker node adapts to the latest cluster operating status and node health status in a timely manner, guaranteeing the real-time nature of quota scheduling.
[0049] In a specific implementation scenario, node permission tokens can be issued through atomic operations in a single thread within shared storage, ensuring consistency in the acquisition of node permission tokens by multiple worker nodes in the target cluster.
[0050] In one implementation scenario, the concurrent quota lease is set with an effective duration. Unused quotas that exceed the effective duration will be automatically reclaimed by the global scheduling center and redistributed to the work nodes that need to perform tasks, thus avoiding long-term quota occupation and waste and improving the utilization rate of global concurrent quotas.
[0051] In a specific implementation scenario, based on a preset period, the historical concurrent quotas allocated to work nodes whose lease time has expired in the target cluster are reclaimed, and the reclaimed historical concurrent quotas are released into the callable quota of the current global concurrent quota of the target cluster. This avoids the quotas being occupied by completed or terminated tasks for a long time, allowing the limited global concurrent quotas to support the execution of more effective tasks and further improve the overall utilization rate of the cluster's computing resources.
[0052] In one implementation scenario, in response to a scheduling request from a task to be executed, it is determined whether the global concurrency quota is greater than the globally occupied quota. If the global concurrency quota is greater than the globally occupied quota, based on the concurrency quota lease of the worker node, it is determined whether the remaining concurrency quota of the worker node meets the quota requirement of the task to be executed. If the remaining concurrency quota meets the requirement, the concurrency quota in the concurrency quota lease of the worker node is deducted, and the task to be executed is allowed to start. The above scheme, through two-layer quota verification, first controls the total concurrency at the global level, and then determines whether the task can be issued based on the remaining available quota of a single node. This ensures that the overall load of the cluster does not exceed the control limit, and also ensures that no single worker node will take on tasks beyond its capacity, thus ensuring the stability of the cluster operation from the global level to the node level.
[0053] In a specific implementation scenario, when the remaining concurrency quota is insufficient to meet the quota requirements of the task to be executed, the task is refused to be issued to the worker node and is transferred to the waiting queue. The task will be re-initiated after the worker node completes the execution of the existing task and releases the quota. When the global concurrency quota is not greater than the global quota already occupied, all new task scheduling requests are directly added to the waiting queue. The scheduling requests will be processed in the order of their requests after the existing task is completed and the quota is released.
[0054] In one implementation scenario, the task to be executed by the worker node is an agent reasoning task driven by a large language model. It is deployed on a large scale by relying on a distributed cluster, and the same cluster can carry parallel reasoning interaction requests from multiple different agents.
[0055] In a specific implementation scenario, the tasks to be executed include image generation tasks. Specifically, image generation tasks rely on worker nodes to call image generation models for inference and generation. In the task concurrency scheduling method of this application, the target concurrency quota can be dynamically allocated according to the current health status of each worker node, and the image generation tasks can be centrally scheduled to worker nodes with good health status and sufficient remaining hardware resources for execution. This avoids the exception triggered by a single node due to too many concurrent requests, and also makes full use of the idle computing resources of each node, ensuring the overall execution efficiency and operational stability of the image generation tasks.
[0056] In another specific implementation scenario, the tasks to be executed include session inference tasks. Specifically, when the target cluster receives several session inference tasks, it distributes tasks according to the current global concurrency quota of the target cluster and the target concurrency quota of each worker node, prioritizing the allocation of new session inference requests to worker nodes with sufficient remaining concurrency quota. The worker nodes execute the corresponding session inference tasks based on the concurrency quota lease, complete the dialogue inference by calling the output results of the corresponding large model, and return the inference results to the request initiator.
[0057] In another specific implementation scenario, the tasks to be executed include multimodal large-scale model cross-modal generation tasks. These tasks rely on each worker node to call the corresponding large model to process multiple computational steps such as text encoding, image decoding, and feature alignment. Using the task concurrency scheduling method of this application, corresponding concurrency quotas can be dynamically allocated based on node health coefficients. Cross-modal generation tasks of different priorities can be assigned to worker nodes with corresponding health statuses for execution. This ensures that high-priority tasks obtain sufficient computing resources while preventing low-priority tasks from occupying the computing quotas of healthy, high-quality nodes, achieving dynamic matching of computing resources and task requirements, and further improving the task scheduling adaptability of the distributed cluster.
[0058] It should be noted that the above embodiments are only possible task types for the tasks to be executed, and the specific task types that the target cluster can execute are not limited in this application.
[0059] Please see Figure 3 , Figure 3 This is a schematic diagram of the framework of an embodiment of the multi-level token bucket module in the task concurrent scheduling method of this application. In one implementation scenario, the multi-level token bucket module includes a token issuance unit, a lease record unit, and a lease reclamation unit. The token issuance unit is used to generate concurrent quota tokens according to the target concurrent quota of the worker nodes. The lease record unit is used to record the status information such as the effective duration, occupied quota, and remaining available quota of the concurrent quota leases allocated to each worker node based on the concurrent quota tokens. The lease reclamation unit periodically scans all lease records, automatically reclaims idle quotas that have exceeded the effective duration, puts the reclaimed quotas back into the global quota pool, and updates the lease records.
[0060] Please see Figure 4 , Figure 4This is a schematic diagram of the framework of an embodiment of the task concurrent scheduling system of this application. In one implementation scenario, the specific implementation steps of the aforementioned task concurrent scheduling method are implemented based on the task concurrent scheduling system. Specifically, the task concurrent scheduling system includes a utilization detection module, an adaptive rate limiting module, a health coefficient calculation module, a two-layer admission decision module, a multi-level token bucket module, a task dispatch module, and a deadlock node self-healing module. The utilization detection module is used to generate the current resource utilization of the target cluster. The adaptive rate limiting module is used to adjust the global concurrent quota of the target cluster based on the current resource utilization of the target cluster. The health coefficient calculation module is used to calculate the health coefficient of the worker nodes in the target cluster. The system employs a health status indicator to obtain the health coefficient of worker nodes. A two-layer admission decision module calculates the target concurrent quota for worker nodes based on the global concurrent quota and the worker node's health coefficient. A multi-level token bucket module issues concurrent quota leases to worker nodes based on their target concurrent quotas. A task dispatch module schedules tasks to be executed on worker nodes based on their concurrent quota leases. A deadlock self-healing module automatically triggers node restarts upon detecting a file system deadlock and re-executes the health coefficient calculation and quota allocation process on the restarted node. The processing logic of each module in the task concurrency scheduling system can be found in the detailed implementation steps of the aforementioned task concurrency scheduling method; for simplicity, it will not be elaborated further here.
[0061] In a specific implementation scenario, the adjustment of global concurrency quotas relies on real-time monitoring of the target cluster's current resource utilization. Specifically, the system periodically captures monitoring metrics from the model inference gateway, calculates the overall cluster utilization scalar, and dynamically adjusts it using an additive increase and multiplicative decrease strategy. When the utilization exceeds a preset high threshold, the concurrency limit is quickly reduced by multiplying by a factor less than 1 to alleviate service pressure; when the utilization falls below the threshold, the concurrency quota is gradually restored by increasing a fixed step size, thus adapting to dynamic load changes. Secondly, the health coefficient of worker nodes is obtained based on multi-dimensional health status indicators. These indicators include node memory baseline, node load indicators, CPU idle rate, monitoring capture status, and restart time. The system classifies and judges faults according to their types: if file system deadlock characteristics are detected (such as high load but high CPU idle rate), the health coefficient is determined to be 0, and the node is blocked; if the node is in a restart ramp-up period, the health coefficient is linearly increased over time; if the node runs at high load for a long period, the health coefficient is reduced proportionally; if monitoring is unavailable, the health coefficient is determined to be 0. The health coefficient is a real number ranging from 0 to 1. Next, the target concurrency quota is obtained by multiplying the global concurrency quota by the health coefficient; this multiplication operation couples global load adaptation with the local health status of nodes. Finally, concurrency quota leases are issued based on the target concurrency quota, and a lease lifecycle is set for automatic recycling. In the lease mechanism, the grant time is recorded and an expiration time is set. When a lease expires or a node exits abnormally, the system automatically deletes the lease record, allowing the concurrency quota to flow back to the global pool.
[0062] The above scheme adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed on the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution.
[0063] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of an embodiment of the task concurrency scheduling device of this application. The task concurrency scheduling device 50 includes: an acquisition module 51, a generation module 52, and a scheduling module 53. The acquisition module 51 is used to adjust the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, and to obtain the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster. The generation module 52 is used to obtain the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes. The scheduling module 53 is used to issue concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and to schedule the tasks to be executed by the worker nodes based on the concurrency quota leases of the worker nodes.
[0064] In the above scheme, the task concurrency scheduling device 50 adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed of the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution.
[0065] In some disclosed embodiments, the acquisition module 51 further includes a status category judgment module (not shown), used to determine the working status category of the working node based on the health status index of the working node; the acquisition module 51 also includes a health coefficient allocation module (not shown), used to allocate a health coefficient to the working node based on the working status category of the working node.
[0066] In some disclosed embodiments, the health coefficient allocation module (not shown) further includes a first allocation submodule (not shown), used to allocate a health coefficient of 0 to the worker node in response to a working state category of file system deadlock; the health coefficient allocation module (not shown) further includes a second allocation submodule (not shown), used to allocate a dynamically increasing health coefficient to the worker node within a first working period after the worker node successfully restarts in response to a working state category of node restart, and the maximum value of the dynamically increasing health coefficient is 1; the health coefficient allocation module (not shown) further includes a third allocation submodule (not shown), used to allocate a health coefficient to the worker node based on a preset ratio that decreases iteratively in response to a working state category of running load exceeding a load threshold, and the minimum value of the iteratively decreased health coefficient is a preset minimum coefficient.
[0067] In some disclosed embodiments, after determining the working status category of a working node based on its health status indicators, the task concurrency scheduling device 50 further includes a node restart module (not shown), which is used to restart the working node in response to a file system deadlock, and after a successful restart, return to the step of assigning a health coefficient to the working node based on its working status category.
[0068] In some disclosed embodiments, after a worker node restarts successfully, a reset signal is generated to identify that the worker node has performed a node restart. The health coefficient allocation module (not shown) also includes a restart allocation submodule (not shown), which, in response to the reset signal, determines that the working state category of the worker node is a node restart, and allocates a dynamically increasing health coefficient to the worker node within the first working time after the worker node restarts successfully, and the maximum value of the dynamically increasing health coefficient is 1.
[0069] In some disclosed embodiments, the acquisition module 51 further includes a comparison result acquisition module (not shown), used to acquire the comparison result between the current resource utilization rate and the utilization rate threshold; the acquisition module 51 further includes a quota reduction module (not shown), used to reduce the global concurrent quota of the target cluster based on a multiplicative reduction factor in response to the current resource utilization rate being greater than the utilization rate threshold; the acquisition module 51 further includes a quota increase module (not shown), used to increase the global concurrent quota of the target cluster based on an additive increase step size in response to the current resource utilization rate not being greater than the utilization rate threshold.
[0070] In some disclosed embodiments, the generation module 52 further includes a quota prediction module (not shown), used to obtain a first predicted quota based on the global concurrency quota and the health coefficient of the worker node, and to obtain a second predicted quota based on the hardware data of the worker node; the generation module 52 further includes a quota determination module (not shown), used to take the smaller value between the first predicted quota and the second predicted quota as the target concurrency quota.
[0071] In some disclosed embodiments, the scheduling module 53 further includes a global quota judgment module (not shown), used to determine whether the global concurrent quota is greater than the global occupied quota in response to the scheduling request of the task to be executed; the scheduling module 53 further includes a remaining quota judgment module (not shown), used to determine whether the remaining concurrent quota of the work node meets the quota requirements of the task to be executed based on the concurrent quota lease of the work node in response to the global concurrent quota being greater than the global occupied quota; the scheduling module 53 further includes a task scheduling module (not shown), used to deduct the concurrent quota of the work node in the concurrent quota lease in response to the remaining concurrent quota meeting the requirements, and allow the task to be executed to start execution.
[0072] In some disclosed embodiments, the concurrent quota lease includes a lease validity period. The task concurrent scheduling device 50 also includes a periodic recycling module (not shown) for recycling the historical concurrent quota allocated to work nodes in the target cluster whose lease time has expired based on a preset period. The task concurrent scheduling device 50 also includes a quota release module (not shown) for releasing the recycled historical concurrent quota to the callable quota of the current global concurrent quota of the target cluster.
[0073] In some disclosed embodiments, the health status indicators of the worker nodes in the task concurrency scheduling device 50 include: node memory baseline, node load indicators, processor idle rate, node capture status, and node restart time; and / or, in response to the unavailability of the health status indicators of the worker nodes, a health coefficient of 0 is assigned to the worker nodes; and / or, the concurrency quota lease is issued based on the node permission token; wherein, the node permission token contains the target concurrency quota of the worker nodes.
[0074] Please see Figure 6 , Figure 6 This is a schematic diagram of a framework of an embodiment of the computer device of this application. The computer device 60 includes at least a memory 61 and a processor 62 coupled to each other. The memory 61 stores at least program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above-described task concurrency scheduling method embodiments. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here. Exemplarily, the computer device 60 may include, but is not limited to, smartphones, tablet computers, etc., and the specific type of the computer device 60 is not limited here.
[0075] Specifically, processor 62 controls itself and memory 61 to implement the steps in any of the above-described task concurrency scheduling method embodiments. Processor 62 can also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 62 can be implemented using integrated circuit chips.
[0076] In the above scheme, computer device 60 adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed on the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution.
[0077] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor. The program instructions 71 are used to implement the steps in any of the above-described embodiments of the task concurrency scheduling method.
[0078] In the above scheme, the computer-readable storage medium 70 adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed of the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution.
[0079] Please see Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of the computer program product of this application. The computer program product 80 includes a computer program / instructions 81, which, when executed by the processor 81, implements the steps in any of the above-described embodiments of the task concurrency scheduling method.
[0080] In the above scheme, the computer program product 80 adjusts the global concurrency quota of the target cluster based on the current resource utilization rate of the target cluster, obtains the health coefficient of the worker nodes based on the health status indicators of the worker nodes in the target cluster, obtains the target concurrency quota of the worker nodes based on the global concurrency quota and the health coefficient of the worker nodes, issues concurrency quota leases to the worker nodes based on the target concurrency quota of the worker nodes, and schedules the tasks to be executed on the worker nodes based on the concurrency quota leases of the worker nodes. On the one hand, dynamically adjusting the global concurrency quota based on the current resource utilization of the target cluster allows it to match the actual operating load of the cluster. This global concurrency quota also influences the overall task concurrency scale of the cluster, thus ensuring operational stability at the cluster level. On the other hand, by sensing the health status of worker nodes based on their health indicators and calculating corresponding health coefficients, and using these health coefficients as a reference, the target concurrency quota is allocated to each worker node based on the global concurrency quota. This allows healthy nodes with stable operating conditions and sufficient resource redundancy to undertake more tasks, while reasonably limiting the concurrency quotas of nodes with abnormal conditions or excessive loads. This ensures that concurrency allocation is simultaneously constrained by the overall cluster load and guided by the individual node health status, improving the accuracy and rationality of worker node concurrency access control. Therefore, it enhances the balance of hardware resource utilization in the target cluster, contributing to improved stability of cluster task execution.
[0081] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0082] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0083] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0084] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0086] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0087] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.
Claims
1. A task concurrent scheduling method, characterized in that, include: Based on the current resource utilization of the target cluster, adjust the global concurrency quota of the target cluster, and obtain the health coefficient of the worker node based on the health status index of the worker node in the target cluster. Based on the global concurrency quota and the health coefficient of the worker node, the target concurrency quota of the worker node is obtained; Based on the target concurrency quota of the work node, a concurrency quota lease is issued to the work node, and based on the concurrency quota lease of the work node, the tasks to be executed on the work node are scheduled.
2. The method according to claim 1, characterized in that, The process of obtaining the health coefficient of the working nodes based on the health status indicators of the working nodes in the target cluster includes: Based on the health status indicators of the work node, determine the work status category of the work node; The health coefficient is assigned to the work node based on its work status category.
3. The method according to claim 2, characterized in that, Assigning the health coefficient to the work node based on its work status category includes: In response to the working state category being file system deadlock, a health coefficient of 0 is assigned to the working node; In response to the work status category being node restart, during the first working period after the work node successfully restarts, a dynamically increasing health coefficient is assigned to the work node, and the maximum value of the dynamically increasing health coefficient is 1. In response to the working status category being that the operating load is higher than the load threshold, a health coefficient is assigned to the working node based on a preset ratio that is iteratively reduced, and the minimum value of the iteratively reduced health coefficient is a preset minimum coefficient.
4. The method according to claim 2, characterized in that, After determining the working status category of the working node based on its health status indicators, the method further includes: In response to the working state category being a file system deadlock, the worker node is restarted, and after a successful restart, the process returns to the step of assigning the health coefficient to the worker node based on its working state category.
5. The method according to claim 4, characterized in that, After the worker node restarts successfully, a reset signal is generated to identify that the worker node has performed a node restart. The process of assigning a health coefficient to the worker node based on its working state category includes: In response to the reset signal, the working state category of the working node is determined to be node restart. During the first working period after the working node restarts successfully, a dynamically increasing health coefficient is assigned to the working node, and the maximum value of the dynamically increasing health coefficient is 1.
6. The method according to claim 1, characterized in that, Adjusting the global concurrency quota of the target cluster based on its current resource utilization includes: Obtain the comparison result between the current resource utilization rate and the utilization rate threshold; In response to the current resource utilization rate being greater than the utilization rate threshold, the global concurrency quota of the target cluster is reduced based on a multiplicative subtraction factor; In response to the current resource utilization rate not being greater than the utilization rate threshold, the global concurrency quota of the target cluster is increased based on an additive increment step size.
7. The method according to claim 1, characterized in that, The step of obtaining the target concurrency quota for the worker node based on the global concurrency quota and the health coefficient of the worker node includes: Based on the global concurrency quota and the health coefficient of the worker node, a first predicted quota is obtained, and based on the hardware data of the worker node, a second predicted quota is obtained. The smaller value between the first predicted quota and the second predicted quota is taken as the target concurrent quota.
8. The method according to claim 1, characterized in that, Based on the concurrent quota lease of the worker node, the tasks to be executed on the worker node are scheduled, including: In response to the scheduling request of the task to be executed, determine whether the global concurrency quota is greater than the global quota already occupied; In response to the global concurrency quota being greater than the global already occupied quota, based on the concurrency quota lease of the worker node, it is determined whether the remaining concurrency quota of the worker node meets the quota requirement of the task to be executed; In response to the remaining concurrency quota meeting the demand, the concurrency quota of the work node in the concurrency quota lease is deducted, and the task to be executed is allowed to start execution.
9. The method according to claim 1, characterized in that, The concurrent quota lease includes a lease validity period, and the method further includes: Based on a preset period, the historical concurrent quotas allocated to the worker nodes in the target cluster whose leases have expired are reclaimed; The recovered historical concurrency quota is released into the callable quota of the current global concurrency quota of the target cluster.
10. The method according to any one of claims 1 to 9, characterized in that, The health status indicators of the working nodes include: node memory baseline, node load indicators, processor idle rate, node capture status, and node restart time. And / or, in response to the inability to obtain the health status indicator of the working node, assign a health coefficient of 0 to the working node; And / or, the concurrent quota lease is issued based on a node permission token; wherein the node permission token contains the target concurrent quota of the worker node.
11. A computer device, characterized in that, It includes at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor executes the program instructions to implement the task concurrency scheduling method according to any one of claims 1 to 10.
12. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the task concurrency scheduling method according to any one of claims 1 to 10.
13. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, implements the task concurrency scheduling method according to any one of claims 1 to 10.
