Task-driven computing resource dynamic allocation system

Abstract

The invention belongs to the technical field of computing resource allocation, and discloses a task-driven computing resource dynamic allocation system, which is characterized in that task differentiation features are extracted through a task feature intelligent analysis module, and global optimal allocation, 'task fingerprint 'matching and opportunity cost evaluation mechanisms are realized in combination with a reinforcement learning scheduling decision module. Mismatching conditions such as allocation of lightweight tasks to a high-video-memory GPU are avoided, meanwhile, the elastic resource pool dynamically adjusts the capacity, reduces resource locking during task pause, remarkably reduces resource waste, improves the overall utilization rate of computing resources such as the GPU, can quickly expand the capacity through the elastic resource pool to cope with burst tasks, and a cross-node resource cooperation module constructs a global view and improves the overall utilization rate of the computing resources. Efficient calling of idle resources is realized; a'logic super node 'and zero-aware migration technology breaks node barriers, avoids global redundancy caused by local optimum, enables resources to flexibly flow along with task loads, and improves the adaptive capacity of a system to dynamic tasks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computing resource allocation, and particularly relates to a task-driven computing resource dynamic allocation system. BACKGROUND

[0002] In the current computing resource allocation field, large model training, inference and other tasks have realized large-scale application. In the prior art, such as a static resource scheduling system of a certain public patent, a fixed threshold is used to divide a GPU resource pool, which cannot dynamically adjust the resource capacity according to the task type (training / inference), resulting in resource expansion delay when an inference task bursts. For example, a certain reinforcement learning scheduling related technology only takes the local node load as the optimization target and does not consider the global opportunity cost, and the problem of delay of a high-priority task due to resource occupation by a low-value task often occurs.

[0003] The specific defects of the traditional system include: insufficient resource matching accuracy, relying on static rule to allocate resources, such as that in some scheduling systems, training tasks and inference tasks share a fixed resource pool, resulting in that a training task with high memory demand is allocated to a low memory GPU; lack of dynamic response capability, the resource pool expansion threshold is fixed, such as that in a certain cluster management system, the resource pool expansion threshold is fixed as a fixed proportion, and the resource is idle when the inference task is in a low valley; inefficient cross-node cooperation, lack of global resource view in distributed scheduling, such as that in some distributed scheduling frameworks, node-level scheduling decisions easily lead to local optimization; weak fault self-healing mechanism, manual intervention is required to migrate tasks when a GPU fails, such as that in a certain resource management tool, manual operation is required for fault recovery, which seriously affects the continuity of large model training. SUMMARY

[0004] The purpose of the present application is to provide a task-driven computing resource dynamic allocation system to solve the problems raised in the background.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a task-driven computing resource dynamic allocation system, which comprises:

[0006] A task feature intelligent analysis module: based on deep learning bidirectional analysis of tasks, basic features such as type and computing intensity are extracted, and historical data models are used to predict resource demand fluctuations; the "task fingerprint" coding feature is introduced to realize fast matching of similar tasks and solve the problem of insufficient dynamic characteristics perception of tasks in traditional systems;

[0007] An elastic resource pool dynamic construction module: according to the features of the task feature intelligent analysis module, GPUs and other resources are abstracted into logical pools, the load is monitored in real time and the capacity is adjusted, the idle resources can be allocated across task pools, an adaptive expansion and contraction algorithm is designed, the sensitivity is adjusted according to the task type, the frequent expansion and contraction loss is avoided, and the resource pool is flexibly adapted to the dynamic changes of tasks;

[0008] Reinforcement learning scheduling decision module: Based on the resource pool built by the elastic resource pool dynamic construction module, a model is built with global resource utilization and task completion time as the objectives. The agent learns the mapping between task characteristics and allocation strategies, outputs the optimal solution, and introduces an opportunity cost assessment factor to avoid short-sighted decisions.

[0009] Cross-node resource collaboration module: After the reinforcement learning scheduling decision module determines the solution, a global resource view is built based on a distributed algorithm, the node status and task progress are synchronized, and when resources are insufficient, cross-node resources are aggregated into "logical super nodes". Through zero-perception migration technology, the task is smoothly transitioned, and the efficiency of cross-node allocation is improved.

[0010] Task lifecycle dynamic adaptation module: After the cross-node resource coordination module completes the cross-node allocation, it tracks the entire task process and adjusts resources according to the status. For example, it reduces the GPU ratio during the training and preprocessing stage, compresses the CPU quota during inference troughs, proposes a resource elasticity coefficient, and adjusts the weights according to the stage efficiency to allow resources to be tilted towards high-value stages.

[0011] Fault prediction and self-healing scheduling module: During task execution, it integrates hardware data and logs to predict GPU failures in advance through anomaly detection. In the event of a failure, it automatically hot-migrates tasks to backup resources, constructs a fault impact map, prioritizes the migration of high-priority tasks, reduces the impact of the failure, and ensures task continuity.

[0012] Closed-loop feedback optimization module: After collaboration with the preceding modules, it collects historical data to optimize algorithm parameters, visualizes resource allocation bottlenecks, and supports manual adjustment of strategy weights; it designs a strategy evolution engine and uses a multi-objective genetic algorithm to optimize rules, enabling the system to adapt to different task load scenarios.

[0013] Preferably, the task feature intelligent analysis module includes:

[0014] (1) Core functions and parsing logic: The input task is parsed bidirectionally through a deep learning model. On the one hand, it extracts a number of basic features such as task type (clearly indicating whether it is training or inference), computational intensity (specific FLOPS requirement), memory peak, and time sensitivity. On the other hand, it uses a prediction model trained from a historical task database to accurately predict the fluctuation of resource requirements during the execution of the task, such as the surge in GPU memory that may occur during the backpropagation stage of large model training.

[0015] Task type classification formula:

[0016] TaskType=Softmax(W·Features+b)

[0017] TaskType represents the final determined task type (training or inference, etc.), presented as a probability distribution vector, where each element in the vector corresponds to the probability of different task types;

[0018] W is the weight matrix, whose dimensions are adapted to the dimensions of the input features and the number of task types, and is used to perform weighted transformations on the input features;

[0019] Features are basic feature vectors extracted from the task, containing basic feature information such as task type (training / inference), computational intensity (FLOPS requirement), peak memory usage, and time sensitivity.

[0020] b is the bias vector used to adjust the result after the weighted transformation;

[0021] The Softmax function converts the weighted result into a probability distribution, ensuring that the sum of the probabilities of each task type is 1. For example, if there are two task types, after Softmax processing, the output vector may be [0.8, 0.2], indicating that the probability of the task being the first type is 0.8 and the probability of it being the second type is 0.2.

[0022] Formula source: Based on classification models in deep learning, such as neural network classification algorithms. When classifying tasks, it borrows the Softmax classification mechanism commonly used in image classification, text classification, and other fields. It determines the task type by performing a linear transformation on the task features and then normalizing them using the Softmax function.

[0023] (2) Innovative design and function: The concept of "task fingerprint" is introduced, and the various features obtained by parsing are encoded into a unique vector. This vector can be used to achieve rapid matching of similar tasks and advance prediction of resource requirements. It effectively solves the problem of insufficient perception of the dynamic characteristics of tasks in traditional systems, and allows resource allocation to closely match the actual needs of tasks from the initial stage.

[0024] Preferably, the elastic resource pool dynamic construction module includes:

[0025] (1) Resource pool construction logic: Based on the task features output by the task feature intelligent parsing module, a flexible resource infrastructure is built; it breaks the traditional fixed hardware partitioning mode and abstracts and integrates various resources such as GPU, CPU, and memory into a logical resource pool. By monitoring the load of physical resources in real time, such as GPU utilization and memory usage, the capacity of the resource pool is dynamically adjusted. When the number of inference tasks surges, the idle training task reserved resources can be automatically temporarily allocated to the inference resource pool. When the task load decreases, the resources will be released back to the original pool, realizing efficient resource flow.

[0026] Formula for dynamic adjustment of resource pool capacity:

[0027] C new =C old +ΔC·AdjustFactor

[0028] In the formula: C new To adjust the capacity of the resource pool, for example, for the GPU resource pool, it may represent the number of GPUs that can be allocated at present; for the memory resource pool, it may represent the total amount of memory that can be allocated at present.

[0029] C old This refers to the capacity of the resource pool before the adjustment.

[0030] ΔC represents the estimated change in resource pool capacity based on changes in task load. When inference tasks surge, ΔC is positive, indicating that the resource pool capacity needs to be increased; when task load decreases, ΔC is negative, indicating that the resource pool capacity can be reduced.

[0031] AdjustFactor is an adaptive adjustment factor with a value range of [0, 1]. This factor is determined by the "Resource Scaling Threshold Adaptive Algorithm" based on the task type. For example, the AdjustFactor value for inference tasks may be relatively large because inference tasks are more sensitive to resource changes and can respond to changes in resource demand more quickly. On the other hand, the AdjustFactor value for training tasks is relatively small to avoid frequent and excessive adjustments to the resource pool capacity.

[0032] Formula source: It is derived from the actual needs of dynamic resource allocation and adaptive control theory. In scenarios such as cloud computing and other dynamic resource allocation, in order to balance resource utilization efficiency and system stability, it is necessary to dynamically adjust the resource pool capacity according to the changes in task load. At the same time, considering the different sensitivity of different task types to resource changes, an adaptive adjustment factor is introduced to correct the amount of capacity change.

[0033] (2) Adaptive scaling mechanism: In order to ensure the stable operation of the resource pool, an "adaptive resource scaling threshold algorithm" is designed. This algorithm can dynamically adjust the scaling sensitivity according to the task type. For example, the trigger threshold of the inference task is lower than that of the training task, so as to avoid performance loss caused by frequent scaling. This adaptive mechanism allows the resource pool to flexibly adapt to the dynamic changes of the task.

[0034] Preferably, the reinforcement learning scheduling decision module includes:

[0035] (1) Decision Model and Objectives: After the resource pool is built by the dynamic construction module of the elastic resource pool, a reinforcement learning model is constructed with the dual objectives of "maximizing global resource utilization" and "minimizing task completion time". The agent in the model continuously interacts with the dynamic resource pool and learns the mapping relationship between different task characteristics and various resource allocation strategies, thereby having the ability to match appropriate resources for different tasks.

[0036] Reinforcement learning scheduling decision formula:

[0037] π(s t ) = argmax a Q(s t ,a;θ)

[0038] π(s t ) is in state s t The optimal strategy adopted by the agent is the optimal resource allocation action selected based on the current system state. For example, when facing a high-priority inference task, the strategy may instruct it to allocate a low-load node's GPU.

[0039] s t The system state at time t includes information such as the resource status of the resource pool (e.g., GPU utilization and memory usage of each node) and the characteristics of tasks in the task queue (task type, priority, resource requirements, etc.), which are represented in vector form.

[0040] 'a' represents the set of actions that the agent can take, such as allocating a certain resource of a specific node to a certain task.

[0041] Q(s t (a; θ) is the state-action value function, evaluating the state s. t After taking action a, the expected cumulative reward that can be obtained by following policy π from this state is θ; θ is the parameter of the Q function, which is continuously optimized through reinforcement learning. For example, if a low-load node GPU is allocated to a high-priority inference task, the overall resource utilization of the system is improved and the task completion time is shortened, and the Q function value will increase accordingly, indicating that the action is a better choice.

[0042] argmax a Find the value of Q(s) t The optimal strategy for an agent in the current state is the action a that yields the maximum expected cumulative reward, denoted as a (θ).

[0043] Formula source: Q-learning algorithm framework in reinforcement learning. In the resource allocation problem, the system state and resource allocation actions are constructed as the state space and action space of reinforcement learning. Through continuous trial and error learning, the agent finds the optimal resource allocation strategy that maximizes the system objectives (maximizing global resource utilization and minimizing task completion time).

[0044] (2) Optimize decision-making mechanism: This module can output the optimal resource allocation scheme in real time, such as allocating GPUs on nodes with low load to high-priority inference tasks, and scheduling cross-node resource groups for long-term training tasks; it introduces an "opportunity cost assessment factor" to quantify the potential losses that resource allocation may bring when making decisions, such as the cost of delaying task B after allocating a GPU to task A, thereby avoiding short-sighted decisions, ensuring that resource allocation reaches the optimal state at the global level, and giving full play to the efficiency of the resource pool.

[0045] Preferably, the cross-node resource collaboration module includes:

[0046] (1) Global resource view construction: After the reinforcement learning scheduling decision module determines the allocation scheme, this module is responsible for realizing the efficient allocation of resources among multiple nodes. Based on the distributed consensus algorithm, a global resource view is constructed, which can synchronize the resource status and task progress of each node in real time, so that the system has a clear understanding of the overall resource situation and provides comprehensive and timely information support for the reasonable allocation of cross-node resources.

[0047] (2) Cross-node allocation mechanism: When the resources of a single node cannot meet the task requirements, this module automatically triggers the cross-node resource aggregation function to virtualize some resources of multiple nodes into "logical super nodes" to meet the multi-card collaboration requirements of large model training; at the same time, a "resource migration zero-awareness mechanism" is designed to achieve a smooth transition of tasks during cross-node resource migration through memory snapshot and task state serialization technology, which effectively solves the problem of excessively long task migration time in traditional distributed scheduling and significantly improves the efficiency of cross-node resource collaboration.

[0048] Preferably, the task lifecycle dynamic adaptation module includes:

[0049] (1) Dynamic adjustment logic: After the cross-node resource coordination module completes the cross-node allocation of resources, the entire process of task execution is tracked in real time, and the resource allocation is flexibly adjusted according to the different states of the task. For example, when the training task enters the data preprocessing stage, the GPU allocation ratio will be automatically reduced and some resources will be released to the inference task because the dependence on GPU is reduced in this stage. When the inference task is in a low traffic period, its CPU quota will be compressed and resources will be tilted to the training task.

[0050] (2) Elasticity coefficient adjustment: The "resource elasticity coefficient" is proposed. Based on the resource utilization efficiency of the current stage of the task, such as the ratio of computation to resource occupancy, the allocation weight is automatically adjusted to ensure that resources can be tilted towards high-value stages. This realizes the dynamic optimization of resources during task execution, so that resource utilization is always synchronized with the actual needs of the task and improves the overall utilization efficiency of resources.

[0051] Formula for calculating resource elasticity coefficient:

[0052]

[0053] ElasticCoefficient is a resource elasticity coefficient used to measure the resource utilization efficiency of a task at its current stage, in order to determine the weight of resource allocation.

[0054] CurrentStageComputeVolume represents the computational load at the current stage of the task. For example, in the data preprocessing stage of a training task, the computational load may be reflected in the number of computational operations involved in data reading, cleaning, and transformation; in the model training stage, the computational load may be the number of floating-point operations during model parameter updates.

[0055] CurrentStageResourceUsage represents the resource usage at the current stage of the task, including the consumption of resources such as GPU, CPU, and memory. For example, in the data preprocessing stage of a training task, it may mainly be reflected in the consumption of CPU and memory; in the model training stage, GPU consumption becomes the main part.

[0056] Formula origin: To achieve dynamic resource optimization during task execution, this formula is constructed based on the definition of resource utilization efficiency. In research related to task scheduling and resource management, in order to rationally allocate resources according to the resource demand characteristics of different stages of a task, resource utilization efficiency-related indicators are often calculated to dynamically adjust resource allocation strategies. The resource elasticity coefficient here is a way to measure resource utilization efficiency.

[0057] Preferably, the fault prediction and self-healing scheduling module:

[0058] (1) Fault prediction mechanism: During the task execution process, hardware sensor data such as GPU temperature and power consumption are integrated with the task execution log. The improved anomaly detection model is used to identify potential GPU faults in advance, such as hardware faults or performance degradation, so as to buy time for fault handling and minimize the impact of faults.

[0059] (2) Self-healing and safeguard measures: When a fault is detected or a task is abnormally interrupted, the "resource hot migration" mechanism is automatically triggered to seamlessly switch the task to the backup resource. At the same time, the fault node information is recorded in detail and isolated to prevent the fault from spreading. In addition, a "fault impact map" is constructed to calculate the impact range of a single node fault on the global task in advance. When migrating tasks, high-priority tasks are prioritized to minimize the impact of the fault on task execution and ensure the continuity of tasks.

[0060] Formula for assessing the scope of fault impact:

[0061]

[0062] ImpactRange is a quantitative value representing the impact range of a faulty node on the global task. The larger the value, the more severe the impact of the fault on the execution of system tasks.

[0063] RelatedTasks is a set of tasks associated with the failed node. These tasks may be affected because they depend on the failed node's resources or data, or because they are in the same task chain.

[0064] Priority i This represents the priority of task i, and its value is usually a positive integer. The higher the priority, the larger the value. For example, high-priority inference tasks may have high requirements for system real-time performance, so their priority values ​​are set to be larger. On the other hand, some non-critical background training tasks have relatively smaller priority values.

[0065] DependencyDegree i,FailedNode DependencyDegree represents the degree of dependence of task i on the faulty node, with a value ranging from [0, 1]. For example, if task i depends entirely on the faulty node to provide data or computing resources in order to run, then DependencyDegree is 1. i,FailedNode =1; if task i is not related to the faulty node, then DependencyDegree is 1. i,FailedNode =0; the degree of dependency is usually determined based on factors such as the amount of data transferred between tasks and nodes, and the relationship between computing resource requests.

[0066] Formula source: Based on the theory of fault propagation and impact analysis, in the research of distributed system fault management, in order to quickly assess the impact of node failure on the execution of tasks in the entire system, the scope of the fault impact is often quantified by considering the priority of the affected tasks and their dependence on the faulty node, thereby providing a basis for task migration and fault recovery strategy formulation.

[0067] Preferably, the closed-loop feedback optimization module includes:

[0068] (1) Data-driven optimization: Through the collaborative work of the previous modules, historical data on resource allocation is collected, including GPU utilization, task completion time, fault recovery time, etc. The parameters of the scheduling algorithm are continuously optimized through offline training. At the same time, a visual interface is provided to intuitively display the efficiency bottlenecks of resource allocation, such as the average waiting time of a certain type of task being too long, etc., and the administrator can manually adjust the strategy weight according to the actual situation.

[0069] (2) Strategy Evolution Mechanism: In order to enable the system to adapt to different task load scenarios, this module is designed with a "strategy evolution engine". The engine continuously optimizes the scheduling rules based on a multi-objective genetic algorithm, so that the system can maintain good performance under various task load scenarios such as peak training period and burst inference period, and promote the system to continuously adapt to new task requirements and resource environment, and achieve long-term stable and efficient operation.

[0070] The beneficial effects of this invention are as follows:

[0071] 1. This invention extracts differentiated features of tasks through a task feature intelligent analysis module, and achieves global optimal allocation by combining a reinforcement learning scheduling decision module. The "task fingerprint" matching and opportunity cost assessment mechanism avoids mismatches such as allocating high-memory GPUs to lightweight tasks. At the same time, the elastic resource pool dynamically adjusts its capacity to reduce resource locking when tasks are paused, significantly reducing resource waste and improving the overall utilization of computing resources such as GPUs.

[0072] 2. This invention enables rapid expansion of the elastic resource pool to cope with sudden tasks, and the cross-node resource collaboration module constructs a global view to achieve efficient use of idle resources; the "logical super node" and zero-perception migration technology break down node barriers, avoid global redundancy caused by local optima, and allow resources to flow flexibly with task load, improving the system's adaptability to dynamic tasks.

[0073] 3. This invention identifies potential faults in advance through a fault prediction and self-healing scheduling module, and automatically hot-migrates tasks to backup resources; the fault impact map prioritizes high-priority tasks, reduces manual intervention, shortens recovery time, and continuously improves system stability by combining a closed-loop feedback optimization module, ensuring the continuity of large model training and inference, and reducing operation and maintenance costs. Attached Figure Description

[0074] Figure 1 This is a flowchart of the task-driven dynamic allocation system for computing resources according to the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] like Figure 1 As shown, this embodiment of the invention provides a task-driven dynamic allocation system for computing resources, the system comprising:

[0077] Intelligent task feature parsing module: Based on deep learning bidirectional parsing of tasks, it extracts basic features such as type and computational intensity, and uses historical data models to predict fluctuations in resource demand; it introduces "task fingerprint" encoding features to achieve rapid matching of similar tasks, solving the problem of insufficient perception of dynamic characteristics of tasks in traditional systems;

[0078] Elastic Resource Pool Dynamic Construction Module: Based on the characteristics of the task, the module intelligently analyzes the features of the module, abstracts resources such as GPUs into logical pools, monitors the load in real time and adjusts the capacity. During idle time, resources can be allocated across task pools. An adaptive scaling algorithm is designed to adjust the sensitivity according to the task type, avoid the loss of frequent scaling, and ensure that the resource pool can flexibly adapt to the dynamic changes of tasks.

[0079] Reinforcement learning scheduling decision module: Based on the resource pool built by the elastic resource pool dynamic construction module, a model is built with global resource utilization and task completion time as the objectives. The agent learns the mapping between task characteristics and allocation strategies, outputs the optimal solution, and introduces an opportunity cost assessment factor to avoid short-sighted decisions.

[0080] Cross-node resource collaboration module: After the reinforcement learning scheduling decision module determines the solution, a global resource view is built based on a distributed algorithm, the node status and task progress are synchronized, and when resources are insufficient, cross-node resources are aggregated into "logical super nodes". Through zero-perception migration technology, the task is smoothly transitioned, and the efficiency of cross-node allocation is improved.

[0081] Task lifecycle dynamic adaptation module: After the cross-node resource coordination module completes the cross-node allocation, it tracks the entire task process and adjusts resources according to the status. For example, it reduces the GPU ratio during the training and preprocessing stage, compresses the CPU quota during inference troughs, proposes a resource elasticity coefficient, and adjusts the weights according to the stage efficiency to allow resources to be tilted towards high-value stages.

[0082] Fault prediction and self-healing scheduling module: During task execution, it integrates hardware data and logs to predict GPU failures in advance through anomaly detection. In the event of a failure, it automatically hot-migrates tasks to backup resources, constructs a fault impact map, prioritizes the migration of high-priority tasks, reduces the impact of the failure, and ensures task continuity.

[0083] Closed-loop feedback optimization module: After collaboration with the preceding modules, it collects historical data to optimize algorithm parameters, visualizes resource allocation bottlenecks, and supports manual adjustment of strategy weights; it designs a strategy evolution engine and uses a multi-objective genetic algorithm to optimize rules, enabling the system to adapt to different task load scenarios.

[0084] The task feature intelligent analysis module includes:

[0085] (1) Core functions and parsing logic: The input task is parsed bidirectionally through a deep learning model. On the one hand, it extracts a number of basic features such as task type (clearly indicating whether it is training or inference), computational intensity (specific FLOPS requirement), memory peak, and time sensitivity. On the other hand, it uses a prediction model trained from a historical task database to accurately predict the fluctuation of resource requirements during the execution of the task, such as the surge in GPU memory that may occur during the backpropagation stage of large model training.

[0086] The historical task database contains 5 task types (image training, text reasoning, etc.), with ≥10,000 samples in each type; the model training uses the cross-entropy loss function and the Adam optimizer (learning rate 0.001), and training stops when the accuracy of the validation set stabilizes above 90% after 50 iterations;

[0087] The deep learning model adopts a BiLSTM (Bidirectional Long Short-Term Memory) architecture, with three hidden layers, each containing 256 neurons. The input features are encoded in 8 dimensions, for a total input dimension of 128 (16 feature classes × 8 dimensions). The input layer dimensions include multiple features such as task type and computational intensity, and each feature class is encoded. The hidden layers have multiple layers, each containing a certain number of neurons, and the activation function is ReLU. The output layer is connected to a Softmax classifier. The historical task database contains a large number of training / inference task records, covering resource requirement data of different models at various stages. The prediction model uses a regressor to predict fluctuations in resource requirements such as memory fluctuations.

[0088] Task type classification formula:

[0089] TaskType=Softmax(W·Features+b)

[0090] TaskType represents the final determined task type (training or inference, etc.), presented as a probability distribution vector, where each element in the vector corresponds to the probability of different task types;

[0091] W is the weight matrix, whose dimensions are adapted to the dimensions of the input features and the number of task types, and is used to perform weighted transformations on the input features;

[0092] Features are basic feature vectors extracted from the task, containing basic feature information such as task type (training / inference), computational intensity (FLOPS requirement), peak memory usage, and time sensitivity.

[0093] b is the bias vector used to adjust the result after the weighted transformation;

[0094] The Softmax function converts the weighted result into a probability distribution, ensuring that the sum of the probabilities of each task type is 1. For example, if there are two task types, after Softmax processing, the output vector may be [0.8, 0.2], indicating that the probability of the task being the first type is 0.8 and the probability of it being the second type is 0.2.

[0095] Formula source: Based on classification models in deep learning, such as neural network classification algorithms. When classifying tasks, it borrows the Softmax classification mechanism commonly used in image classification, text classification, and other fields. It determines the task type by performing a linear transformation on the task features and then normalizing them using the Softmax function.

[0096] (2) Innovative design and function: The concept of "task fingerprint" is introduced, and the various features obtained by parsing are encoded into a unique vector. This vector can be used to achieve rapid matching of similar tasks and advance prediction of resource requirements. It effectively solves the problem of insufficient perception of the dynamic characteristics of tasks in traditional systems, and allows resource allocation to closely match the actual needs of tasks from the initial stage.

[0097] The task fingerprint is encoded using a 128-dimensional embedding vector, where the first 32 dimensions map the task type and computation intensity (One-Hot + normalized), and the last 96 dimensions are generated by the output of the BiLSTM hidden layer. The encoding mapping table is stored locally in the system and supports matching when the cosine similarity of fingerprints of the same type is ≥0.9.

[0098] The dynamic construction module for the elastic resource pool includes:

[0099] (1) Resource pool construction logic: Based on the task features output by the task feature intelligent parsing module, a flexible resource infrastructure is built; it breaks the traditional fixed hardware partitioning mode and abstracts and integrates various resources such as GPU, CPU, and memory into a logical resource pool. By monitoring the load of physical resources in real time, such as GPU utilization and memory usage, the capacity of the resource pool is dynamically adjusted. When the number of inference tasks surges, the idle training task reserved resources can be automatically temporarily allocated to the inference resource pool. When the task load decreases, the resources will be released back to the original pool, realizing efficient resource flow.

[0100] GPU utilization is sampled every second, and the average of 5 consecutive samples is taken as the load value; memory usage is statistically analyzed at the process level, excluding cache usage (only the actual usage is calculated);

[0101] Formula for dynamic adjustment of resource pool capacity:

[0102] C new =C old +ΔC·AdjustFactor

[0103] In the formula: C new To adjust the capacity of the resource pool, for example, for the GPU resource pool, it may represent the number of GPUs that can be allocated at present; for the memory resource pool, it may represent the total amount of memory that can be allocated at present.

[0104] C old This refers to the capacity of the resource pool before the adjustment.

[0105] ΔC represents the estimated change in resource pool capacity based on changes in task load. When inference tasks surge, ΔC is positive, indicating that the resource pool capacity needs to be increased; when task load decreases, ΔC is negative, indicating that the resource pool capacity can be reduced.

[0106] AdjustFactor is an adaptive adjustment factor with a value range of [0, 1]. This factor is determined by the "Resource Scaling Threshold Adaptive Algorithm" based on the task type. For example, the AdjustFactor value for inference tasks may be relatively large because inference tasks are more sensitive to resource changes and can respond to changes in resource demand more quickly. On the other hand, the AdjustFactor value for training tasks is relatively small to avoid frequent and excessive adjustments to the resource pool capacity.

[0107] Formula source: It is derived from the actual needs of dynamic resource allocation and adaptive control theory. In scenarios such as cloud computing and other dynamic resource allocation, in order to balance resource utilization efficiency and system stability, it is necessary to dynamically adjust the resource pool capacity according to the changes in task load. At the same time, considering the different sensitivity of different task types to resource changes, an adaptive adjustment factor is introduced to correct the amount of capacity change.

[0108] (2) Adaptive scaling mechanism: In order to ensure the stable operation of the resource pool, an "adaptive resource scaling threshold algorithm" is designed. This algorithm can dynamically adjust the scaling sensitivity according to the task type. For example, the trigger threshold of the inference task is lower than that of the training task, so as to avoid performance loss caused by frequent scaling. This adaptive mechanism allows the resource pool to flexibly adapt to the dynamic changes of the task.

[0109] The specific steps of the adaptive scaling algorithm are as follows:

[0110] Calculate the task type fluctuation coefficient σ: σ = standard deviation of task load / mean of task load;

[0111] Determine the AdjustFactor: The AdjustFactor is calculated based on σ, and there are differences between the AdjustFactor for the training task and the inference task;

[0112] Dynamic threshold adjustment: When the resource load exceeds or falls below the threshold calculated based on AdjustFactor for multiple consecutive sampling periods (sampling once at a fixed time interval), expansion or contraction is triggered respectively.

[0113] The expansion threshold is 80% × AdjustFactor, and the reduction threshold is 30% × AdjustFactor. The typical value of σ for inference tasks is 0.2 (with small load fluctuations), and AdjustFactor = 1 - 0.2 × 0.5 = 0.9. Therefore, the expansion threshold is 72%, and the reduction threshold is 27%. The typical value of σ for training tasks is 0.6, and AdjustFactor = 0.7. Therefore, the expansion threshold is 56%, and the reduction threshold is 21%.

[0114] The reinforcement learning scheduling decision module includes:

[0115] (1) Decision Model and Objectives: After the resource pool is built by the dynamic construction module of the elastic resource pool, a reinforcement learning model is constructed with the dual objectives of "maximizing global resource utilization" and "minimizing task completion time". The agent in the model continuously interacts with the dynamic resource pool and learns the mapping relationship between different task characteristics and various resource allocation strategies, thereby having the ability to match appropriate resources for different tasks.

[0116] Reinforcement learning scheduling decision formula:

[0117] π(s t ) = argmax a Q(s t ,a;θ)

[0118] π(s t ) is in state s t The optimal strategy adopted by the agent is the optimal resource allocation action selected based on the current system state. For example, when facing a high-priority inference task, the strategy may instruct it to allocate a low-load node's GPU.

[0119] s t The system state at time t includes information such as the resource status of the resource pool (e.g., GPU utilization and memory usage of each node) and the characteristics of tasks in the task queue (task type, priority, resource requirements, etc.), which are represented in vector form.

[0120] 'a' represents the set of actions that the agent can take, such as allocating a certain resource of a specific node to a certain task.

[0121] Q(s t (a; θ) is the state-action value function, evaluating the state s. t After taking action 'a', the expected cumulative reward obtained by following policy 'π' from that state is determined by 'θ'. θ is a parameter of the Q-function, which is continuously optimized through reinforcement learning. For example, if a low-load node GPU is allocated to a high-priority inference task, the overall resource utilization of the system increases and the task completion time is shortened, resulting in a corresponding increase in the Q-function value, indicating that the action is a better choice. θ uses the Adam optimizer (learning rate 0.001, decay rate 0.9), the agent interacts with the resource pool 100,000 times (each interaction includes 20 randomly generated tasks), the experience replay pool has a capacity of 1 million records, and the target Q-network is updated every 1000 steps.

[0122] argmax a Find the value of Q(s) t The optimal strategy for an agent in the current state is the action a that yields the maximum expected cumulative reward, denoted as a (θ).

[0123] Formula source: Q-learning algorithm framework in reinforcement learning. In the resource allocation problem, the system state and resource allocation actions are constructed as the state space and action space of reinforcement learning. Through continuous trial and error learning, the agent finds the optimal resource allocation strategy that maximizes the system objectives (maximizing global resource utilization and minimizing task completion time).

[0124] (2) Optimize decision-making mechanism: This module can output the optimal resource allocation scheme in real time, such as allocating GPUs on nodes with low load to high-priority inference tasks, and scheduling cross-node resource groups for long-term training tasks; it introduces an "opportunity cost assessment factor" to quantify the potential losses that resource allocation may bring when making decisions, such as the cost of delaying task B after allocating a GPU to task A, thereby avoiding short-sighted decisions, ensuring that resource allocation reaches the optimal state at the global level, and giving full play to the efficiency of the resource pool;

[0125] Opportunity cost assessment factor = ∑(delay time of affected task × task priority), where delay time = duration of resource occupation - original expected completion time; this factor is fused with the Q function through weight λ (λ = 0.3), that is, the optimization objective is max(Q - λ × opportunity cost).

[0126] The cross-node resource collaboration module includes:

[0127] (1) Global resource view construction: After the reinforcement learning scheduling decision module determines the allocation scheme, this module is responsible for realizing the efficient allocation of resources among multiple nodes. Based on the distributed consensus algorithm, a global resource view is constructed, which can synchronize the resource status and task progress of each node in real time, so that the system has a clear understanding of the overall resource situation and provides comprehensive and timely information support for the reasonable allocation of cross-node resources.

[0128] Distributed consistency is achieved using the Raft algorithm, with an election period of 500ms. Log synchronization uses incremental replication. The master node is responsible for view aggregation, and the slave nodes report their status to the master node every 50ms.

[0129] The global resource view is stored in the master node's in-memory database (such as Redis) as key-value pairs (the key is the node ID, and the value is the resource status JSON); an RPC interface (port 8080) is provided for each module to query, and in case of status conflict, the status with the newer timestamp shall prevail;

[0130] (2) Cross-node allocation mechanism: When the resources of a single node cannot meet the task requirements, this module automatically triggers the cross-node resource aggregation function to virtualize some resources of multiple nodes into "logical super nodes" to meet the multi-card collaboration requirements of large model training; at the same time, a "resource migration zero-awareness mechanism" is designed to achieve a smooth transition of tasks during cross-node resource migration through memory snapshot and task state serialization technology, which effectively solves the problem of excessively long task migration time in traditional distributed scheduling and significantly improves the efficiency of cross-node resource collaboration.

[0131] Memory snapshots employ a copy-on-write (COW) mechanism, generating incremental snapshots at fixed time intervals; task state serialization uses a specific format, containing key information such as model parameters, current iteration count, and gradient data; zero-perceptual transfer is achieved through the above techniques, ensuring a smooth task transition;

[0132] Memory snapshots are generated every 100ms. The task state is serialized in Protobuf format, including model parameters (float32 type), iteration count (int64 type), and gradient data (tensor format).

[0133] The task lifecycle dynamic adaptation module includes:

[0134] (1) Dynamic adjustment logic: After the cross-node resource coordination module completes the cross-node allocation of resources, the entire process of task execution is tracked in real time, and the resource allocation is flexibly adjusted according to the different states of the task. For example, when the training task enters the data preprocessing stage, the GPU allocation ratio will be automatically reduced and some resources will be released to the inference task because the dependence on GPU is reduced in this stage. When the inference task is in a low traffic period, its CPU quota will be compressed and resources will be tilted to the training task.

[0135] (2) Elasticity coefficient adjustment: The "resource elasticity coefficient" is proposed. Based on the resource utilization efficiency of the current stage of the task, such as the ratio of computation to resource occupancy, the allocation weight is automatically adjusted to ensure that resources can be tilted towards high-value stages. This realizes the dynamic optimization of resources during task execution, so that resource utilization is always synchronized with the actual needs of the task and improves the overall utilization efficiency of resources.

[0136] Formula for calculating resource elasticity coefficient:

[0137]

[0138] ElasticCoefficient is a resource elasticity coefficient used to measure the resource utilization efficiency of a task at its current stage, in order to determine the weight of resource allocation.

[0139] CurrentStageComputeVolume represents the computational load at the current stage of the task. For example, in the data preprocessing stage of a training task, the computational load may be reflected in the number of computational operations involved in data reading, cleaning, and transformation; in the model training stage, the computational load may be the number of floating-point operations during model parameter updates.

[0140] CurrentStageComputeVolume is measured in FLOPS (e.g., approximately 10 for the preprocessing stage). 8 FLOPS, training phase approximately 10 11 FLOPS); CurrentStageResourceUsage is expressed as a percentage of resource usage (e.g., 30% GPU usage); the coefficient is recalculated every 30 seconds;

[0141] CurrentStageResourceUsage represents the resource usage at the current stage of the task, including the consumption of resources such as GPU, CPU, and memory. For example, in the data preprocessing stage of a training task, it may mainly be reflected in the consumption of CPU and memory; in the model training stage, GPU consumption becomes the main part.

[0142] Formula Origin: To achieve dynamic resource optimization during task execution, this formula is constructed based on the definition of resource utilization efficiency. In research related to task scheduling and resource management, in order to rationally allocate resources according to the resource demand characteristics of different stages of a task, resource allocation strategies are often dynamically adjusted by calculating resource utilization efficiency-related indicators. When the elasticity coefficient > 1.2, the resource allocation weight for that stage is increased by 20%; when the coefficient < 0.8, the weight is decreased by 10%. After adjustment, it is necessary to ensure that the resource occupancy of a single task is not lower than the minimum threshold (e.g., GPU ≥ 10%). The resource elasticity coefficient here is a way to measure resource utilization efficiency.

[0143] Among them, the fault prediction and self-healing scheduling module:

[0144] (1) Fault prediction mechanism: During the task execution process, hardware sensor data such as GPU temperature and power consumption are integrated with the task execution log. The improved anomaly detection model is used to identify potential GPU faults in advance, such as hardware faults or performance degradation, so as to buy time for fault handling and minimize the impact of faults.

[0145] (2) Self-healing and safeguard measures: When a fault is detected or a task is abnormally interrupted, the "resource hot migration" mechanism is automatically triggered to seamlessly switch the task to the backup resource. At the same time, the fault node information is recorded in detail and isolated to prevent the fault from spreading. In addition, a "fault impact map" is constructed to calculate the impact range of a single node fault on the global task in advance. When migrating tasks, high-priority tasks are prioritized to minimize the impact of the fault on task execution and ensure the continuity of tasks.

[0146] RelatedTasks are identified through a task dependency table (stored in a distributed database). For example, if task A depends on the data output of node X, it is considered a related task. DependencyDegree = (average daily data transfer volume between the task and the faulty node / total data volume of the task). For example, if the transfer volume accounts for 60%, the dependency degree = 0.6.

[0147] The anomaly detection model adopts an LSTM-based temporal prediction framework: the input is a temporal window of GPU temperature (sampled at fixed time intervals) and power consumption (sampled at fixed time intervals), features are extracted by multiple LSTM layers, and the output layer judges anomalies by reconstruction error (the threshold is calculated based on the reconstruction error of the training set), so as to realize the early identification of potential GPU faults.

[0148] The LSTM model contains two hidden layers (128 neurons per layer), with an input window of 30 steps (each step corresponds to 5 seconds, for a total of 150 seconds of data); training uses the MSE loss function, and the validation set precision is ≥90% and recall is ≥85%; the anomaly threshold is calculated as the mean reconstruction error of the training set plus twice the standard deviation (mean = 0.1, standard deviation = 0.03, threshold = 0.16).

[0149] Threshold = Mean reconstruction error of training set + 2 standard deviation, where the training set contains GPU running data (100,000+ samples) from the past 3 months;

[0150] Formula for assessing the scope of fault impact:

[0151]

[0152] ImpactRange is a quantitative value representing the impact range of a faulty node on the global task. The larger the value, the more severe the impact of the fault on the execution of system tasks.

[0153] RelatedTasks is a set of tasks associated with the failed node. These tasks may be affected because they depend on the failed node's resources or data, or because they are in the same task chain.

[0154] Priority i This represents the priority of task i, and its value is usually a positive integer. The higher the priority, the larger the value. For example, high-priority inference tasks may have high requirements for system real-time performance, so their priority values ​​are set to be larger. On the other hand, some non-critical background training tasks have relatively smaller priority values.

[0155] DependencyDegree i,FailedNode DependencyDegree represents the degree of dependence of task i on the faulty node, with a value ranging from [0, 1]. For example, if task i depends entirely on the faulty node to provide data or computing resources in order to run, then DependencyDegree is 1. i,FailedNode =1; if task i is not related to the faulty node, then DependencyDegree is 1. i,FailedNode =0; the degree of dependency is usually determined based on factors such as the amount of data transferred between tasks and nodes, and the relationship between computing resource requests.

[0156] Formula source: Based on the theory of fault propagation and impact analysis, in the research of distributed system fault management, in order to quickly assess the impact of node failure on the execution of tasks in the entire system, the scope of the fault impact is often quantified by considering the priority of the affected tasks and their dependence on the faulty node, thereby providing a basis for task migration and fault recovery strategy formulation.

[0157] The closed-loop feedback optimization module includes:

[0158] (1) Data-driven optimization: Through the collaborative work of the previous modules, historical data on resource allocation is collected, including GPU utilization, task completion time, fault recovery time, etc. The parameters of the scheduling algorithm are continuously optimized through offline training. At the same time, a visual interface is provided to intuitively display the efficiency bottlenecks of resource allocation, such as the average waiting time of a certain type of task being too long, etc., and the administrator can manually adjust the strategy weight according to the actual situation.

[0159] Historical data includes the following fields: task ID, type, number of GPUs allocated, CPU quota, start / end time, and fault flag; the data volume consists of 100,000 records from the past 6 months, divided into training and validation sets in an 8:2 ratio, with offline training executed every 24 hours.

[0160] (2) Strategy Evolution Mechanism: In order to enable the system to adapt to different task load scenarios, this module is designed with a "strategy evolution engine". The engine continuously optimizes the scheduling rules based on a multi-objective genetic algorithm, so that the system can maintain good performance under various task load scenarios such as peak training period and burst inference period, and promote the system to continuously adapt to new task requirements and resource environment, and achieve long-term stable and efficient operation.

[0161] The multi-objective genetic algorithm has a population size of 50, a crossover probability of 0.8 (single-point crossover), and a mutation probability of 0.05 (Gaussian mutation, standard deviation 0.1). The optimization objectives are resource utilization and task completion time, with weights of 0.6 and 0.4, respectively. The algorithm terminates after 50 iterations, retaining the top 10% of the best strategies.

[0162] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0163] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A task-driven dynamic allocation system for computing resources, characterized in that: The system includes: The task feature intelligent analysis module is based on deep learning bidirectional analysis of tasks, extracts features including type and computational intensity, uses historical data models to predict resource demand fluctuations, and introduces task fingerprint coding features. Elastic resource pool dynamic construction module: Based on the characteristics of the task, the module intelligently analyzes the characteristics of the task, abstracts the GPU into a logical pool, monitors the load in real time and adjusts the capacity, designs an adaptive scaling algorithm, and adjusts the sensitivity according to the task type; Reinforcement learning scheduling decision module: The model is built with global resource utilization and task completion time as the objectives. The agent learns the mapping between task characteristics and allocation strategies, outputs the optimal solution, and introduces an opportunity cost evaluation factor to avoid short-sighted decisions. Cross-node resource collaboration module: After the reinforcement learning scheduling decision module determines the solution, a global resource view is constructed based on a distributed algorithm, the node status and task progress are synchronized, and when resources are insufficient, cross-node resources are aggregated into a logical super node, and the task is smoothly transitioned through zero-perception migration technology. Task lifecycle dynamic adaptation module: After completing cross-node allocation, it tracks the entire task process and adjusts resources according to status, using resource elasticity coefficients and adjusting weights based on stage efficiency; Fault prediction and self-healing scheduling module: During task execution, hardware data and logs are integrated to predict GPU failures in advance through anomaly detection. When a failure occurs, the task is automatically hot-migrated to a backup resource, and a fault impact map is constructed. Closed-loop feedback optimization module: collects historical data to optimize algorithm parameters, supports manual adjustment of policy weights, designs a policy evolution engine, and optimizes rules using a multi-objective genetic algorithm.

2. The task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The task feature intelligent analysis module includes: (1) Core functions and parsing logic: The input task is parsed bidirectionally through a deep learning model. On the one hand, the task type, computational intensity, memory peak and time sensitivity are extracted in detail; on the other hand, the prediction model trained by the historical task database is used. (2) Innovative design and function: Introducing task fingerprints, the parsed features are encoded into unique vectors to achieve rapid matching of similar tasks and advance prediction of resource requirements.

3. The task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The elastic resource pool dynamic construction module includes: (1) Resource pool construction logic: Based on the task characteristics output by the task characteristic intelligent parsing module, a resource infrastructure is built, integrating GPU, CPU and memory into a logical resource pool. The capacity of the resource pool is dynamically adjusted by monitoring the load of physical resources in real time. (2) Adaptive scaling mechanism: Design an adaptive algorithm for resource scaling threshold, which can dynamically adjust the scaling sensitivity according to the task type, so that the resource pool can flexibly adapt to the dynamic changes of the task.

4. The task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The reinforcement learning scheduling decision module includes: (1) Decision Model and Objectives: After the resource pool is built by the dynamic construction module of the elastic resource pool, a reinforcement learning model is constructed with the dual objectives of maximizing global resource utilization and minimizing task completion time. The agent in the model learns the mapping relationship between different task characteristics and various resource allocation strategies by continuously interacting with the dynamic resource pool. (2) Optimize the decision-making mechanism: output the optimal resource allocation plan in real time, introduce opportunity cost assessment factors, and quantify the potential losses caused by resource allocation when making decisions.

5. A task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The cross-node resource collaboration module includes: (1) Global resource view construction: After the reinforcement learning scheduling decision module determines the allocation scheme, a global resource view is constructed based on the distributed consensus algorithm to synchronize the resource status and task progress of each node in real time. (2) Cross-node allocation mechanism: When the resources of a single node cannot meet the task requirements, the cross-node resource aggregation function is automatically triggered. A zero-awareness mechanism for resource migration is designed. Through memory snapshot and task state serialization technology, the smooth transition of tasks during cross-node resource migration is achieved.

6. The task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The task lifecycle dynamic adaptation module includes: (1) Dynamic adjustment logic: After the cross-node resource coordination module completes the cross-node allocation of resources, the resource allocation during the task execution process is dynamically adjusted, the entire process of task execution is tracked in real time, and the resource allocation is adjusted according to the different states of the task. (2) Elasticity coefficient adjustment: The resource elasticity coefficient is designed to automatically adjust the allocation weight according to the resource utilization efficiency at the current stage of the task, thereby realizing the dynamic optimization of resources during the task execution process and ensuring that resource utilization is always synchronized with the actual needs of the task.

7. A task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The fault prediction and self-healing scheduling module: (1) Fault prediction mechanism: During task execution, hardware sensor data including GPU temperature and power consumption are integrated with task execution logs, and potential GPU faults are identified in advance through an anomaly detection model; (2) Self-healing and safeguard measures: When a fault is detected or a task is abnormally interrupted, the resource hot migration mechanism is automatically triggered to seamlessly switch the task to the backup resource. At the same time, the fault node information is recorded in detail and isolated, a fault impact map is constructed, and the impact range of a single node fault on the global task is calculated in advance.

8. A task-driven dynamic allocation system for computing resources according to claim 1, characterized in that: The closed-loop feedback optimization module includes: (1) Data-driven optimization: Collect historical data on resource allocation, including GPU utilization, task completion time, and fault recovery time. Continuously optimize the parameters of the scheduling algorithm through offline training. Provide a visual interface to intuitively display the efficiency bottlenecks in resource allocation. Support administrators to manually adjust the policy weights according to the actual situation. (2) Strategy evolution mechanism: A strategy evolution engine was designed. This engine continuously optimizes the scheduling rules based on a multi-objective genetic algorithm, which drives the system to adapt to new task requirements and resource environment.

Images