Task scheduling method and device, electronic equipment and storage medium

Abstract

The invention is suitable for the technical field of data processing, and provides a task scheduling method which comprises the steps that task information of a task to be scheduled and load information of all nodes are obtained, and the nodes are used for providing computing power resources for task processing; based on the task information and the load information, calculating a task weighting cost when each node processes the to-be-scheduled task; determining the node with the minimum task weighting cost as a target node; and scheduling the to-be-scheduled task to the target node for processing. According to the method, node computing power indexes are collected in real time, task features are analyzed and reasoned, intelligent distribution and continuous optimization of tasks are achieved through an adaptive scheduling algorithm, and the problems that an existing task scheduling method is likely to be caused in a high-concurrency or heterogeneous computing power environment, and the task scheduling efficiency is high are solved. The calculation resource utilization rate is low; the delay between nodes is not uniform; and the model reasoning response time is not stable.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a task scheduling method, apparatus, electronic device and storage medium. Background Technology

[0002] With the widespread application of deep learning models, AI inference tasks are commonly deployed in distributed clusters (such as Kubernetes and Docker) in production environments. However, existing inference task scheduling methods typically employ fixed-weight allocation or round-robin strategies, which cannot dynamically adjust based on the real-time computing power status of nodes. This leads to some nodes being overloaded while others are idle, resulting in low overall cluster computing power utilization. For latency-sensitive inference tasks (such as real-time video analytics and interactive applications), round-robin strategies cannot guarantee that tasks are always assigned to the node with the lightest load and best network performance, resulting in significant fluctuations in response time. Furthermore, when a node fails or its performance degrades, it is impossible to detect and reschedule tasks in a timely manner. Therefore, there is an urgent need for a task scheduling method that can dynamically select the optimal node for inference based on the real-time load and task characteristics of each node. This would address the problems of existing task scheduling methods, such as low utilization of computing resources, uneven latency between nodes, and unstable model inference response time, which are prone to occur in high-concurrency or heterogeneous computing environments. Summary of the Invention

[0003] This application provides a task scheduling method that addresses the problems of existing task scheduling methods, such as low utilization of computing resources, uneven latency between nodes, and unstable model inference response time, which are prone to occur in high-concurrency or heterogeneous computing environments. This invention solves these problems by real-time collection of node computing power indicators, analysis of inference task characteristics, and the use of an adaptive scheduling algorithm to achieve intelligent task allocation and continuous optimization.

[0004] In a first aspect, embodiments of this application provide a task scheduling method, the method comprising the following steps:

[0005] Obtain task information for the task to be scheduled, as well as load information for each node, wherein the nodes are used to provide computing resources for task processing;

[0006] Based on the task information and the load information, calculate the task-weighted cost for each node when processing the scheduled task;

[0007] The node with the lowest weighted cost of the task is identified as the target node;

[0008] The task to be scheduled is scheduled to the target node for processing.

[0009] Optionally, calculating the task-weighted cost for each node to process the scheduled task based on the task information and the load information includes:

[0010] Based on the task information, the task-sensitive feature vector of the task to be scheduled is extracted;

[0011] And, based on the load information, extract the node state vector of the node;

[0012] Based on the task-sensitive feature vector and the node state vector, the task-weighted cost of the node when processing the task to be scheduled is calculated.

[0013] Optionally, calculating the task-weighted cost when the node processes the task to be scheduled based on the task-sensitive feature vector and the node state vector includes:

[0014] The task sensitivity coefficient is obtained by calculating the task-sensitive feature vector using a preset dynamic coefficient function.

[0015] Furthermore, the node cost is calculated by using a preset cost function on the node state vector;

[0016] The node cost is weighted and calculated based on the task sensitivity coefficient to obtain the task-weighted cost when the node processes the task to be scheduled.

[0017] Optionally, after scheduling the task to be scheduled to the target node for processing, the method further includes:

[0018] Obtain the processing result feedback of the target node for the target task, wherein the target task is the task scheduled to the target node;

[0019] Based on the feedback of the processing results, determine whether the current scheduling strategy needs to be adjusted;

[0020] If the current scheduling strategy needs to be adjusted, then the current scheduling strategy is adjusted to transfer the target task to a new node.

[0021] Optionally, determining whether the scheduling strategy needs adjustment based on the processing result feedback includes:

[0022] Based on the processing results, the execution metrics of the target node are extracted;

[0023] The metric is compared with the expected requirements of the target task, and the feedback error vector is calculated.

[0024] Based on the feedback error vector, determine whether the current scheduling strategy needs to be adjusted.

[0025] Optionally, determining whether the current scheduling strategy needs adjustment based on the feedback error vector includes:

[0026] Based on the feedback error vector and the preset penalty function, the reward function of the current scheduling strategy is constructed, and the reward value is calculated.

[0027] If the reward value is less than the preset reward value threshold, it is determined that the current scheduling strategy needs to be adjusted.

[0028] If the reward value is greater than or equal to a preset reward value threshold, then the current scheduling strategy does not need to be adjusted.

[0029] Optionally, adjusting the current scheduling strategy to perform node transfer of the target task includes:

[0030] The task weighted cost in the current scheduling strategy is recalculated using the node state vectors collected over N window periods.

[0031] Based on the weighted cost calculation results of the tasks within the N window periods, it is determined that the target task will be rescheduled to a new node.

[0032] Secondly, embodiments of this application provide a task scheduling device, the task scheduling device comprising:

[0033] The acquisition module is used to acquire task information of the task to be scheduled, as well as load information of each node, wherein the nodes are used to provide computing resources for task processing;

[0034] The calculation module is used to calculate the task-weighted cost for each node when processing the scheduled task based on the task information and the load information.

[0035] The determination module is used to determine the node with the lowest weighted cost of the task as the target node;

[0036] The processing module is used to schedule the task to be scheduled to the target node for processing.

[0037] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the task scheduling method provided in embodiments of the present invention.

[0038] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps in the task scheduling method provided in the embodiments of the present invention.

[0039] The above-mentioned solution of this application has the following beneficial effects: It obtains task information of the task to be scheduled and load information of each node, with nodes providing computing resources for task processing; based on the task information and load information, it calculates the weighted cost of each node when processing the task to be scheduled; it determines the node with the smallest weighted cost as the target node; and it schedules the task to be scheduled to the target node for processing. This invention solves the problems of existing task scheduling methods, such as low utilization of computing resources, uneven latency between nodes, and unstable model inference response time, which are easily caused in high-concurrency or heterogeneous computing environments, by real-time collection of node computing power indicators, analysis of inference task characteristics, and the use of adaptive scheduling algorithms to achieve intelligent allocation and continuous optimization of tasks.

[0040] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a task scheduling method provided in an embodiment of this application;

[0043] Figure 2 This is a schematic diagram of the structure of a task scheduling device provided in an embodiment of this application;

[0044] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0046] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0047] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0048] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0049] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0050] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0051] like Figure 1 As shown, Figure 1 This is a flowchart of a task scheduling method provided in an embodiment of the present invention. The task scheduling method includes the following steps:

[0052] 101. Obtain task information for the tasks to be scheduled, as well as load information for each node.

[0053] In this embodiment of the invention, the task scheduling method described above can be applied to a task scheduling platform, which can be built on or distributed servers. The task scheduling platform includes a data interface (for sensors or users to upload data), a knowledge database, and a knowledge database construction program. The data interface can be used to obtain a first feature vector to be searched, and the knowledge database construction program can be used to construct the knowledge database. The knowledge database is specifically used to provide additional association information for the identified data entities, thereby improving the depth of the data recognition system's understanding of the content.

[0054] The tasks to be scheduled mentioned above can be tasks that need to be processed.

[0055] The aforementioned task information can be descriptive data associated with the task to be scheduled, such as model identifier, input data size, real-time requirements, etc.

[0056] The aforementioned nodes are used to provide computing resources for task processing.

[0057] The aforementioned load information can be multi-dimensional indicators describing the overall load status and availability of a node at a given moment, such as CPU / GPU utilization, video memory usage, I / O latency, average response latency, task failure rate, and network latency. The load information reflects the resource consumption, health status, performance, and trustworthiness of each computing node in real time.

[0058] Node load information can be collected by the system every T seconds (e.g., 5 seconds) via Prometheus or a custom agent. Prometheus is an open-source monitoring and alerting tool used to collect, store, and query time-series data. Prometheus' core function is to periodically pull metric data from configured targets (such as Exporters) and store it in a built-in time-series database.

[0059] 102. Based on task information and load information, calculate the task-weighted cost for each node when processing the scheduled tasks.

[0060] In this embodiment of the invention, the above-mentioned task weighted cost can be the execution cost of the task to be scheduled on each node, calculated based on the task information and the load size of the node.

[0061] Furthermore, a fitness function can be used to evaluate the cost of each node in handling a certain task. The fitness function mentioned above is the core indicator for evaluating the quality of individuals in genetic algorithms. It measures the fitness of individuals through non-negative values, guides the direction of population evolution, and serves as a selection criterion.

[0062] In one possible implementation, for example, the task-weighted cost = w1 * f1 (resource matching degree) + w2 * f2 (node ​​load) + w3 * f3 (other factors)

[0063] Where w1, w2, and w3 represent weighting coefficients; f represents the fitness function, f1 represents the cost of resource matching, f2 represents the cost of node load, and f3 represents the cost of other factors. The resource matching degree can be the degree of matching between task information and available node resources; the node load can be indicators such as node CPU / GPU utilization, video memory usage, IO latency, average response latency, task response failure rate, and network latency. The other factors can be node stability indicators, node security indicators, etc.

[0064] It should be noted that the task weighted cost can reflect the execution cost of the task to be scheduled at a certain node, thereby helping the system to allocate tasks more effectively.

[0065] 103. The node with the lowest weighted cost of the task is identified as the target node.

[0066] In this embodiment of the invention, after calculating the weighted cost of each available node in processing the current scheduled task, the node with the smallest weighted cost is selected as the target node through a global comparison.

[0067] The target node mentioned above can be a computing node used to carry out and execute the scheduled task.

[0068] Understandably, the lower the weighted cost of the task, the lower the expected overall negative effect on the node corresponding to the currently scheduled task during execution. These overall negative effects can include delays, instability risks, and security costs.

[0069] 104. Schedule the tasks to be scheduled to the target node for processing.

[0070] In this embodiment of the invention, the task to be scheduled can be assigned to the target node, and the target node can process the task to be scheduled.

[0071] The above processing can be the process by which the target node performs inference calculations and data operations on the task to be scheduled.

[0072] In this embodiment of the invention, the invention can call the container orchestration platform API to send the task to be scheduled to the target node. During the execution of the task to be scheduled, the monitoring component of the target node synchronously collects the actual execution indicators and reports the actual execution indicators as the processing result when the task ends.

[0073] In this embodiment of the invention, task information of the task to be scheduled and load information of each node are obtained. Nodes provide computing resources for task processing. Based on the task information and load information, the weighted cost of each node in processing the task to be scheduled is calculated. The node with the lowest weighted cost is determined as the target node. The task to be scheduled is then scheduled to the target node for processing. This invention solves the problems of low computing resource utilization, uneven latency between nodes, and unstable model inference response time that exist in existing task scheduling methods, which are prone to occur in high-concurrency or heterogeneous computing environments. This is achieved by real-time collection of node computing power indicators, analysis of inference task characteristics, and the use of an adaptive scheduling algorithm to realize intelligent allocation and continuous optimization of tasks.

[0074] It is understood that in the specific implementation of this application, data such as load data, scheduling data, node data, and task data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0075] Optionally, in the step of calculating the task-weighted cost of each node when processing the task to be scheduled based on task information and load information, the task-sensitive feature vector of the task to be scheduled can be extracted based on task information; and the node state vector of the node can be extracted based on load information; and the task-weighted cost of the node when processing the task to be scheduled can be calculated based on the task-sensitive feature vector and the node state vector.

[0076] In this embodiment of the invention, the above-mentioned task information may be descriptive data associated with the task to be scheduled, such as model identifier, input data size, real-time requirements, etc.

[0077] The aforementioned task-sensitive feature vector can transform the multi-dimensional, unstructured requirements of task information into a unified, computable mathematical expression. The task-sensitive feature vector can be a fixed-dimensional mathematical vector composed of multiple scalar values, where the value of each dimension quantitatively represents the sensitivity or constraint strength of the task information to a certain type of system performance or service quality indicator.

[0078] For example, a task-sensitive feature vector can be represented as:

[0079] S_t = {delay, accuracy, security} ∈ [0,1]^3

[0080] Where S_t represents the task-sensitive feature vector of task t; delay represents latency sensitivity; accuracy represents accuracy / stability sensitivity; and security represents security sensitivity. The larger the value of Delay, the more stringent the requirements of task t for processing latency; the larger the value of accuracy, the higher the requirements of task t for the accuracy of inference results and the stability of the computation process; and the larger the value of security, the higher the requirements of task t for the security isolation of the computing environment and data privacy.

[0081] The aforementioned load information can be a multi-dimensional indicator describing the overall carrying capacity and available capabilities of a node at a certain moment, such as CPU / GPU utilization, video memory usage, IO wait time, average response latency, task response failure rate, network latency, and other indicators.

[0082] The aforementioned node state vector can be used to comprehensively and quantitatively describe a node's resource usage, performance, health status, and security attributes at a given moment.

[0083] Specifically, the node state vector can be represented as:

[0084] E_i={Lgpu_i,Lcpu_i,Lmem_i,MemFree_i,IOwait_i,NetLatency_i,NodeTrust_i,ModelStability_i,HistoricalAcc_i,FailureRate_i}

[0085] Where E_i represents the node state vector of node i; i represents node i; Lgpu represents the GPU processor load; Lcpu represents the CPU computing load; Lmem represents the memory load; MemFree represents the ratio of free memory to GPU memory; IOwait represents the IO wait time; NetLatency represents the network latency; NodeTrust represents the node security trust score; ModelStability represents the node stability score; HistoricalAcc represents the historical inference accuracy of similar tasks; and FailureRate represents the recent task execution failure rate.

[0086] The aforementioned task-weighted cost can be the execution cost of the task to be scheduled on each node, calculated based on the task-sensitive feature vector and the node state vector.

[0087] It should be noted that a fitness function can be used to evaluate the task-weighted cost of each node in handling a certain task. The fitness function mentioned above is the core indicator for evaluating the quality of individuals in genetic algorithms. It measures the fitness of individuals through non-negative values, guides the direction of population evolution, and serves as a selection criterion.

[0088] Optionally, in the step of calculating the task-weighted cost of a node processing a scheduled task based on the task-sensitive feature vector and the node state vector, the task-sensitive feature vector can be calculated using a preset dynamic coefficient function to obtain the task sensitivity coefficient; and the node state vector can be calculated using a preset cost function to obtain the node cost value; and the node cost value can be weighted according to the task sensitivity coefficient to obtain the task-weighted cost of the node processing the scheduled task.

[0089] In this embodiment of the invention, the aforementioned preset dynamic coefficient function can be a pre-set dynamic coefficient function of the system, or a learnable dynamic coefficient function. The parameters of the learnable dynamic coefficient function are mathematical functions whose parameters can be optimized and adjusted through data-driven methods (such as machine learning), and are used to describe the changes in the system state over time.

[0090] The aforementioned task sensitivity coefficients can be a set of weight values ​​calculated using a preset dynamic coefficient function based on the specific values ​​in the task sensitivity feature vector. Task sensitivity coefficients may include latency sensitivity coefficients, accuracy sensitivity coefficients, security sensitivity coefficients, etc.

[0091] Specifically, the task sensitivity coefficient can be expressed as:

[0092] α(delay) = 1 + k1 * exp( c1 * (delay - 0.5) )

[0093] β(accuracy) = 1 + k2 * (accuracy - 0.5)

[0094] γ(security) = 1 + k3 / (1 + exp( -c3*(security - 0.5) ))

[0095] Where α(·), β(·), and γ(·) represent preset dynamic coefficient functions; α(delay) represents the delay sensitivity coefficient; k1 represents the scaling coefficient, used to control the change range of the α coefficient; c1 represents the shape coefficient, used to control the rate of exponential decay; delay represents the task delay sensitivity; β(accuracy) represents the accuracy sensitivity coefficient; k2 represents the scaling coefficient, controlling the slope of the linear change of the β coefficient; accuracy represents the task accuracy sensitivity; γ(security) represents the task security sensitivity coefficient; k3 represents the scaling coefficient, controlling the maximum increment of the γ coefficient; - represents the direction controller, which determines the monotonicity of the exponential function exp(·), ensuring that the output value of γ(security) will smoothly transition from a lower value to a higher value as security increases; c3 represents the scaling coefficient, used to control the steepness of the sigmoid function; security represents the security sensitivity.

[0096] The aforementioned preset cost function can be a pre-set cost function of the system. This cost function measures the difference between the predicted and actual values, and its core function is to find the optimal parameters by minimizing the cost function.

[0097] The aforementioned node cost value can be a numerical value obtained by evaluating the node's state vector, used to measure the quality of the node's task processing. Node cost value can include node latency cost value, node stability cost value, and node security cost value, etc.

[0098] Specifically, the node latency generation value can be expressed as:

[0099] F_delay = w1*norm(Lgpu)+w2*norm(Lcpu)+w* norm(NetLatency) +w4*norm(IOwait)

[0100] Where F_delay represents the node latency generation value; Lgpu represents the GPU load; Lcpu represents the CPU load; NetLatency represents the network latency; IOwait represents the IO wait time; w1, w2, w3, and w4 represent weight coefficients, which are fixed values.

[0101] It is understandable that norm(x) = (x - min_x) / (max_x - min_x) is normalized using a sliding window or historical distribution.

[0102] The cost of node stability can be expressed as:

[0103] F_accuracy=u1*(1-norm(HistoricalAcc))+u2*norm(ModelVersionInstability)+u3*(required_mem / MemFree_i > 1 ? 1 : 0)

[0104] Wherein, F_accuracy represents the cost of node stability; u1, u2, u3, and u4 are fixed values; HistoricalAcc represents the historical inference accuracy of similar tasks, with higher accuracy resulting in lower costs; ModelVersionInstability represents the node stability score, with higher scores indicating instability; required_mem represents the GPU memory required for the task; MemFree_i represents the node's free GPU memory; (required_mem / MemFree_i > 1 ? 1 : 0) represents a memory sufficiency check item, with insufficient GPU memory incurring high costs.

[0105] The security cost of a node can be expressed as:

[0106] F_security=v1*(1-NodeTrustScore)+v2*(isTEE ? 0 : 1)+v3* (network_isolation_score)

[0107] Wherein, F_security represents the node security cost; NodeTrustScore∈[0,1] represents the trust score (a comprehensive score of certificate, whitelist, geographical location, operation and maintenance isolation, and compliance attributes); isTEE represents the cost of TEE absence, whether the node supports a trusted execution environment (boolean). If the node supports TEE, the cost of TEE absence is 0 (best), otherwise, the cost of TEE absence is 1 (highest cost); network_isolation_score represents the network security score∈[0,1].

[0108] The above weighted calculation can be a process of multiplying the node cost value by the corresponding task sensitivity coefficient to obtain the weighted node cost value, and then integrating the weighted node cost values ​​to obtain the task weighted cost.

[0109] The aforementioned task weighted cost can be the execution cost of the task to be scheduled on each node after weighting the node cost based on the task sensitivity coefficient.

[0110] Specifically, the task-weighted cost can be expressed as:

[0111] W_i(t) = α(delay)·F_delay(E_i, t)

[0112] + β(accuracy)·F_accuracy(E_i, t)

[0113] + γ(security)·F_security(E_i, t)

[0114] Where W_i(t) represents the weighted cost of node i processing the current scheduled task t; i represents node i; t represents the current scheduled task; α(·), β(·), and γ(·) represent preset dynamic coefficient functions; α(delay) represents the delay sensitivity coefficient; F_delay(E_i, t) represents the node delay cost; β(accuracy) represents the accuracy sensitivity coefficient; F_accuracy(E_i, t) represents the node stability cost; γ(security) represents the task security sensitivity coefficient; and F_security(E_i, t) represents the node security cost.

[0115] It should be noted that the preset dynamic coefficient function can be a value obtained by using the task sensitivity vector as input parameter.

[0116] Optionally, after scheduling the task to be scheduled to the target node for processing, the processing result feedback of the target node on the target task can be obtained. The target task is the task scheduled to the target node. Based on the processing result feedback, it can be determined whether the current scheduling strategy needs to be adjusted. If the current scheduling strategy needs to be adjusted, the current scheduling strategy is adjusted to transfer the target task to another node.

[0117] In this embodiment of the invention, the target task is a task scheduled to the target node, which may be a task that has been scheduled to the target node, is being executed, or has been completed.

[0118] The feedback of the above processing results can be information about the task processing status returned to the system by the target node after completing the target task, such as task completion status, processing time, error information, etc.

[0119] The aforementioned current scheduling strategy can serve as the basis for the entire decision-making process of scheduling the target task to the target node.

[0120] Furthermore, after a task is assigned to a target node and executed, the effectiveness of the current scheduling strategy needs to be evaluated based on the processing results from the target node. If the current scheduling strategy is found to be ineffective in completing the task, then the strategy needs to be adjusted. For example, if the target node times out, it indicates that the current scheduling strategy needs to be adjusted.

[0121] The above adjustments can be part of the process of adjusting the current scheduling strategy.

[0122] The aforementioned node migration can be a process of migrating the workload or services of a target node to other nodes.

[0123] It should be noted that when the current scheduling strategy needs to be adjusted, it can be adjusted to transfer the target task to a different node to ensure that the task is processed effectively, thereby improving overall performance.

[0124] Optionally, in the step of determining whether the scheduling strategy needs to be adjusted based on the processing result feedback, the execution indicators of the target node can be extracted based on the processing result feedback; the indicators can be compared with the expected requirements of the target task to calculate the feedback error vector; and based on the feedback error vector, it can be determined whether the current scheduling strategy needs to be adjusted.

[0125] In this embodiment of the invention, the above-mentioned processing result feedback may be information about the task processing status returned by the target node to the system after completing the target task, such as task completion status, processing time, error information, etc.

[0126] The aforementioned execution metrics can be a set of performance, quality, and status data extracted from the target node's task execution process in the feedback of processing results, used to quantitatively evaluate the actual effectiveness of the current scheduling decision. Execution metrics can include real-time inference latency, actual inference accuracy, and actual security level assessment metrics, etc.

[0127] The expected requirements of the aforementioned target task can be quantified target values ​​extracted from the task information corresponding to the target task, regarding the core dimensions expected to be achieved, such as latency, accuracy / stability, and security. The expected requirements of the target task include target latency, target accuracy, and security level.

[0128] The above comparison can be a process of comparing the performance indicators of the target node with the expected requirements of the target task.

[0129] Furthermore, the execution metrics of the target node can be compared with the expected requirements of the target task to calculate the error value of each metric and form a feedback error vector.

[0130] The aforementioned feedback error vector can be a quantified deviation between the performance indicators and the expected requirements of the target task. The error feedback vector reflects the gap between the actual performance indicators and the expected requirements of the target task.

[0131] Specifically, the feedback error vector can be represented as:

[0132] Feedback error vector:

[0133] Δ={Δdelay,Δaccuracy,Δsecurity}

[0134] Where Δdelay represents the delay error value; Δaccuracy represents the accuracy error value; and Δsecurity represents the security error value.

[0135] The time delay error value can be expressed as:

[0136] Δdelay=(T_real−T_target) / T_target

[0137] Where Δdelay represents the delay error value; T_real represents the actual inference delay; and T_target represents the target delay.

[0138] The accuracy error value can be expressed as:

[0139] Δaccuracy = A_target − A_real

[0140] Where Δaccuracy represents the accuracy error value; A_target represents the target accuracy; and A_real represents the actual inference accuracy.

[0141] The safety margin can be expressed as:

[0142] Δsecurity = S_target − S_real

[0143] Where Δsecurity represents the security error value; S_target represents the target security level; and S_real represents the actual security level assessment.

[0144] Furthermore, the error feedback vector can be used to determine whether the current scheduling strategy needs adjustment. It's understandable that a large error feedback vector indicates that the current scheduling strategy needs adjustment.

[0145] Optionally, in the step of determining whether the current scheduling strategy needs to be adjusted based on the feedback error vector, a reward function for the current scheduling strategy can be constructed based on the feedback error vector and a preset penalty function, and the reward value can be calculated; if the reward value is less than a preset reward value threshold, it is determined that the current scheduling strategy needs to be adjusted; if the reward value is greater than or equal to the preset reward value threshold, it is determined that the current scheduling strategy does not need to be adjusted.

[0146] In this embodiment of the invention, the aforementioned preset penalty function can be a penalty function pre-set by the system, or it can be an error-based penalty function. The error-based penalty function is a type of penalty function used for constrained optimization problems. By adding a penalty function related to the degree of constraint violation to the objective function, the constrained problem can be transformed into a series of unconstrained problems for solution. The core idea of ​​the error-based penalty function is that when a solution violates a constraint, the penalty function significantly increases the objective function value, thereby "penalizing" infeasible solutions and guiding the optimization process to approach the feasible region boundary.

[0147] The reward function described above can be constructed based on the feedback error vector and a preset penalty function. The reward function is used to measure the quality of the current scheduling strategy and whether there are significant fluctuations in the metrics of each node.

[0148] The reward value mentioned above can be the output of the reward function or a scalar evaluation score obtained by calculating the feedback error vector. The reward value quantitatively measures the overall performance of the current scheduling strategy in meeting the latency, accuracy, and security requirements of the target task.

[0149] Specifically, the reward value is the output of the reward function, and the calculation formula can be:

[0150] R(t)=λ1⋅f1(Δdelay)+λ2⋅f2(Δaccuracy)+λ3⋅f3(Δsecurity)

[0151] Where R(t) represents the reward function; Δdelay represents the delay error value; Δaccuracy represents the accuracy error value; Δsecurity represents the security error value; λ1, λ2, and λ3 are configurable weights that reflect the system's emphasis on performance, accuracy, and security, and satisfy λ1 + λ2 + λ3 = 1; f1, f2, and f3 are error-based penalty functions.

[0152] Understandably, for f1, when Δdelay > 0, the delay increases, f1(Δdelay) = e^−a1|Δdelay|, resulting in a penalty; when Δdelay <= 0, the delay decreases, f1(Δdelay) = 1−b1|Δdelay|, resulting in an appropriate reward. a1 represents the configurable penalty coefficient, and b1 is the configurable reward coefficient. a1 controls the severity of the penalty, and b1 controls the magnitude of the reward. For example, in a time-sensitive scenario, a1 = 3, b1 = 1.2, allowing the delay error to have a greater impact on the reward.

[0153] For f2, when Δaccuracy < 0, accuracy stability decreases, f2(Δaccuracy) = 1−a2|Δaccuracy|, resulting in a penalty; when Δaccuracy >= 0, accuracy stability increases, f2(Δaccuracy) = 1+b2|Δaccuracy|, resulting in an appropriate reward. a2 represents the configurable penalty coefficient, and b2 is the configurable reward coefficient. a2 controls the severity of the penalty, and b2 controls the magnitude of the reward.

[0154] For f3, when Δsecurity > 0, an insecurity has occurred, f3(Δsecurity) = e^−a3|Δsecurity|, and a penalty is imposed; when Δsecurity = 0, no insecurity has occurred, and f3(Δsecurity) = 1. a3 is a configurable penalty function used to control the severity of the penalty.

[0155] The aforementioned preset reward value threshold can be a reward value threshold pre-set by the system.

[0156] It should be noted that when the reward value is less than the preset reward value threshold, it can be determined that the current scheduling strategy needs to be adjusted; when the reward value is greater than or equal to the preset reward value threshold, it can be determined that the current scheduling strategy does not need to be adjusted.

[0157] Optionally, in the step of adjusting the current scheduling strategy to transfer the target task to a new node, the node state vectors collected over N window periods can be used to recalculate the weighted cost of the task in the current scheduling strategy; based on the result of the weighted cost calculation over N window periods, it is determined to reschedule the target task to a new node.

[0158] In this embodiment of the invention, the aforementioned N window periods can be N different time windows used to collect node state vectors. The time window can be a preset, fixed-length time interval, such as each window period = 30 minutes. N can be an integer greater than or equal to 1, representing the number of consecutive observation window periods.

[0159] The recalculation described above can be a process of recalculating the weighted cost of tasks in the current scheduling strategy using the node state vectors collected over N window periods.

[0160] The above calculation result can be a set of task weighted costs within NN time windows obtained by recalculating the task weighted costs in the current scheduling strategy using node state vectors collected over N time windows.

[0161] It should be noted that the node state vectors collected over N window periods can be used to recalculate the weighted cost of tasks in the current scheduling strategy, obtaining the weighted cost calculation results for tasks within N window periods. Based on the weighted cost calculation results for tasks within N window periods, the new node to which the target task will be rescheduled can be determined, and the target task can be rescheduled to the new node. This can avoid frequent scheduling caused by instantaneous fluctuations and improve the stability of the system.

[0162] In this embodiment of the invention, the invention can call the container orchestration platform API for a smooth, canary switchover. Specifically, a new task container can be started on the target node first to ensure high availability of the system during the switchover process. Then, it is confirmed that the task container on the new node is running normally and can process newly arrived data. Furthermore, it is necessary to wait for the task container on the target node to finish processing the existing data before gradually destroying the task container on the target node to avoid data loss or task interruption. The aforementioned container orchestration platform API is a software interface used to manage and automate the deployment, expansion, and operation of containerized applications. The container orchestration platform API defines the rules and methods for interaction between clients (such as developers or tools) and container orchestration systems (such as Kubernetes). The aforementioned smooth, canary switchover can be a system update strategy that combines the characteristics of canary release and smooth release, aiming to minimize the impact of the update process on tasks. Canary release refers to allowing some tasks to access the new version first, gradually expanding the scope, and finally completing the switchover; while smooth release emphasizes that the service is not interrupted during the update, ensuring that tasks are unaware of the change.

[0163] like Figure 2 As shown, an embodiment of the present invention provides a task scheduling device, which includes:

[0164] The acquisition module 201 is used to acquire task information of the task to be scheduled, as well as load information of each node, wherein the nodes are used to provide computing resources for task processing.

[0165] The calculation module 202 is used to calculate the task-weighted cost for each node when processing the task to be scheduled, based on the task information and the load information.

[0166] The determination module 203 is used to determine the node with the lowest weighted cost of the task as the target node;

[0167] The processing module 204 is used to schedule the task to be scheduled to the target node for processing.

[0168] Optionally, the calculation module 202 is further configured to extract the task-sensitive feature vector of the task to be scheduled based on the task information; and extract the node state vector of the node based on the load information; and calculate the task-weighted cost of the node when processing the task to be scheduled based on the task-sensitive feature vector and the node state vector.

[0169] Optionally, the calculation module 202 is further configured to calculate the task-sensitive feature vector using a preset dynamic coefficient function to obtain a task-sensitive coefficient; and to calculate the node state vector using a preset cost function to obtain a node cost; and to perform a weighted calculation on the node cost based on the task-sensitive coefficient to obtain the task-weighted cost when the node processes the task to be scheduled.

[0170] Optionally, the device is further configured to obtain the processing result feedback of the target node on the target task, wherein the target task is a task scheduled to the target node; determine whether the current scheduling strategy needs to be adjusted based on the processing result feedback; if the current scheduling strategy needs to be adjusted, then the current scheduling strategy is adjusted to transfer the target task to a node.

[0171] Optionally, the device is further configured to extract the execution indicators of the target node based on the processing result feedback; compare the indicators with the expected requirements of the target task to calculate the feedback error vector; and determine whether the current scheduling strategy needs to be adjusted based on the feedback error vector.

[0172] Optionally, the device is further configured to construct a reward function for the current scheduling strategy based on the feedback error vector and a preset penalty function, and calculate a reward value; if the reward value is less than a preset reward value threshold, it is determined that the current scheduling strategy needs to be adjusted; if the reward value is greater than or equal to the preset reward value threshold, it is determined that the current scheduling strategy does not need to be adjusted.

[0173] Optionally, the device is further configured to recalculate the task weighted cost in the current scheduling strategy using the node state vector collected over N window periods; and determine, based on the task weighted cost calculation results within the N window periods, to reschedule the target task to a new node.

[0174] like Figure 3 As shown, this embodiment of the invention also provides an electronic device, including a processor, which can execute any of the above-described task scheduling methods.

[0175] Specifically, it includes a processor 301 and a memory 302, as well as a computer program stored in the memory 302 and capable of running on the processor 301 to execute a task scheduling method, wherein:

[0176] Processor 301 executes the calculator program for the task scheduling method stored in memory 302, and performs the following steps:

[0177] Obtain task information for the task to be scheduled, as well as load information for each node, wherein the nodes are used to provide computing resources for task processing;

[0178] Based on the task information and the load information, calculate the task-weighted cost for each node when processing the scheduled task;

[0179] The node with the lowest weighted cost of the task is identified as the target node;

[0180] The task to be scheduled is scheduled to the target node for processing.

[0181] Optionally, the processor 301 performs the calculation of the task-weighted cost for each node to process the scheduled task based on the task information and the load information, including:

[0182] Based on the task information, the task-sensitive feature vector of the task to be scheduled is extracted;

[0183] And, based on the load information, extract the node state vector of the node;

[0184] Based on the task-sensitive feature vector and the node state vector, the task-weighted cost of the node when processing the task to be scheduled is calculated.

[0185] Optionally, the process executed by processor 301 to calculate the task-weighted cost of the node processing the task to be scheduled based on the task-sensitive feature vector and the node state vector includes:

[0186] The task sensitivity coefficient is obtained by calculating the task-sensitive feature vector using a preset dynamic coefficient function.

[0187] Furthermore, the node cost is calculated by using a preset cost function on the node state vector;

[0188] The node cost is weighted and calculated based on the task sensitivity coefficient to obtain the task-weighted cost when the node processes the task to be scheduled.

[0189] Optionally, after scheduling the task to be scheduled to the target node for processing, the method executed by the processor 301 further includes:

[0190] Obtain the processing result feedback of the target node for the target task, wherein the target task is the task scheduled to the target node;

[0191] Based on the feedback of the processing results, determine whether the current scheduling strategy needs to be adjusted;

[0192] If the current scheduling strategy needs to be adjusted, then the current scheduling strategy is adjusted to transfer the target task to a new node.

[0193] Optionally, the step of processor 301 determining whether the scheduling strategy needs adjustment based on the processing result feedback includes:

[0194] Based on the processing results, the execution metrics of the target node are extracted;

[0195] The metric is compared with the expected requirements of the target task, and the feedback error vector is calculated.

[0196] Based on the feedback error vector, determine whether the current scheduling strategy needs to be adjusted.

[0197] Optionally, the process executed by processor 301 to determine whether the current scheduling policy needs adjustment based on the feedback error vector includes:

[0198] Based on the feedback error vector and the preset penalty function, the reward function of the current scheduling strategy is constructed, and the reward value is calculated.

[0199] If the reward value is less than the preset reward value threshold, it is determined that the current scheduling strategy needs to be adjusted.

[0200] If the reward value is greater than or equal to a preset reward value threshold, then the current scheduling strategy does not need to be adjusted.

[0201] Optionally, the processor 301's adjustment of the current scheduling policy to perform node transfer of the target task includes:

[0202] The task weighted cost in the current scheduling strategy is recalculated using the node state vectors collected over N window periods.

[0203] Based on the weighted cost calculation results of the tasks within the N window periods, it is determined that the target task will be rescheduled to a new node.

[0204] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the task scheduling method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0205] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A task scheduling method, characterized in that, The method includes the following steps: Obtain task information for the task to be scheduled, as well as load information for each node, wherein the nodes are used to provide computing resources for task processing; Based on the task information and the load information, calculate the task-weighted cost for each node when processing the scheduled task; The node with the lowest weighted cost of the task is identified as the target node; The task to be scheduled is scheduled to the target node for processing.

2. The task scheduling method as described in claim 1, characterized in that, The step of calculating the task-weighted cost for each node to process the scheduled task based on the task information and the load information includes: Based on the task information, the task-sensitive feature vector of the task to be scheduled is extracted; And, based on the load information, extract the node state vector of the node; Based on the task-sensitive feature vector and the node state vector, the task-weighted cost of the node when processing the task to be scheduled is calculated.

3. The task scheduling method as described in claim 2, characterized in that, The step of calculating the task-weighted cost when the node processes the scheduled task based on the task-sensitive feature vector and the node state vector includes: The task sensitivity coefficient is obtained by calculating the task-sensitive feature vector using a preset dynamic coefficient function. Furthermore, the node cost is calculated by using a preset cost function on the node state vector; The node cost is weighted and calculated based on the task sensitivity coefficient to obtain the task-weighted cost when the node processes the task to be scheduled.

4. The task scheduling method as described in claim 3, characterized in that, After scheduling the task to be scheduled to the target node for processing, the method further includes: Obtain the processing result feedback of the target node for the target task, wherein the target task is the task scheduled to the target node; Based on the feedback of the processing results, determine whether the current scheduling strategy needs to be adjusted; If the current scheduling strategy needs to be adjusted, then the current scheduling strategy is adjusted to transfer the target task to a new node.

5. The task scheduling method as described in claim 4, characterized in that, The step of determining whether the scheduling strategy needs to be adjusted based on the feedback of the processing result includes: Based on the processing results, the execution metrics of the target node are extracted; The metric is compared with the expected requirements of the target task, and the feedback error vector is calculated. Based on the feedback error vector, determine whether the current scheduling strategy needs to be adjusted.

6. The task scheduling method as described in claim 5, characterized in that, The step of determining whether the current scheduling strategy needs to be adjusted based on the feedback error vector includes: Based on the feedback error vector and the preset penalty function, the reward function of the current scheduling strategy is constructed, and the reward value is calculated. If the reward value is less than the preset reward value threshold, it is determined that the current scheduling strategy needs to be adjusted. If the reward value is greater than or equal to a preset reward value threshold, then the current scheduling strategy does not need to be adjusted.

7. The task scheduling method as described in any one of claims 4 to 6, characterized in that, The adjustment of the current scheduling policy to perform node transfer of the target task includes: The task weighted cost in the current scheduling strategy is recalculated using the node state vectors collected over N window periods. Based on the weighted cost calculation results of the tasks within the N window periods, it is determined that the target task will be rescheduled to a new node.

8. A task scheduling device, characterized in that, The task scheduling device includes: The acquisition module is used to acquire task information of the task to be scheduled, as well as load information of each node, wherein the nodes are used to provide computing resources for task processing; The calculation module is used to calculate the task-weighted cost for each node when processing the scheduled task based on the task information and the load information. The determination module is used to determine the node with the lowest weighted cost of the task as the target node; The processing module is used to schedule the task to be scheduled to the target node for processing.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the task scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the task scheduling method as described in any one of claims 1 to 7.

Images