Impact-dependency-resistant task scheduling method and system for edge computing network based on affinity

Abstract

The invention provides an affinity-based edge computing network damage-resistant dependency task scheduling method and system, and the method comprises the steps: receiving a dependency task, judging whether to carry out task segmentation or not according to a resource demand, constructing a directed acyclic graph, and merging repeated subtasks; screening alternative nodes and detecting schedulable subtasks; sorting the tasks based on a multi-factor priority model; performing task-node affinity and task-task anti-affinity judgment, identifying key subtasks and initiating redundancy calculation; extracting enhanced features by using a multi-head attention map neural network, and generating a scheduling decision in combination with a reinforcement learning model; scheduling is executed, the state is monitored, and rescheduling is started for failed tasks. According to the method, the execution time delay of a large computing power dependency task can be effectively reduced, the robustness and generalization ability of the system in a dynamic edge environment are enhanced, and the damage resistance is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of computing resource scheduling and management technology, and in particular to an edge computing network anti-damage dependency task scheduling method and system based on affinity. Background Technology

[0002] In the digital age, the rapid development of fields such as artificial intelligence, big data analytics, and scientific computing has spurred a massive number of high-computing-power tasks. These tasks are typically computationally intensive, time-consuming, and involve complex dependencies between their subtasks, often modeled as directed acyclic graphs (DAGs). Traditional centralized cloud computing models, due to high transmission latency and bandwidth constraints, struggle to meet the real-time demands of such tasks. Therefore, "edge computing networks," which integrate edge computing and computing power networks, have emerged. These networks push computing resources to the network edge, aiming to provide users with convenient, agile, and efficient computing services.

[0003] However, scheduling computationally demanding tasks in edge computing environments faces significant challenges. First, edge nodes typically have limited resources and are highly heterogeneous, making it difficult for a single node to independently handle large tasks, resulting in persistently high latency. Second, the complex sequential dependencies between tasks necessitate strict adherence to the execution order during scheduling; any delay or failure of a preceding task will propagate throughout the entire Directed Acyclic Graph (DAG), causing blockages in subsequent tasks. Furthermore, unlike the stable environment of a data center, edge network link quality fluctuates greatly, and nodes may fail due to excessive load, energy depletion, or physical damage. This inherent uncertainty in the task execution environment significantly increases the risk of task failure.

[0004] To address these challenges, researchers both domestically and internationally have undertaken numerous explorations. Existing technical solutions can be broadly categorized into two types. The first is a scheduling method based on the Simple Tree (ST) model. This method simplifies dependencies into a tree structure, determining task priority through the hierarchical relationship between the root node and its child nodes, thus enabling the scheduling of periodic tasks. However, this method is overly simplified, making it difficult to represent complex DAG dependencies in reality. Furthermore, it lacks consideration for dynamic task arrival and non-periodic scenarios, resulting in insufficient generalization ability. The second is a scheduling strategy based on task divisibility. This strategy breaks down large tasks into subtasks and allocates them across multiple processing slots on edge servers. While this strategy improves resource utilization, it has weak global representation of dependencies during task allocation, particularly prone to errors when handling complex dependencies involving a mix of serial and parallel connections. Moreover, it lacks an effective mechanism to address subtask failures caused by node corruption, resulting in poor system robustness.

[0005] In summary, existing technologies generally suffer from three major shortcomings when dealing with high-computing-power-dependent tasks in edge computing networks: First, they lack the ability to model and process task dependencies, resulting in low scheduling efficiency; second, they lack effective adaptation to dynamic and unstable edge environments, and their scheduling strategies have weak generalization; and third, they have not built a systematic damage resistance mechanism, which cannot guarantee the final completion of tasks in the event of failure of some nodes or subtasks. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide an edge computing network anti-damage dependency task scheduling method and system based on affinity, in order to solve the problems of high execution latency, weak generalization ability of scheduling strategies and poor overall system robustness in edge computing networks caused by insufficient dependency modeling capabilities, poor adaptability to dynamic environments and lack of systematic fault tolerance mechanisms.

[0007] On one hand, this invention provides an affinity-based edge computing network robustness-dependent task scheduling method, applied to an edge computing network containing multiple edge computing nodes, the method comprising: The system receives dependent tasks to be processed and determines whether to split the dependent tasks based on their resource requirements and preset node resource thresholds. If splitting is required, the system divides the dependent tasks into multiple sub-tasks with dependencies and constructs a directed acyclic graph (DAG) representing the dependencies and data transmission volume between the sub-tasks. DAGs that are repeated in the DAG are merged to generate a unified DAG. Based on the resource requirements of the subtask, a set of candidate nodes that meet the conditions is selected from the edge computing network, and all schedulable subtasks that have completed their preceding tasks or have no preceding tasks are detected from the current task waiting queue. Based on a preset priority calculation model, the priority of each schedulable subtask is calculated, and the schedulable subtasks are sorted according to the priority; wherein, the priority calculation model takes into account at least the importance of the task, the latency flexibility, the task execution speed, the timeliness of the task, and the task dependency factors. The schedulable subtasks are evaluated for task-node affinity and task-task anti-affinity. Based on the evaluation results, key subtasks are identified, and redundant computation is initiated for the key subtasks when resources permit. The task-node affinity is used to mark the main resource requirement type of the task and the main resource type remaining in the node, and the task-task anti-affinity is used to mark the importance of the task in its corresponding directed acyclic graph. Using a graph neural network based on a multi-head attention mechanism, the features of the schedulable subtask and the features of the candidate node set are fused and enhanced to extract enhanced features containing the dependencies between tasks and nodes. The enhanced features and the real-time network link status are input into a scheduling decision model based on reinforcement learning to obtain the scheduling decision of unloading the schedulable subtask to the target node in the candidate node set. The scheduling decision is executed based on task allocation, and the execution status is monitored. For subtasks that fail to execute but do not exceed their maximum tolerable latency, a rescheduling mechanism is initiated.

[0008] In some embodiments of the present invention, constructing a directed acyclic graph characterizing the dependencies and data transfer volume between the subtasks includes: Each of the divided subtasks is treated as a task node in a directed acyclic graph; Connect dependent subtasks with directed edges; Each directed edge is assigned a weight, which is used to characterize the amount of data transmission between the corresponding two task nodes; Based on the task nodes, the directed edges, and the weights, the directed acyclic graph is constructed.

[0009] In some embodiments of the present invention, merging repeated subtasks in the directed acyclic graph to generate a unified directed acyclic graph includes: Identify and obtain repetitive subtasks of the same type from different dependent tasks; The edges of each directed acyclic graph containing the repeated subtasks before merging are combined into a single shared node to generate the unified directed acyclic graph. Virtual entry and exit tasks are added to the unified directed acyclic graph to form unique task start and end nodes.

[0010] In some embodiments of the present invention, the latency elasticity is characterized by the reciprocal of the maximum tolerable time for task execution, the task execution speed is characterized by the reciprocal of the task computational load, and the priority is calculated using the following formula: ; in, This indicates that the schedulable subtask is in Time priority; , , , , All are weighting factors, and their sum is 1; , , , , These represent the normalized task importance, latency flexibility, task execution speed, task timeliness, and task dependency, respectively.

[0011] In some embodiments of the present invention, the enhanced features are extracted using the graph neural network, including: An initial node feature set is constructed based on the features of the schedulable subtasks and the features of the candidate node set. The initial node feature set is linearly transformed using different weight matrices to obtain the query matrix, key matrix, and value matrix, and the attention score matrix is ​​calculated. Based on the attention score matrix, calculate the attention coefficient between the task node and its first-order neighboring task nodes; Based on the attention coefficients, a multi-head attention mechanism is adopted to aggregate the outputs of multiple attention heads, and a nonlinear transformation is performed through a preset activation function to obtain the final enhanced features containing dependencies.

[0012] In some embodiments of the present invention, the attention score matrix is ​​calculated using the following formula: ; in, This represents the attention score matrix; Represents the query matrix; Represents the key matrix; Represents the value matrix; For activation functions; Indicates transpose; This indicates the dimension of the key matrix; The attention coefficient is calculated using the following formula: ; in, Represents task node and task nodes Attention coefficient between them; This represents the preset activation function; This represents the attention score matrix; Indicates transpose; Represents the weight matrix; Indicates a splicing operation; Represents task node Features; Represents task node Features; Used to traverse task nodes All first-order nearest neighbors; Represents first-order nearest neighbor nodes Its characteristics.

[0013] In some embodiments of the present invention, the method further includes training the scheduling decision model using the Actor-Critic algorithm framework, including: A state space is constructed using the enhanced features output by the graph neural network and the real-time network link state, wherein the enhanced features include enhanced task features and node features; The action space is constructed based on scheduling decisions that allow schedulable subtasks to be unloaded to the target node; The reward is a negative value for the task execution delay; State transition samples are collected by interacting with the environment and stored in the experience replay pool; A batch of state transition samples are randomly sampled from the experience replay pool, and the parameters of the Critic network are updated by minimizing the error between the predicted value of the Critic network and the target value calculated based on the target network. By using the policy gradient method and the gradient signal provided by the updated Critic network, the parameters of the Actor network are updated to maximize the expected cumulative reward, and the scheduling decision model is finally trained.

[0014] In some embodiments of the present invention, the method further includes, during the process of dividing the dependent task into multiple dependent subtasks, if the resource requirements of the divided subtasks still exceed the node resource threshold and cannot be further functionally divided, then they are regarded as a single task, and algorithmic division is performed, including: The algorithm model corresponding to the unit task is divided into multiple sub-models that are executed sequentially according to the computational logic; The sub-models are assigned to different edge computing nodes; During execution, the output of the sub-model of the previous node serves as the input of the sub-model of the next node, so as to collaboratively complete the calculation of the unit task.

[0015] In some embodiments of the present invention, the startup rescheduling mechanism includes: The failed schedulable subtasks are put back into the task waiting queue and their priorities are recalculated. The recalculated priorities of the schedulable subtasks will be used in the scheduling decisions of subsequent scheduling cycles.

[0016] On the other hand, the present invention also provides an affinity-based edge computing network damage-resistant dependency task scheduling system, including a processor, a memory, and a computer program or instructions stored in the memory, wherein the processor is used to execute the computer program or instructions, and when the computer program or instructions are executed, the system implements the steps of any of the methods mentioned above.

[0017] This invention provides an affinity-based edge computing network damage-resistant dependency task scheduling method and system, which has the following beneficial effects: By employing task partitioning and directed acyclic graph (DAG) modeling, parallel processing of high-performance computing tasks was achieved, effectively reducing overall task execution latency. Simultaneously, task-node affinity determination ensured efficient matching of computing tasks with heterogeneous edge resources, avoiding resource mismatch and significantly improving system resource utilization.

[0018] By introducing task-to-task anti-affinity determination and critical subtask redundancy calculation mechanisms, the system can proactively identify critical tasks and schedule multiple replicas when resources permit, effectively avoiding the blocking of the entire task flow due to single-point failures. Combined with the subtask rescheduling mechanism, the system has the ability to quickly self-heal from node failures, greatly improving task completion rate and system stability in unreliable edge environments.

[0019] By combining graph neural networks with multi-head attention mechanisms and reinforcement learning, this approach can automatically learn and extract key features from complex task dependencies and dynamic network environments to generate optimal scheduling strategies. This enables the scheduling system to not only have a deep understanding of the current task topology but also possess strong generalization capabilities, allowing it to adapt to diverse and dynamically changing edge computing scenarios and overcoming the poor generalization shortcomings of traditional scheduling algorithms.

[0020] The dynamic priority calculation model, which integrates multiple dimensions such as task importance, latency flexibility, and timeliness, can ensure that highly urgent and highly dependent tasks are executed first, while preventing tasks from "starving" through timeliness factors. This ensures the deadlines of critical tasks and comprehensively guarantees the service quality needs of users.

[0021] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0022] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1This is a schematic diagram illustrating the steps of an edge computing network damage-resistant dependency task scheduling method based on affinity in one embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the framework of an edge computing network damage-resistant dependency task scheduling method based on affinity in one embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of task dependency relationship types in one embodiment of the present invention.

[0026] Figure 4 This is a schematic diagram of a model segmentation task in one embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of a graph neural network layer structure based on multi-head attention in one embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0029] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0030] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0031] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0032] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0033] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0034] To address the problems of high latency, weak generalization ability of scheduling strategies, and poor overall system robustness in edge computing networks caused by insufficient dependency modeling capabilities, poor adaptability to dynamic environments, and lack of systematic fault tolerance mechanisms in existing technologies, this invention provides an affinity-based edge computing network anti-damage dependency task scheduling method. This method is applied to edge computing networks containing multiple edge computing nodes, such as... Figure 1 As shown, the method includes the following steps S101~S106: Step S101: Receive the dependent tasks to be processed, and determine whether to split the dependent tasks based on the resource requirements of the dependent tasks and the preset node resource threshold. If it is determined that splitting is required, the dependent tasks are divided into multiple sub-tasks with dependencies, and a directed acyclic graph representing the dependencies and data transmission volume between sub-tasks is constructed. Duplicate sub-tasks in the directed acyclic graph are merged to generate a unified directed acyclic graph.

[0035] Step S102: Based on the resource requirements of the subtask, select a set of candidate nodes that meet the conditions from the edge computing network, and detect all schedulable subtasks that have completed the execution of the preceding tasks or have no preceding tasks from the current task waiting queue.

[0036] Step S103: Based on a preset priority calculation model, calculate the priority of each schedulable subtask and sort the schedulable subtasks according to their priorities. The priority calculation model comprehensively considers at least the importance of the task, latency flexibility, task execution speed, task timeliness, and task dependency factors.

[0037] Step S104: Perform task-node affinity and task-task anti-affinity determination on schedulable subtasks, identify key subtasks based on the determination results, and initiate redundant computation for key subtasks when resources permit. Specifically, task-node affinity is used to label the main resource requirement type of a task and the remaining main resource types within a node, while task-task anti-affinity is used to label the importance of a task in its corresponding directed acyclic graph.

[0038] Step S105: Using a graph neural network based on a multi-head attention mechanism, the features of the schedulable subtasks and the features of the candidate node set are fused and enhanced to extract enhanced features containing the dependencies between tasks and nodes. The enhanced features and the real-time network link status are input into a scheduling decision model based on reinforcement learning to obtain the scheduling decision of unloading the schedulable subtasks to the target node in the candidate node set.

[0039] Step S106: Execute task allocation according to scheduling decisions, monitor execution status, and initiate a rescheduling mechanism for subtasks that fail to execute but have not exceeded their maximum tolerable latency.

[0040] like Figure 2 The diagram shows a framework for an affinity-based edge computing network damage-resistant dependency task scheduling method.

[0041] Step S101 mainly involves task partitioning and the construction of a directed acyclic graph (DAG). This step is the preprocessing and modeling stage of the entire scheduling scheme. Its core purpose is to decompose complex, high-computing-power tasks in a structured manner, laying the foundation for subsequent intelligent scheduling.

[0042] This invention proposes a subtask partitioning model. When the scale of dependent tasks is large, the task is partitioned. After partitioning, the subtasks can be processed in parallel to solve resource bottlenecks, reduce execution latency, and improve system resource utilization.

[0043] In some embodiments, such as Figure 3 As shown, the task dependencies between the divided subtasks fall into four categories: no dependency, sequential dependency, parallel dependency, and general dependency. Taking sequential dependency as an example, when executing the task forward, the result needs to be calculated for each subtask according to the order of division.

[0044] Therefore, when a new dependency task is received, it is first determined whether the resource requirement of the dependency task exceeds the resource threshold of the node. If it does not exceed the threshold, there is no need to split it, and the task is directly scheduled as a whole. If it exceeds the threshold, it is split into multiple subtasks according to the internal logic of the task and a DAG is constructed.

[0045] In some embodiments, DAG construction mainly includes the following steps: Node definition: Each subtask obtained from the partitioning is defined as a node in the DAG. Node information typically includes the computational cost of the subtask and the type of resources required (e.g., CPU-intensive, memory-intensive).

[0046] Edges and Dependencies: Directed edges connect subtasks that have dependencies. For example, if subtasks... Must be in subtask If executed previously, a subtask will be created. Pointing to subtasks The directed edge.

[0047] Edge weights and data transmission: Each directed edge is assigned a weight to quantify the amount of data transmitted between the corresponding two task nodes. This links the "computational dependency" and "communication overhead" in scheduling, ensuring that the scheduler must consider the transmission latency between nodes when making decisions. Based on the task nodes, directed edges and their weights mentioned above, a DAG can be constructed.

[0048] In some embodiments, considering that multiple nodes may request the same type of dependent tasks, and these dependent tasks may contain duplicate subtasks. For example, simultaneously searching a set of related data, or simultaneously recognizing the same set of images. Therefore, to improve resource utilization and system performance, after identifying duplicate subtasks (such as those of the same type) in different DAGs, the system merges them into a shared task through a set union operation, integrates their DAG structures, and finally adds a virtual entry task and a virtual exit task as unique task start and end nodes to form a unified DAG.

[0049] Specifically, assume the system detects duplicate subtasks in the DAGs of different dependent tasks within the same time slot. The two dependent tasks are represented by formulas (1) and (2), respectively: ; (1) ; (2) Remove duplicate subtask nodes from their respective sets and merge them into a shared node. Perform a union operation on the edge set and weights of the original DAG to obtain formula (3): ; (3) in, , , The three terms respectively satisfy the following formulas (4), (5), and (6): ; (4) ; (5) ; (6) in, Indicates at time The DAG corresponding to the first dependent task that needs to be merged. , Similarly, each DAG consists of three elements: nodes, edges, and edge weights. Taking formula (1) as an example, express The set of all subtask nodes in this DAG. express The set of all directed edges in this DAG. express The set of weights for each edge in this DAG. This indicates the union operation.

[0050] In some embodiments, for resource-intensive and dependent tasks that cannot be broken down by function, they are treated as unit tasks, and their internal computing models (such as machine learning models) are divided into multiple sequentially executed sub-models according to computing logic and distributed to different nodes for collaborative computing.

[0051] Specifically, such as Figure 4 As shown, a machine learning model will be used as an example for explanation. Let's assume this machine learning model is denoted as... It is divided into sub-model sequences by layer. ,in, This represents the number of nodes participating in the computation; the hidden layer output is denoted as... .

[0052] In the forward propagation, the data passes through the sub-model at the first node. Calculations were performed to obtain intermediate results. ; It is transmitted to the second node as a sub-model. The input is repeated until the final result is obtained.

[0053] In backward updates, that is, during training, the gradient of the loss function propagates backward from the last node back to the first node in the opposite direction, and each node updates the parameters of the sub-model it is responsible for. .

[0054] In step S102, based on the type of the segmented subtask and resource requirements (such as the need for GPU, large memory, etc.), the real-time status of all nodes in the edge computing network, such as remaining CPU, memory, GPU resources, dedicated accelerators, etc., is matched and filtered to generate a set of candidate nodes that meet the conditions for each subtask.

[0055] Iterate through the current task waiting queue and examine each subtask. Filter out all subtasks whose predecessor tasks have been successfully executed or have no predecessor tasks, mark them as ready tasks or schedulable tasks, and wait for scheduling.

[0056] In step S103, after obtaining the schedulable subtasks, the priority of each schedulable subtask is calculated using preset dynamic and comprehensive evaluation criteria to determine which task should be scheduled first.

[0057] In this invention, the priority calculation model is a multi-factor weighted decision system that integrates factors such as task importance, time delay flexibility, task execution speed, task timeliness, and task dependency.

[0058] Task importance measures a task's structural influence on the overall system, preventing its failure to execute from impacting the processing of subsequent tasks. A higher out-degree (how many tasks it depends on) means it affects more subsequent tasks; a higher in-degree (how many tasks depend on it) may indicate it's a convergence point for multiple computational paths. Furthermore, a high number of reuses suggests it might be a shared computational unit, indicating higher importance.

[0059] Latency elasticity is characterized by the reciprocal of the maximum tolerable execution time of a task, and is used to characterize the urgency of the task. The maximum tolerable latency is the final deadline that a subtask must complete, indicating the task's latency elasticity, i.e., the time remaining until the deadline. A smaller tolerable latency means that the task has low latency elasticity and high urgency. Slow execution of subtasks will affect the execution latency of the corresponding high-computing-power task; the latency elasticity of a subtask is determined based on its task size.

[0060] Task execution speed is represented by the reciprocal of the task's computational load. Specifically, the lower a task's computational resource requirements, the faster it executes. Prioritizing the execution of such tasks helps to quickly free up computational resources in environments with limited computing resources, reducing queuing time for subsequent tasks and minimizing task backlog.

[0061] Task timeliness is the ratio of task waiting time to the task's maximum tolerable time. It quantifies the severity of a task remaining unprocessed due to its consistently low priority. The closer the ratio is to 1, the more urgent the task needs to be executed.

[0062] Task dependencies are calculated by the system based on the priorities of all preceding tasks of the given task and the amount of data transfer between them. For example, suppose a preceding task of this task has a priority of... Then task dependency can be expressed as That is, the amount of data transferred between tasks. It is related to the priority of the preceding tasks, and task dependency can ensure that the priority of the subsequent tasks is always lower than that of the preceding tasks.

[0063] In some embodiments, normalizing the above five indicators yields the priority of schedulable subtasks, as shown in formula (7): ; (7) in, Indicates schedulable subtasks in Time priority; , , , , All are weighting factors, and their sum is 1; , , , , These represent the normalized task importance, latency flexibility, task execution speed, task timeliness, and task dependency, respectively.

[0064] After priority calculation is completed, the schedulable subtasks in the task waiting queue of each node are sorted from high to low priority.

[0065] In step S104, the system adds resource type tags to each schedulable subtask and each edge computing node, and adds a criticality tag to each subtask. This includes: Task-node affinity: This is used to identify the main resource requirement type of a task and the remaining main resource types within a node. It can be understood as a matching rule that aims to schedule the task to the node with the best matching resource type.

[0066] Task-Task Anti-Affinity: Used to indicate the importance of a task in its corresponding directed acyclic graph. It can be understood as an isolation rule, which aims to distribute highly critical tasks to avoid single points of failure.

[0067] Based on task-task anti-affinity tags, critical subtasks can be identified, and redundant computations can be initiated when resources permit. This involves proactively creating and scheduling replicas of these tasks on multiple different nodes. It is a proactive and forward-looking fault tolerance mechanism that ensures the safety of the critical path by executing backup tasks in advance before a failure occurs.

[0068] In step S105, feature enhancement is first performed based on a graph neural network with multi-head attention to transform the original, isolated task and node features into enhanced features rich in context and topological relationships. This enables the subsequent reinforcement learning scheduler to make more intelligent and robust scheduling decisions based on a deep understanding of global task dependencies and node network structure, rather than making decisions based on superficial, independent features.

[0069] Each subtask and each candidate node is considered as a node in a graph. Their dependencies (between tasks), network connections (between nodes), and possible task-node matching relationships constitute the edges of the graph. The set of all initial node features constitutes the input matrix. For example, the features of a task node can be [computational cost, required memory, latency resilience]; the features of a node can be [remaining CPU, remaining memory, network bandwidth].

[0070] Different weight matrices are used to perform linear transformations on the initial node feature set (i.e., the input matrix) to project the original features into different semantic spaces, resulting in the query matrix, key matrix, and value matrix, as shown in formulas (8), (9), and (10): ; (8) ; (9) ; (10) in, Represents the query matrix; Represents the key matrix; Represents a value matrix; Represents the initial set of node features; , , All of these are trainable weight matrices.

[0071] Based on the query matrix, key matrix, and value matrix, the attention score matrix is ​​calculated as shown in formula (11): ; (11) in, Represents the attention score matrix; For activation functions; Indicates transpose; This indicates the dimension of the key matrix.

[0072] The calculated attention score matrix is ​​the processed data that includes the dependencies between the data points. When calculating the representation of each node in the graph, the attention-based graph neural network assigns different weights to it based on the characteristics of its neighboring nodes. Here, the attention score matrix... It can be viewed as a single-layer feedforward neural network, such as Figure 5 As shown.

[0073] The attention coefficients between nodes are obtained by linearly transforming the initial node feature set using the attention score matrix, as shown in formula (12): ;(12) in, Represents task node and task nodes Attention coefficient between them; Represents the weight matrix; Represents task node Features; Represents task node Its characteristics.

[0074] In practical graph attention networks, full graph connectivity is usually not performed; instead, only the first-order neighbor nodes with the highest influence are considered. The attention coefficients are shown in formula (13). (13) in, Represents task node and task nodes Attention coefficient between them; This represents the linear rectification function; Represents the attention score matrix; Indicates transpose; Represents the weight matrix; Indicates a splicing operation; Used to traverse task nodes All first-order nearest neighbors; Represents first-order nearest neighbor nodes Its characteristics.

[0075] Based on the attention coefficient, a multi-head attention mechanism is adopted to aggregate the outputs of multiple attention heads, and a nonlinear transformation is performed through a preset activation function to obtain the final enhanced features containing dependencies, as shown in formula (14): ;(14) in, Indicates enhanced features; Represents a nonlinear activation function; This represents the total number of heads receiving multi-head attention. Indicates the first Attention on the task node For task nodes The normalized attention coefficient; No. The weight matrix of each attention head.

[0076] Then, a scheduling decision is generated based on reinforcement learning. This step is the intelligent decision-making center of the entire system. By receiving enhanced features containing rich topology information and real-time network status, it outputs an optimal scheduling strategy. Its core purpose is to autonomously find a scheduling rule that minimizes the overall task execution latency through continuous learning, thereby replacing traditional heuristic algorithms that rely on human experience and achieving dynamic optimization and adaptability of scheduling efficiency.

[0077] First, the construction and training of the scheduling decision model will be explained.

[0078] The reinforcement learning scheduling decision model of this invention is constructed and trained using the Actor-Critic algorithm framework. Its training process is a continuous, iterative process involving interaction with the simulation environment. Specifically: First, the scheduling problem is modeled as a Markov Decision Process (MDP) to formalize the scheduling problem, providing a basis for reinforcement learning to solve it.

[0079] The state space includes information such as augmented features, real-time network link states, and computation task states. Among them, augmented features refer to features that contain the dependencies between tasks and nodes after processing by the graph attention network; real-time network link states include information such as bandwidth, latency, and packet loss rate between nodes; and computation task states include information such as task out-degree and the maximum distance of the task from the final exit task (critical path).

[0080] Action space: The scheduling decision made by the scheduling decision model, specifically: for one or more currently schedulable subtasks, select a target node from the set of candidate nodes.

[0081] Reward: A scalar signal fed back by the environment after an action is performed. In this invention, the negative value of the task execution latency is used as the primary reward. Thus, maximizing the cumulative reward is equivalent to minimizing the overall task execution latency.

[0082] The model is trained using the Actor-Critic framework, and its process is a closed loop: The Actor network (policy network) outputs an action based on the current state (e.g., which node the task should be scheduled to). This action is executed in a simulation environment, the task execution delay is calculated, a reward is obtained, and the new system state is observed. This interaction experience is recorded as a state transition sample. Store in the experience replay pool.

[0083] A batch of state transition samples were randomly sampled from the experience replay pool.

[0084] Updating the Critic Network (Value Network): The Critic Network is used to evaluate the quality of performing action A in state S. The Critic Network parameters are updated by calculating the temporal difference error between the predicted value and the target value generated by the target network, and minimizing this error. This is equivalent to making the Critic more accurately predict future cumulative rewards.

[0085] Update the Actor network: Use the updated Critic network to evaluate the effectiveness of the policies adopted by the Actor network. Using the policy gradient method, adjust the parameters of the Actor network in directions that increase the Critic's output value (i.e., expected reward). This is equivalent to teaching the Actor to choose actions that yield higher rewards.

[0086] Repeat the above process until the model's performance (i.e., average reward / negative latency) converges to a high level, training is complete, and the scheduling decision model required by this invention is obtained for online scheduling in actual edge computing environments.

[0087] The enhanced features output by the graph attention network and the real-time network link state are combined to form the state, which is then input into the pre-trained Actor network. The output is a specific action (scheduling decision), which explicitly specifies which specific target node in the candidate node set should be unloaded to each schedulable subtask.

[0088] In step S106, the system executes the scheduling decision obtained in step S105, assigning tasks to designated nodes for computation. The system monitors the execution status and initiates a rescheduling mechanism for subtasks that fail but do not exceed their maximum tolerable latency, ensuring that the entire dependent task flow is not completely blocked or fails due to localized faults. This significantly enhances the system's fault tolerance and task completion rate in dynamic and unreliable edge computing environments.

[0089] Specifically, in each scheduling time slot (one unit of time), a status check is performed on all executing subtasks to detect the execution time of each task and compare it with the expected execution time of the task.

[0090] If the execution time equals the estimated execution time: the task is considered complete. The system will safely remove the task from the global DAG and the current ready queue, marking the successful completion of the task and potentially triggering its subsequent tasks to become schedulable.

[0091] If the execution time exceeds the expected execution time: the task is considered to have failed or timed out severely. This usually means that the node hosting the task may be damaged, frozen, or under extreme high load. The task deemed failed will be re-inserted into the task ready queue.

[0092] After processing all task states, the system re-examines the currently schedulable tasks (because some tasks may have been completed, releasing new schedulable tasks), and recalculates the priorities of all ready tasks (including newly added and rescheduled ones). Finally, it enters the next scheduling cycle and generates a new scheduling policy.

[0093] In summary, this invention provides an affinity-based edge computing network damage-resistant task scheduling scheme. By deeply integrating task segmentation and graph modeling, dynamic priority scheduling, affinity determination and redundant computation, and intelligent decision-making through graph neural networks and reinforcement learning, it can significantly reduce the execution latency of high-computing-power-dependent tasks and effectively cope with dynamic uncertainties such as node damage. While improving scheduling efficiency, it greatly enhances the robustness and adaptability of the system in the edge computing network environment.

[0094] Corresponding to the above method, the present invention also provides an affinity-based edge computing network damage-resistant dependency task scheduling system, including a processor, a memory, and a computer program or instructions stored in the memory. The processor is used to execute the computer program or instructions, and when the computer program or instructions are executed, the system implements the steps of the method described above.

[0095] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0096] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0097] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0098] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An affinity-based edge computing network damage-resistant dependency task scheduling method, characterized in that, The method is applied to an edge computing network containing multiple edge computing nodes, and the method includes: The system receives dependent tasks to be processed and determines whether to split the dependent tasks based on their resource requirements and preset node resource thresholds. If splitting is required, the system divides the dependent tasks into multiple sub-tasks with dependencies and constructs a directed acyclic graph (DAG) representing the dependencies and data transmission volume between the sub-tasks. DAGs that are repeated in the DAG are merged to generate a unified DAG. Based on the resource requirements of the subtask, a set of candidate nodes that meet the conditions is selected from the edge computing network, and all schedulable subtasks that have completed their preceding tasks or have no preceding tasks are detected from the current task waiting queue. Based on a preset priority calculation model, the priority of each schedulable subtask is calculated, and the schedulable subtasks are sorted according to the priority; wherein, the priority calculation model takes into account at least the importance of the task, the latency flexibility, the task execution speed, the timeliness of the task, and the task dependency factors. The schedulable subtasks are evaluated for task-node affinity and task-task anti-affinity. Based on the evaluation results, key subtasks are identified, and redundant computation is initiated for the key subtasks when resources permit. The task-node affinity is used to mark the main resource requirement type of the task and the main resource type remaining in the node, and the task-task anti-affinity is used to mark the importance of the task in its corresponding directed acyclic graph. Using a graph neural network based on a multi-head attention mechanism, the features of the schedulable subtask and the features of the candidate node set are fused and enhanced to extract enhanced features containing the dependencies between tasks and nodes. The enhanced features and the real-time network link status are input into a scheduling decision model based on reinforcement learning to obtain the scheduling decision of unloading the schedulable subtask to the target node in the candidate node set. The scheduling decision is executed based on task allocation, and the execution status is monitored. For subtasks that fail to execute but do not exceed their maximum tolerable latency, a rescheduling mechanism is initiated.

2. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, The construction of the directed acyclic graph representing the dependencies and data transfer volume between the subtasks includes: Each of the divided subtasks is treated as a task node in a directed acyclic graph; Connect dependent subtasks with directed edges; Each directed edge is assigned a weight, which is used to characterize the amount of data transmission between the corresponding two task nodes; Based on the task nodes, the directed edges, and the weights, the directed acyclic graph is constructed.

3. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, The step of merging repeated subtasks in the directed acyclic graph to generate a unified directed acyclic graph includes: Identify repetitive subtasks of the same type from different dependent tasks; The edges of each directed acyclic graph containing the repeated subtasks before merging are combined into a single shared node to generate the unified directed acyclic graph. Virtual entry and exit tasks are added to the unified directed acyclic graph to form unique task start and end nodes.

4. The edge computing network anti-damage dependency task scheduling method based on affinity as described in claim 1, characterized in that, The latency elasticity is characterized by the reciprocal of the maximum tolerable time for task execution, the task execution speed is characterized by the reciprocal of the task computational load, and the priority is calculated using the following formula: ； in, This indicates that the schedulable subtask is in Time priority; , , , , All are weighting factors, and their sum is 1; , , , , These represent the normalized task importance, latency flexibility, task execution speed, task timeliness, and task dependency, respectively.

5. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, Extracting the enhanced features using the graph neural network includes: An initial node feature set is constructed based on the features of the schedulable subtasks and the features of the candidate node set. The initial node feature set is linearly transformed using different weight matrices to obtain the query matrix, key matrix, and value matrix, and the attention score matrix is ​​calculated. Based on the attention score matrix, calculate the attention coefficient between the task node and its first-order neighboring task nodes; Based on the attention coefficients, a multi-head attention mechanism is adopted to aggregate the outputs of multiple attention heads, and a nonlinear transformation is performed through a preset activation function to obtain the final enhanced features containing dependencies.

6. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 5, characterized in that, The attention score matrix is ​​calculated using the following formula: ； in, This represents the attention score matrix; Represents the query matrix; Represents the key matrix; Represents the value matrix; For activation functions; Indicates transpose; This indicates the dimension of the key matrix; The attention coefficient is calculated using the following formula: ； in, Represents task node and task nodes Attention coefficient between them; This represents the preset activation function; This represents the attention score matrix; Indicates transpose; Represents the weight matrix; Indicates a splicing operation; Represents task node Features; Represents task node Features; Used to traverse task nodes All first-order nearest neighbors; Represents first-order nearest neighbor nodes Its characteristics.

7. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, The method further includes training the scheduling decision model using the Actor-Critic algorithm framework, including: A state space is constructed using the enhanced features output by the graph neural network and the real-time network link state, wherein the enhanced features include enhanced task features and node features; The action space is constructed based on scheduling decisions that allow schedulable subtasks to be unloaded to the target node; The reward is a negative value for the task execution delay; State transition samples are collected by interacting with the environment and stored in the experience replay pool; A batch of state transition samples are randomly sampled from the experience replay pool, and the parameters of the Critic network are updated by minimizing the error between the predicted value of the Critic network and the target value calculated based on the target network. By using the policy gradient method and the gradient signal provided by the updated Critic network, the parameters of the Actor network are updated to maximize the expected cumulative reward, and the scheduling decision model is finally trained.

8. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, The method further includes, during the process of dividing the dependent task into multiple dependent subtasks, if the resource requirements of the divided subtasks still exceed the node resource threshold and cannot be further functionally divided, then treating them as a single task and performing algorithmic division, including: The algorithm model corresponding to the unit task is divided into multiple sub-models that are executed sequentially according to the computational logic; The sub-models are assigned to different edge computing nodes; During execution, the output of the sub-model of the previous node serves as the input of the sub-model of the next node, so as to collaboratively complete the calculation of the unit task.

9. The edge computing network damage-resistant dependency task scheduling method based on affinity as described in claim 1, characterized in that, The startup rescheduling mechanism includes: The failed schedulable subtasks are put back into the task waiting queue and their priorities are recalculated. The recalculated priorities of the schedulable subtasks will be used in the scheduling decisions of subsequent scheduling cycles.

10. An affinity-based edge computing network robust task scheduling system, comprising a processor, a memory, and computer programs or instructions stored in the memory, characterized in that, The processor is configured to execute the computer program or instructions, and when the computer program or instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 9.

Images