Task-Oriented Communication and Aware Resource Management for DAG-Based Edge AI Inference

Abstract

The invention relates to a resource management framework that enables resource-constrained Internet of Things (NI) devices to offload latency-sensitive AI inference workloads to edge servers, performing directed loop graph (DGP)-based task scheduling under stochastic network conditions. The resource management framework can be used for multi-stage pipeline workflows such as real-time video analysis, intelligent transportation, and industrial monitoring. It allows operators to monitor resource utilization and deadline compliance in edge infrastructure in real-time, anticipate and respond to HCI (Health, Safety, and Environment) violation risks, and serve more enterprise workflows on the same physical infrastructure.

Description

1 TARIFF Task-Oriented Communication Awareness Resource for DAG-Based Edge AI Inference Management Technical Area The invention enables latency-sensitive artificial intelligence inference in resource-constrained Internet of Things (IoT) devices. It enables the transfer of workloads to edge servers under stochastic network conditions. with a resource management framework that performs task scheduling based on non-looping graph (NAG) It is related. 10 State of the Art Current DAG scheduling methods have two main shortcomings. The first shortcoming is that the existing Schedulers should assign equal value to all tasks and consider the contribution of each task to inference quality. It is a matter of ignoring it. Classical methods like HEFT, which are based on the earliest completion time criterion, 15 It assumes deterministic communication cost and instantaneous failure with variable bandwidth. It cannot take into account the possibility. The second shortcoming is that the task-oriented communication approach is only for individuals. its focus on link transmission and DAG in multi-hop stochastic network conditions The problem is that it is not integrated with the scheduling. From the operator's perspective, this situation is in the edge infrastructure. 20 unpredictable increases in delays and the path to HKA violations in critical workflows This means a lack of visibility. Due to the negative aspects described above and the current situation... Due to the inadequacy of the solutions regarding the issue, an improvement should be made in the relevant technical field. It has been made necessary. Purpose of the Invention 25 The invention was created by drawing inspiration from existing situations and overcoming the aforementioned drawbacks. It aims to solve the problem. The main purpose of the invention is to improve the latency sensitivity of resource-constrained Internet of Things (IoT) devices. Stochastic network conditions enable AI to offload inference workloads to edge servers. a resource management system that performs directed non-looping graph (DGP) based task scheduling under 30 The aim is to provide a framework. Another aim of the invention is real-time video analysis, intelligent transportation, and industrial applications. a resource management framework that can be used for multi-stage pipeline workflows such as monitoring to provide. Another objective of the invention is to improve resource utilization in the operator's edge infrastructure and the deadline of 35. enabling real-time monitoring of compliance, anticipating and intervening in the risk of HKA violations. 2 to enable and serve more enterprise workflow services on the same physical infrastructure. The goal is to enable it to provide; an integrated scheduling framework consisting of four components. The goal is to provide; DAG analyzer and critical path rater models the workflow and each task The task is to calculate the criticality score; the inference kernel and decision are determined using a task weight assigner. By assigning greater weight to tasks of high importance, such as those mentioned above, resource allocation can be improved. The goal is to relate it to the task value; bandwidth and with the Monte Carlo cost estimation engine. by modeling the delay variability with a log-normal distribution for each task and node pair The goal is to generate mean and variance estimates; significance is determined using a composite timing rater, and finally... a utility function that multiplies the proximity of the delivery date and node reliability The goal is to assign each remaining task after pruning to the most suitable node; adaptive DAG pruning 10 Scheduling low-value branch tasks that are not located on the critical path using the mechanism. The goal is to significantly reduce the communication load by eliminating it beforehand; operator scheduling and With the service quality panel, you can monitor all these steps in real time and anticipate the risk of HKA violations. It is a way of conveying a warning. Figures that will help understand the invention. Figure 1 shows the general architecture of the system that is the subject of the invention. Explanation of Part References 1. DAG Analyzer and Critical Path Rater 2. Task Meaning Weight Assigner 20 3. Monte Carlo Cost Estimation Engine 4. Compound Timing Rater 5. Adaptive DAG Pruning Mechanism 6. Operator Scheduling and Service Quality Panel Detailed Description of the Invention This detailed description outlines the preferred configurations of the invention, not only for better understanding the subject matter. It is intended to facilitate understanding and will not impose any limiting effects. The invention is limited by resource constraints. Internet of Things (IoT) devices enable latency-sensitive AI inference workloads. Directed Non-Cyclic Graph (DCG) 30 under Stochastic Network Conditions, enabling data transfer to servers. It is a resource management framework that performs task scheduling based on real-world scenarios. Its areas of use are real-world scenarios. real-time video analysis, smart transportation, and industrial monitoring are examples of multi-stage pipeline projects. These are the flows. Through this framework, the operator monitors resource utilization in the edge infrastructure and the end We can monitor deadline compliance in real time, and anticipate and intervene in case of HKA (Health, Safety, Environment) violation risks. It can and will serve more enterprise workflows on the same physical infrastructure. 35 3 The resource management framework described in the invention is an integrated scheduling system consisting of four components. It provides a framework. DAG analyzer and critical path rater (1) model the workflow and each Calculates the criticality score for the task. Task meaning weight assigner (2), inference kernel and decision by allocating resources by giving greater weight to tasks of high importance, such as those mentioned above. It associates the task value with the Monte Carlo cost estimation engine (3), bandwidth and 5 by modeling the delay variability with a log-normal distribution for each task and node pair It produces mean and variance estimates. Compound timing rater (4), meaning significance, final a utility function that multiplies the proximity of the delivery date and node reliability It assigns all remaining tasks after pruning to the most suitable node. Adaptive DAG pruning. mechanism (5), scheduling low-value branch tasks not located on the critical path 10 By pre-filtering, it significantly reduces the communication load. Operator scheduling and Service Quality Panel (6) monitors all these steps in real time and anticipates the risk of HKA violations. It is transmitted as a warning. The elements and functions used within the framework of resource management, which is the subject of this invention, are as follows: DAG Analyzer and Critical Path Rater (1), workflow directional loopless graph (DAG) 15 Models are used to assign a lower-level priority (BL) score and a normalized criticality score for each task. Accounts. Critical tasks are routed to fast nodes. The Task Meaning Weight Assigner (2) assigns a significance weight to each task according to its meaning type. Inference Core and decision tasks receive the highest weight. Low-weight tasks are pruning candidates. Okay. 20 Monte Carlo Cost Estimation Engine (3) bandwidth and delay variability log-normal by modeling with distribution, averaging from independent network replicates for each task and node pair. It produces a variance estimate. Compound Timing Scorer (4), meaning significance, deadline proximity and node Calculates a task-node utility function that combines reliability in a multiplicative manner. High 25 For the benefit to kick in, all multipliers must simultaneously exceed the threshold. Adaptive DAG Pruning Mechanism (5), not located on the critical path and low significance It eliminates major branch tasks before scheduling. It significantly reduces the communication burden. It reduces and does not create deadline violations. The Operator Scheduling and Service Quality Panel (6) defines the meaning type of each task, to which it is assigned. The node monitors completion delays and deadline compliance status in real time. It allows the operator to set pruning thresholds and define priority rules. HKA violation. It conveys the risk as an early warning. The invention's resource management framework performs the following functions: DAG solver (1) decomposes the workflow as a directed non-cycle graph and 35 for each task Calculation of criticality score using low-level priority score, 4 By assigning the task meaning weight assigner (2) to each task the meaning type specific meaning weight assigner marking pruning candidates, Branches that do not meet the criticality and weight thresholds of the adaptive DAG pruning mechanism (5) leaving tasks off schedule, The Monte Carlo cost estimation engine (3) independent network for each remaining task and node pair 5 Calculating average delay and variability estimates from its copies, The meaning of the compound timing scorer (4) for each of the remaining tasks after pruning The most suitable node by combining importance, proximity to deadlines, and node reliability. choosing, Assigning a task to the selected node and updating the node order, 10 The operator scheduling panel (6) displays completed task delays and HKA compliance status. and real-time reporting of resource utilization efficiency, If a risk of HKA violation is detected, the operator panel (6) is sent to the AIM team for the relevant workflow. Sending a request for increased priority. The working principle of the resource management framework covered by the invention is as follows: DAG Analyzer 15 and Critical Path Rater (1) decomposes the workflow as a directed loopless graph and each Calculates the criticality score of the task. Task Meaning Weight Assigner (2) assigns a meaning type to each task. It assigns importance weights accordingly and marks low-weight tasks as pruning candidates. Adaptive DAG Pruning Mechanism (5), not located on the critical path and low Eliminates weighted branch tasks before scheduling. Monte Carlo Cost Estimation Engine (3), 20 Bandwidth and latency variability for each remaining task and node pair is log-normal. It generates mean and variance estimates by modeling the distribution. Compound Timing Scorer (4), meaning, proximity to deadline and node for each of the remaining tasks By multiplying their reliability, it selects the most suitable node and assigns the task to that node. Operator Scheduling and Service Quality Panel (6) monitors all results in real time and 25 The reports send a priority upgrade request to the AIM team if a risk of HKA (Health, Safety, and Environment) violation is detected.

Claims

REQUESTS 1. Latency-sensitive AI inference tasks in resource-constrained Internet of Things (IoT) devices. Enables the transfer of payloads to end servers, directional loop-free under stochastic network conditions. It is a resource management framework that performs graph-based task scheduling (TAG), and its feature is; 5 that decomposes the workflow as a directed, non-looping graph and calculates the criticality score of each task. DAG analyzer and critical path grader (1), It assigns importance weights to each task according to its meaning type and prunes low-weighted tasks as candidates. task meaning weight assigner (2), which marks as such, Schedule low-weight branch missions that are not located on the critical path beforehand. The adaptive DAG pruning mechanism (5), 10 Bandwidth and latency variability for each remaining task and node pair is log-normal. Monte Carlo cost estimation produces mean and variance estimates by modeling the distribution. engine (3), For each of the remaining tasks, the significance, deadline proximity, and node reliability are important. Compound 15, which selects the most suitable node by combining them multiplywise and assigns the task to that node. timing scorer (4), Monitoring and reporting all results in real time, and notifying the AIM team if a risk of HKA violation is detected. Priority upgrade call sending operator scheduling and service quality panel (6) It includes.

2. It is a resource management framework compliant with Request 1, and its features include a DAG analyzer and critical path 20. The rater (1) lower priority (BL) score and normalized criticality for each task It involves calculating the score and directing critical tasks to fast nodes.

3. It is a resource management framework that complies with Request 1, and its feature is; task meaning weight assigner (2) It involves assigning the highest weight to the inference core and decision tasks.

4. It is a resource management framework that complies with Claim 1, and its feature is; a composite scheduling rater of 25. (4) all multipliers must exceed the threshold at the same time for the high benefit to come into play It requires or includes.

5. It is a resource management framework compliant with Claim 1, and its feature is adaptive DAG pruning. the mechanism (5) significantly reduces the communication load and deadline violation It does not include creating anything. 30 6. It is a resource management framework compliant with Claim 1, and its features include operator scheduling and service. The quality panel (6) determines the meaning type of each task, the node to which it is assigned, the completion delay and Real-time monitoring of deadline compliance, operator setting of pruning threshold. and includes the possibility of defining priority rules.

Images