EdgeManager framework based on Lyapunov optimization and AIoT system resource management method

Abstract

The invention discloses an EdgeManager framework and AIoT system resource management method and detection method based on Lyapunov optimization, the method comprises the first step of EdgeManager system model construction and the second step of dynamic resource management, and the dynamic resource management method comprises task dynamic allocation, communication and computing resource allocation, a HyDRL-MO hybrid optimization algorithm and Lyapunov optimization. And the Lyapunov optimization comprises queue stability processing based on the Lyapunov optimization and deep learning online resource management based on an EdgeManager framework. Through the EdgeManager framework provided by the invention, the resource management efficiency and reasoning performance in an edge intelligent environment can be remarkably improved, and effective support is provided for industrial automation, large-scale video monitoring and other application scenes with high QoS (Quality of Service) requirements.

Description

Technical Field

[0001] The present invention relates to the field of edge computing and artificial intelligence optimization, and specifically to an EdgeManager framework and an AIoT system resource management method based on Lyapunov optimization. Background Art

[0002] In recent years, the device-edge-cloud collaborative inference architecture has been proposed as an effective solution to these challenges. Collaborative DNN inference consists of two key phases: offline training and online inference. In the offline training phase, models are iteratively optimized, and pre-trained models of varying sizes (such as lightweight DNNs (LDNNs), medium-sized DNNs (MDNNs), and high-dimensional DNNs (HDNNs)) are stored on cloud servers. In the online inference phase, these models are deployed to edge servers or end devices to process real-time inference tasks. Although this collaborative framework has achieved some progress, current solutions primarily focus on DNN deployment and task offloading, and still suffer from three major issues: 1. Lack of holistic coordination between data quality, communication resources, and computing resources. 2. The inherent complexity of dynamic systems, which face combinatorial optimization problems involving discrete decisions and interdependent continuous resource variables. 3. Inadequate optimization mechanisms for dynamic task queuing systems. Efficient task queuing is crucial for processing real-time data streams, but many existing approaches ignore this issue, leading to increased system instability and long-term performance degradation.

[0003] To solve the above problems, this patent proposes the EdgeManager framework and its corresponding optimization method, aiming to integrate edge devices, edge cloud servers and centralized cloud servers to achieve collaborative online resource management and adaptive hierarchical DNN reasoning to adapt to heterogeneous edge environments. Summary of the Invention

[0004] In order to solve the resource management and DNN reasoning performance bottleneck problems in the current edge intelligence collaborative environment, especially in the dynamic scenario of multi-tasking, multi-user, and multi-server. To this end, the present invention proposes an EdgeManager framework, which aims to achieve online adaptive resource management and hierarchical DNN reasoning. The EdgeManager framework of the present invention can significantly improve the resource management efficiency and reasoning performance in the edge intelligence environment, providing effective support for industrial automation, large-scale video surveillance and other application scenarios with high QoS requirements.

[0005] The technical solution adopted by the present invention to solve its technical problems is: an EdgeManager framework and AIoT system resource management method based on Lyapunov optimization, wherein the method includes the following steps 1, EdgeManager system model construction and step 2, dynamic resource management method, the step 2 is based on the deep learning online resource management of the EdgeManager framework and specifically includes the state, action and reward module interaction process, the module interaction process includes a model-free actor module, a model-based critic module and a strategy update module, the model-free actor module uses DNN and quantizer to generate task allocation and model selection decisions, the model-based critic module evaluates multiple decision scenarios, and the strategy update module dynamically adjusts the decision strategy according to feedback.

[0006] Step 1 includes system architecture, pre-training DNN model, task definition, and inference accuracy modeling.

[0007] The system architecture first divides the system into the terminal equipment industrial gateway layer (IGW), the edge cloud server layer (ECS) and the centralized cloud server layer (CS);

[0008] The pre-trained DNN model first divides the DNN model into lightweight (LDNN), medium-sized (MDNN), and high-dimensional (HDNN). The pre-trained deep neural network DNN model is defined as Expressed as: Where n∈{0,1,2} represents the level of the model; Indicates that the model level n is in task U jk The inference accuracy on Indicates the size of the hierarchical model, the decision variable Indicates model selection, used to indicate the selected model level n, and its constraints are:

[0009] The task definition is that in each time slot t, the kth task of AIoT device j is defined as U jk (t), the data block size task is defined as D jk (t), the arrival rate of the task is defined as v jk (t); Task U jk (t) DNN model on the corresponding locations of industrial gateway (IGW), edge cloud server (ECS), and cloud server (CS) Availability of a jk Representation, definition and is a binary variable, where definition Respectively represent the task U jkDecisions allocated to the connected IGW, ECS, and CS, where Each task U jk Can only be assigned to one node.

[0010] Furthermore, the inference accuracy Affected by data size, noise level θ jk And the model type affects, when there is no noise θ jk =0, the reasoning accuracy of lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is θ jk =1 means the data is noisy, and the reasoning accuracy of lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is The total accuracy is expressed as:

[0011] The dynamic resource management method in step 2 includes dynamic task allocation, communication and computing resource allocation, the HyDRL-MO hybrid optimization algorithm, and Lyapunov optimization. The Lyapunov optimization includes queue stability processing based on Lyapunov optimization and deep learning online resource management based on the EdgeManager framework. The dynamic task allocation includes a task queue model, which is divided into IGW queue length and ECS queue length. The dynamic task allocation decision is controlled by the following variables:

[0012] Decision variables for task allocation to Industrial Gateway (IGW),

[0013] Decision variables for task allocation to edge cloud servers (ECS),

[0014] Decision variables for task allocation to centralized cloud servers (CS),

[0015] Task allocation follows the constraints:

[0016] The communication and computing resource allocation includes a task queue model and a computing delay model. The computing delay model is divided into IGW processing delay and ECS, CS processing delay; the task queue model IGW queue length calculation model Represents time slot U on IGW at time t jk The queue length is: t∈T, where represents the amount of tasks processed by the IGW in time slot t, represents the amount of tasks offloaded from IGW to ECS in time slot t, when reformulated without using {.,0} The IGW queue length can also be as follows: t∈T. The ECS queue length calculation is assumed to be Indicates the U processed by ECS jk Task volume, set Indicates the U processed by ECS jk Task volume, set For U jk The task queue: in Indicates the amount of data that is overloaded during transmission. The ECS queue can also be expressed as: t∈T.

[0017] When the task U jk When (t) is offloaded to IGW, ECS or CS, the calculation delay model is divided into IGW processing delay and ECS, CS processing delay. and (Unit: GHz) represents the IGW, ECS and CS in executing the task U jk The CPU frequency at that time. and Indicates the IGW, ECS and CS for task U jk The computing speed of other resources, in FLOPS. l , G e , and G c Represent the total computing capabilities of IGW, ECS, and CS respectively. and Represents the computing resources allocated to the inference task. The total completion latency of DNN inference on IGW, ECS, and CS is as follows: IGW: ECS: CS: The total service time U to complete the inference task i,t formula:

[0018] The HyDRL-MO hybrid optimization algorithm includes a multi-stage MINLP problem and a target optimization model; the multi-stage MINLP problem is proposed to include target optimization, and the optimization target is defined as a method that considers both accuracy and inverse delay benefits. To maximize the benefits of the system. The optimization model uses an objective function to quantify the system performance at any given time, defined as: Where (1-μ jk and (μ jk ) are the weights of inference accuracy and service time in the objective function, where (0≤μ jk ≤1). Let X={X t} t∈T represents the combination of all optimized variables over time. t Represents the optimization variable combination vector for time period t. Therefore, the multi-stage MINLP problem can be expressed as follows: Set constraints C1-C14. C1 ensures that the inference process can only be completed on one computing node. C2 ensures that only one pre-trained DNN model can be selected to complete the inference task in each time period. C3 ensures that the bandwidth resources from the IGW to the ECS and then to the CS do not exceed the total available bandwidth. C4 ensures that the computing resources used by all users do not exceed the total available capacity of the IGW, ECS, and CS. C5-C10 ensure that the CPU frequency allocated to users does not exceed the maximum CPU frequency of the IGW, ECS, and CS. C11-C12 ensure that the number of tasks processed in a time period does not exceed the number of tasks backlogged in the task queue during that time period. C13-C14 ensure that the average queue length does not exceed the queue threshold.

[0019] The queue stability processing based on Lyapunov optimization includes virtual queue definition, Lyapunov function and drift, drift plus penalty, and the Lyapunov function and drift are quadratic Lyapunov function and single time slot drift.

[0020] Virtual Queue Definition To solve the above C1-C14 problems, we use the Lyapunov optimization framework to transform all inequality and equality constraints into queue stability problems. Define a virtual queue with the update equation

[0021] Lyapunov function with drift and drift plus penalty: Consider a system with N queues, so use Θ(t) instead of (Q(t)) to represent the queue vector, and use represents the queue vector, where and is the vector of actual network queues, and is a properly chosen virtual queue. Therefore, the quadratic Lyapunov function is defined as: Where $w_1+w_2+w_3+w_4=1$, the function L(Θ(t)) is always non-negative, and it is equal to zero if and only if all components of L(Θ(t)) are zero.

[0022] Single-slot drift: Drift is the expected change in the Lyapunov number within a time slot, where the current state in time slot t is L(Θ(t)). Single-slot drift is defined as: Virtual Queue and the Lyapunov drift L(Θ(t)) only help to ensure that the required time average constraint is met. To ensure the long-term stability of the queue, a drift penalty is added: V is a non-negative control parameter that is used to balance the average system benefit and the average queuing delay. Under any control algorithm, the drift-plus-penalty expression has the following P2 upper bound for all possible values ​​of t, θ(t), and all parameters V ≥ 0: Because (Θ(t)) does not change with time Therefore, the multi-stage MINLP problem of system benefits can be transformed into a deterministic problem. In each time period, solve a tuple containing variables (X t ) and remove the constant term (t) in each time period.

[0023] The state, action, and reward definitions and module interaction process of step 2 are as follows:

[0024] The state consists of tasks and model information. The state at time slot $t$ is defined as follows:

[0025]

[0026] Actions determine task offloading and resource allocation, specifically as follows:

[0027] The reward of the system is represented by r(t), and its formula is:

[0028] Furthermore, during the optimization process of the model-free actor module in step 2, it is necessary to distinguish between discrete variables and continuous variables.

[0029] The set of discrete variables includes: The set of continuous variables includes:

[0030] Furthermore, the model-based critic module in step 2 utilizes the system model information and the potential task offloading decision from the model-free actor module Each decision is accurately evaluated through the model optimization process. The strategy update module updates the actor strategy through the reinforcement learning algorithm based on the critic evaluation results to optimize the long-term system benefits. Specifically:

[0031] The critic module solves the remaining continuous resource variables y(t) to make the quantized unloading decisions of the L groups. Conduct scoring assessment. For a given and the system state s(t), the optimal value of the objective function P2. Problem P2 can be equivalently transformed into problem P3, and the solution goal of P3 is: In decision-making Under the optimal resource allocation strategy, the optimal target value is obtained by optimizing the resource allocation variable y(t). The problem of P2 in each time slot can be solved by decomposing it into the following three sub-problems: 1) bandwidth allocation optimization of IGW, ECS and CS; 2) computing resource allocation optimization of IGW, ECS and CS; 3) optimization of the offloaded data volume of IGW.

[0032] The beneficial effects of the present invention are:

[0033] 1. Through efficient resource management and task scheduling, the accuracy and response speed of edge reasoning are improved, ensuring high QoS on resource-constrained edge nodes.

[0034] 2. By combining the DRL of the model with model-based optimization, the problems of high computational complexity and slow convergence in online resource optimization are solved.

[0035] 3. Lyapunov optimization theory is used to maintain long-term system efficiency and queue stability in dynamic task queuing systems, preventing resource overload or insufficient resource allocation from causing system performance degradation.

[0036] 4 In the end-edge-cloud collaborative architecture, dynamically select the appropriate-scale DNN model (such as LDNN, MDNN, HDNN) based on the complexity of the task and the real-time network status to optimize the balance between inference results and service time.

[0037] 5. Achieve overall coordination among data quality, computing resources, and communication resources to solve the problems of low resource utilization and single decision-making in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is the EdgeManager architecture system model of the present invention.

[0039] Figure 2 It is the HyDRL-MO algorithm framework of the present invention.

[0040] Figure 3 For experimental testbed environment.

[0041] Figure 4 This is a comparison chart of experimental queue lengths.

[0042] Figure 5 This is a comparison chart of the ratio of experimental communication and computing resources.

[0043] Figure 6 This is a comparison chart of the impact of experimental bandwidth.

[0044] Figure 7 This is a comparison chart of experimental queue lengths.

[0045] Figure 8 This is a comparison chart of the impact of experimental arrival rate on the average queue. DETAILED DESCRIPTION

[0046] The technical solutions of the present invention will be described clearly and completely below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0047] This patent proposes an EdgeManager framework that aims to integrate edge devices, edge cloud servers, and centralized cloud servers to achieve collaborative online resource management and adaptive hierarchical DNN reasoning to adapt to heterogeneous edge environments. However, to implement the EdgeManager service solution, the following challenges need to be addressed:

[0048] 1. How can we efficiently address the high computational complexity and slow convergence of online resource optimization in multi-task, multi-server DNN inference environments? Resource management optimization in multi-task, multi-server edge networks requires solving the Mixed Integer Nonlinear Programming (MINLP) problem. The demands of DNN models and inference tasks require significant computational power and memory, making dynamic resource allocation crucial. Furthermore, communication resources are crucial for the real-time transmission and processing of data-intensive DNN models, requiring efficient management of network bandwidth and spectrum to maximize system performance. Consequently, addressing these resource optimization challenges requires significant computational resources, resulting in high computational time complexity. To address this issue, model-free deep reinforcement learning (DRL) offers a promising solution [1-4], enabling systems to directly interact with the environment and adapt to complex, high-dimensional state spaces without the need for predefined models. However, model-free DRL often faces challenges such as training instability and hyperparameter sensitivity. Combining model-free DRL with traditional methods can improve algorithm convergence and provide more robust solutions. However, current research on combining model-free DRL with model-based methods mainly focuses on single decisions and often ignores complex scenarios that require multi-factor considerations.

[0049] 2. How to address the uncertainty of system parameters to ensure that online decision-making in dynamic queuing systems achieves long-term system efficiency and stability? Existing DRL-based methods manage resources by introducing penalty mechanisms (such as delayed and erroneous DNN inference) [5-7]. However, these methods often ignore long-term performance, which can lead to over- or under-allocation of resources and ultimately reduce system efficiency. Lyapunov optimization theory is used to transform complex multi-state dynamic problems into deterministic problems within each time slice, ensuring the long-term stability of the system.

[0050] Therefore, the present invention proposes an EdgeManager framework and AIoT system resource management method based on Lyapunov optimization, which combines model-free DRL with model-based methods and the Lyapunov framework, providing an effective strategy for managing multi-dimensional resources in DNN reasoning.

[0051] The EdgeManager framework and AIoT system resource management method of the present invention are specifically designed for online adaptive resource management and hierarchical DNN inference in collaborative edge environments to address the computational and communication resource management challenges of heterogeneous edge devices in multi-task, multi-user scenarios. The EdgeManager framework proposed in this invention, through online adaptive resource management and hierarchical DNN inference, offers significant advantages and benefits in edge intelligent collaborative environments. Specifically, it introduces an end-edge-cloud collaborative architecture and dynamically adjusts task offloading and resource allocation strategies. This effectively reduces task response time and transmission latency, significantly reducing system service time and meeting real-time inference requirements. Lyapunov optimization theory is employed to transform the dynamic task queuing system into manageable deterministic subproblems, ensuring the long-term stability of the queue. This avoids resource allocation overload or under-allocation, resulting in more stable system operation and significantly improved long-term performance. By combining model-free DRL with model-based multi-decision optimization, the high computational complexity and slow convergence issues in resource management are addressed. This accelerates algorithm convergence, reduces computational resource requirements, and improves the system's real-time performance and adaptability. This technology achieves overall coordination of data quality, computing resources, and communication resources, dynamically adjusting bandwidth, computing power, and data transmission strategies. Resource utilization is significantly improved, and the system can maintain efficient operation even under resource constraints. The specific contents of this technical solution are as follows:

[0052] 1. EdgeManager Architecture Design

[0053] The EdgeManager framework consists of three main layers:

[0054] Industrial Gateway Layer (IGW) of terminal devices: Deploys lightweight DNN (LDNN) models for collecting real-time data and performing local reasoning.

[0055] Edge Cloud Server (ECS): stores medium-sized DNN (MDNN) models and provides collaborative reasoning support.

[0056] Central Cloud Server (CS): stores high-dimensional DNN (HDNN) models and is responsible for offline training and model updates for complex tasks.

[0057] This architecture achieves dynamic allocation and efficient reasoning of different tasks by collaboratively scheduling the computing and communication resources of these three layers.

[0058] 2. Dynamic resource management method

[0059] a. Dynamic task allocation: For inference tasks of varying complexity, EdgeManager can dynamically allocate tasks to end devices, edge cloud servers, or centralized cloud servers based on real-time task requirements and network status. Task allocation decisions are controlled by the following variables:

[0060] Decision variables for assigning tasks to end devices.

[0061] Decision variables for allocating tasks to edge cloud servers.

[0062] Decision variables for task allocation to centralized cloud servers.

[0063] Task allocation is subject to the following constraints:

[0064] b. Communication and computing resource allocation

[0065] To optimize transmission and computing latency, EdgeManager provides a dynamic communication and computing resource allocation mechanism:

[0066] Communication resource allocation: Dynamically adjust network bandwidth and transmission power to optimize data transmission speed.

[0067] Computing resource allocation: Dynamically allocate CPU and GPU computing resources based on task load.

[0068] c. Hybrid Optimization Algorithm (HyDRL-MO)

[0069] A hybrid algorithm combining model-free DRL and model-based multi-decision optimization is designed. The algorithm includes:

[0070] -Model-free actor module: Generates task assignment and model selection decisions using a DNN and a quantizer.

[0071] -Model-based critic module: evaluates multiple decision scenarios to ensure decision accuracy and system convergence.

[0072] -Strategy update module: Dynamically adjusts decision-making strategies based on feedback.

[0073] d.Lyapunov optimization technique

[0074] Lyapunov optimization theory is used to decompose complex multi-stage MINLP problems into manageable deterministic subproblems, ensuring the long-term stability of the task queue.

[0075] The specific implementation process is as follows:

[0076] like Figure 1-2As shown in Figure 2, the system architecture is divided into three main layers: Cloud Server (CS), Edge Cloud Server (ECS), and End Layer. For each time slot t, we define the kth task of AIoT device j as U jk (t), where D jk (t) represents the data block size of the task, which directly affects the quality of model training. In time slot t, the arrival rate of the task is recorded as v jk (t).

[0077] a) DNN model pre-training: The pre-trained deep neural network (DNN) model is denoted as Defined as: Where n∈{0,1,2} represents the level of the model, corresponding to lightweight DNN (LDNN), medium-sized DNN (MDNN) and high-dimensional DNN (HDNN). In this model, Indicates that the model level n is in task U jk The inference accuracy on . Indicates the size of the model at this level, which is also the size of the task U jk (t) Required model size. Decision variables It is used to indicate the selected model level n, with the following constraints:

[0078] b) DNN accuracy reasoning: For task U jk (t), the accuracy of DNN reasoning is affected by the data size (such as the resolution of the image data), the data quality, and the type of DNN model used. By changing the resolution of each image in the dataset, we analyzed the effect of the task data size D jk and inference accuracy and As shown in Table 1, we divide the task U jk The data quality of (t) is expressed as θ jk , where θ jk ∈{0,1}. When θ jk = 0, indicating that the data is noise-free. At this time, the reasoning accuracy of the lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is When θ jk =1, indicating that the data is noisy, and the reasoning accuracy of the lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is The reasoning accuracy of the task can be expressed as:

[0079] Table 1 Relationship between pre-trained DNN, data scale and data quality

[0080]

[0081] b1) DNN model deployment: For any given task U jk (t), the model at the corresponding location (industrial gateway IGW, edge cloud server ECS, cloud server CS) The availability of a jk express. and is a binary variable, where by For example, if Indicates that there is no task U in the IGW cache pool jk (t) Required model Need to download from ECS to IGW; conversely, if This means that the IGW already has the model and does not need to be downloaded. Therefore, the transmission delay of downloading the DNN model from ECS to IGW can be expressed as: in, Indicates the transmission rate of downloading the DNN model from the edge cloud server (ECS) to the industrial gateway (IGW). The calculation formula is as follows: in, Indicates downloading the model from the Edge Cloud Server (ECS) Channel bandwidth to the industrial gateway IGW, is the transmission power of the device, represents the channel gain, N σ Represents the channel noise power.

[0082] if Indicates that there is no task U in the cache pool of the edge cloud server (ECS) jk (t) Required model Then download the model from the cloud server (CS) cache The formula for calculating the transmission delay to the edge cloud server (ECS) is as follows: in, Indicates the transmission rate of downloading the DNN model from the cloud server (CS) to the edge cloud server (ECS).

[0083] c) Task allocation decision: When the inference task is offloaded to IGW, ECS or CS, the delay mainly includes the uplink transmission delay. Definition Respectively represent the task U jk Decisions allocated to the connected IGW, ECS, and CS, where It is important that each task U jk Can only be assigned to one node, so the allocation decision variable needs to satisfy Represents the reasoning task U jk is assigned to IGW, otherwise Similarly, Represents the reasoning task U jk (t) is assigned to ECS, otherwise and Indicates that the reasoning task $U_{jk}(t)$ is assigned to CS, otherwise When the reasoning task U jk When offloaded to the IGW for processing, the latency of upstream transmission is as follows: Similarly as well as in and Respectively represent the task U jk Transmission delay of offloading to IGW and ECS. Assuming r l up (t) = r l down (t), and where r up (t) is the uplink transmission rate. l up (t) is calculated as follows: in, and Respectively and bandwidth allocation ratio.

[0084] When IGW is overloaded, inference tasks will be redirected to ECS for support. The latency incurred by this redirection process is as follows: in, Indicates the size of the overload data transmitted from the IGW to the ECS. fwd (t) represents the forwarding delay from IGW to ECS.

[0085] d) Task Queue Model: To be more realistic, we assume that most tasks are offloaded to IGW or ECS, and only a few tasks are offloaded to CS. Therefore, we focus on the task queues on IGW and ECS. At the beginning of each time slot t, many tasks arrive at and The bits are buffered in the ECS and IGW, waiting to be processed in the subsequent time slot. Represents time slot U on IGW at time t jk The queue length is calculated as follows: t∈T, where represents the amount of tasks (bits) processed by the IGW in time slot t, represents the amount of tasks (in bits) offloaded from the IGW to the ECS in time slot t. Re-expressed in the form without using {.,0}, The queue lengths are as follows: t∈T. Similarly, let Indicates the U processed by ECS jk Task volume (bits). ECS is specifically designed for U jk The task queue is recorded as It can be expressed as: in in Indicates the amount of data that is overloaded during transmission. t∈T.

[0086] e) Calculation delay model: Assume and (Unit: GHz) represents the IGW, ECS and CS in executing the task U jk The CPU frequency at that time. and Indicates the IGW, ECS and CS for task U jk The computing speed of other resources (such as GPU or dedicated AI accelerator) in FLOPS. l , G e , and G c Represent the total computing capabilities of IGW, ECS, and CS respectively. and Represents the computing resources allocated to the inference task. jk When processed by the LDNN on terminal device j, the task processing delay is entirely composed of the LDNN's computational delay. The computational delay is calculated as follows: in, Indicates the fraction of IGW computing resources allocated to the inference task. jk When offloaded to ECS for processing, the task processing delay is as follows: in, Indicates the fraction of ECS computing resources allocated to inference tasks. Indicates the task U executed using HDNN in ECS jk The inference delay is calculated as follows: in, represents the fraction of CS computing resources allocated to inference tasks. Therefore, the total completion latency of DNN inference on IGW, ECS, and CS is as follows: The total service time U to complete the inference task i,k It can be calculated using the following formula:

[0087] f) The proposal of the multi-stage MINLP problem of maximizing inference accuracy: The optimization objective is defined as taking both accuracy and latency benefits into account. To maximize the benefits of the system. The model uses an objective function to quantify the system performance at any given time, which is defined as follows: Where (1-μ jk ) and (μ jk ) are the weights of inference accuracy and service time in the objective function, where (0≤μ jk ≤1),X={X t} t∈T represents the combination of all optimized variables over time. represents the optimization variable combination vector for time period t. Then this problem can be expressed as a multi-stage MINLP problem:

[0088] The constraints are as follows:

[0089]

[0090] Constraint C1 ensures that the inference process can only be completed on a single computing node. Constraint C2 ensures that only one pre-trained DNN model can be selected to complete the inference task in each time period. Constraint C3 ensures that the bandwidth resources from the IGW to the ECS and then to the CS do not exceed the total available bandwidth. Constraint C4 ensures that the computing resources used by all users do not exceed the total available capacity of the IGW, ECS, and CS. Constraints C5-C10 ensure that the CPU frequency allocated to a user does not exceed the maximum CPU frequency of the IGW, ECS, and CS. Constraints C11-C12 ensure that the number of tasks processed in a time period does not exceed the number of tasks backlogged in the task queue during that time period. Constraints C13-C14 ensure that the average queue length does not exceed the queue threshold.

[0091] g) Queue stability problem with Lyapunov drift and Lyapunov optimization: To solve the C1-C14 problems, the Lyapunov optimization framework is used to transform all inequality and equality constraints into queue stability problems. Specifically, a virtual queue with the update equation is defined. Consider a system with N queues. We use Θ(t) instead of (Q(t)) to represent the queue vector and use represents the queue vector, and is the vector of actual network queues, and is a properly chosen virtual queue. The quadratic Lyapunov function L(Θ(t)) is defined as follows: $w_1+w_2+w_3+w_4=1$. The function L(Θ(t)) is always non-negative and is equal to zero if and only if all components of L(Θ(t)) are zero. Define the single-slot condition LyapunovdriftΔ(Θ(t)) as follows: Where drift is the expected change of the Lyapunov function within a time slot, given that the current state in time slot t is L(Θ(t)). Virtual Queue and the Lyapunov drift L(Θ(t)) only helps to ensure that the required time average constraint is met. To ensure the long-term stability of the queue, the drift plus penalty is defined as follows: V is a non-negative control parameter that is used to balance the average system benefit and the average queuing delay. Under any control algorithm, the drift-plus-penalty expression has the following P2 upper bound for all possible values ​​of t, θ(t), and all parameters V ≥ 0.

[0092] Since (Θ(t)) does not change with time We can transform the multi-stage MINLP problem of system benefits into a deterministic problem. At each time period, solve a tuple containing variables (X t ) and remove the constant term (t) in each time period. P2 is expressed as follows:

[0093] h) Integrated online resource management based on deep reinforcement learning: Introducing the EdgeManager framework to solve the per-slot MINLP problem P2. Figure 2 As shown in Figure 2, the proposed method HyDRL-MO consists of three main interacting modules: a model-free actor module, a model-based critic module, and a policy update module. The main components of the method are as follows:

[0094] 1) State: The initial state consists of tasks and model information. The state at time slot $t$ is defined as follows:

[0095]

[0096] 2) Action: Determines task offloading and resource allocation. The specific contents of these actions are as follows:

[0097]

[0098] 3) Reward: The system reward is represented by r(t), and its formula is Model-free actor module: During the optimization process, it is necessary to distinguish between discrete variables and continuous variables. The discrete variable set includes: The set of continuous variables includes:

[0099] Model-based critic module: Different from traditional actor-critic methods, the proposed method replaces the DNN in the critic module with a model-based approach. The model-based critic module utilizes system model information and potential task offloading decisions from the actor module. Each decision is accurately evaluated through the model optimization process. The Critic module evaluates the L group of quantified unloading decisions by solving the remaining continuous resource variables y(t). Conduct scoring assessment. For a given and the system state s(t), the optimal value of the objective function P2. Problem P2 can be equivalently transformed into another problem P3, the solution goal of which is: in, In decision-making Under the optimal resource allocation strategy, the optimal target value is obtained by optimizing the resource allocation variable y(t). The problem P2 at each time slot can be decomposed into the following three sub-problems: 1) bandwidth allocation optimization of IGW, ECS and CS; 2) computing resource allocation optimization of IGW, ECS and CS; 3) optimization of the offloaded data volume of IGW.

[0100] Experimental verification

[0101] Assume the average channel gain The following path loss model is followed: Where Ag = 3 represents the antenna gain, f cr =915MHz is the carrier frequency, the path loss index dl=3, and (Unit: meter) indicates the distance between the mission and the IGW. Channel gain Obey the independent and identically distributed Rician distribution and satisfy Other channel gains and The calculation method and The detailed parameter information is shown in Table 2.

[0102] Table 2 Parameter default values

[0103]

[0104]

[0105] This section presents the simulation results and test platform used to evaluate the performance of the proposed online framework EdgeManager. YoloV7-Tiny, YoloV10, and Faster-RCNN are selected as LDNN (lightweight neural network), MDNN (medium complexity neural network), and HDNN (high complexity neural network), respectively, and deployed on Jetson NX, Legion R9000K, and cloud server. The detailed device information is shown in Table 3 and Figure 3 shown.

[0106] Table 3 Hardware specifications

[0107]

[0108] Experimental case 1: System benefit under queue threshold constraints.

[0109] like Figure 4 (a) and Figure 4 As shown in (b), thanks to the optimization of the critic module, all methods can control the queue lengths of IGW and ECS within the preset threshold range or just reach the threshold. Figure 5 (a) and Figure 5 In (b), the data arrival rate vjk is set to vary between 2.5Mbps and 3Mbps. Figure 5 As shown in (a), as v jk With the increase of , the data queue length of the three schemes increases. It is worth noting that in all v jk In this case, both CG-GRA and HyDRL-MO can maintain a stable data queue, while when vjk ≥ 2.7, the queue length of the QL-GRA scheme diverges. Figure 5 As shown in (b), under all set vjk conditions, the system utility achieved by HyDRL-MO is 23.02% and 47.51% higher than that of CG-GRA and QL-GRA, respectively, while maintaining the queue stability of EdgeManager.

[0110] like Figure 6 As shown in (a), as bandwidth increases, the queue lengths of all algorithms decrease. It is noteworthy that HyDRL-MO consistently maintains a shorter queue length than the other two algorithms, demonstrating superior data queue management stability and efficiency within the EdgeManager framework. Figure 6(b) shows that system utility improves with increasing bandwidth, with HyDRL-MO showing the most significant increase. This improvement is primarily due to more bandwidth being allocated to end devices for task offloading. Specifically, the proposed algorithm achieves 20.06% and 44.62% higher average system benefits than CG-GRA and QL-GRA, respectively.

[0111] Experimental case 2: Queue stability stress test.

[0112] Figure 7 The queue stability key performance indicators (KPIs) of each solution under high traffic conditions (i.e., vjk = 15) are shown. Each data point represents the average of a sliding window over 3000 time slots. Figure 7 (a) to Figure 7 As shown in (b), the task data backlog queues for both the QL-GRA and CG-GRA schemes exhibit a near-linear growth trend, indicating queue instability. This suggests that, in the absence of effective channel and computing resource allocation, the system's task processing capabilities are limited and it struggles to cope with the current task arrival rate. In contrast, although HyDRL-MO experienced rapid growth in user queue length in the early stages, this growth trend gradually slowed, ultimately achieving queue stability. This shift further validates the EdgeManager framework's effectiveness and convergence performance under high computational load challenges.

[0113] like Figure 8 As shown in the figure, the number of potential actions, L, is set to 10, 20, and 40, respectively, to evaluate the average data queue length under different arrival rates. The quantization parameter, L, plays a key role in the convergence speed of the method. It can be observed that the queue length for all policies increases with the arrival rate, indicating that higher arrival rates increase data processing requirements. Specifically, when L = 10, the queue length is relatively large, while the performance is relatively similar when L = 20 and L = 40. This is because a larger value of L expands the exploration space of binary decisions, accelerates DNN convergence, and improves the quality of the offloading policy. However, increasing L also increases the computational complexity of HyDRL-MO when handling online offloading decisions. To balance convergence speed and computational complexity, HyDRL-MO sets a larger value of L in the initial stage and adjusts it appropriately as the DNN policy gradually approaches the optimal solution. This strategy enables HyDRL-MO to maintain low computational complexity even with a large action space.

[0114] Finally, it should be noted that the above implementation methods are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An EdgeManager framework and AIoT system resource management method based on Lyapunov optimization, characterized by: The method comprises: step 1, building an EdgeManager system model; step 2, a dynamic resource management method, wherein the dynamic resource management method includes dynamic task allocation, communication and computing resource allocation, a HyDRL-MO hybrid optimization algorithm, and Lyapunov optimization, wherein the Lyapunov optimization includes queue stability processing based on Lyapunov optimization and deep learning online resource management based on the EdgeManager framework; Step 1 includes system architecture, pre-training DNN model, task definition, and inference accuracy modeling. The dynamic task allocation in step 2 includes a task queue model, which is divided into the IGW queue length and the ECS queue length. The dynamic task allocation decision is controlled by the following variables: Decision variables for task allocation to Industrial Gateway (IGW), Decision variables for task allocation to edge cloud servers (ECS), Decision variables for task allocation to centralized cloud servers (CS), Task allocation follows the constraints: The communication and computing resource allocation in step 2 includes a computing delay model, which is divided into IGW processing delay and ECS and CS processing delay; The step 2 HyDRL-MO hybrid optimization algorithm includes a multi-stage MINLP problem and a target optimization model; The queue stability processing based on Lyapunov optimization in step 2 includes virtual queue definition, Lyapunov function and drift, and drift plus penalty, wherein the Lyapunov function and drift are quadratic Lyapunov function and single-slot drift; The step 2 of deep learning online resource management based on the EdgeManager framework specifically includes the interaction process of state, action and reward modules. The module interaction process includes a model-free actor module, a model-based critic module and a strategy update module. The model-free actor module uses a DNN and a quantizer to generate task assignment and model selection decisions. The model-based critic module evaluates multiple decision scenarios, The strategy updating module dynamically adjusts the decision strategy based on the feedback.

2. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The described system architecture first divides the system into the terminal equipment industrial gateway layer (IGW), the edge cloud server layer (ECS) and the centralized cloud server layer (CS); The pre-trained DNN model first divides the DNN model into lightweight (LDNN), medium-sized (MDNN), and high-dimensional (HDNN). The pre-trained deep neural network DNN model is defined as Expressed as: Where n∈{0,1,2} represents the level of the model; Indicates that the model level n is in task U jk The inference accuracy on Indicates the size of the hierarchical model, the decision variable Indicates model selection, used to indicate the selected model level n, and its constraints are: The task is defined as: in each time slot t, the kth task of AIoT device j is defined as U jk (t), the data block size task is defined as D jk (t), the arrival rate of the task is defined as v jk (t); Task U jk (t) DNN model on the corresponding locations of industrial gateway (IGW), edge cloud server (ECS), and cloud server (CS) Availability of a jk Representation, definition and is a binary variable, where definition Respectively represent the task U jk Decisions allocated to the connected IGW, ECS, and CS, where Each task U jk Can only be assigned to one node.

3. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 2, characterized in that: The inference accuracy Affected by data size, noise level θ jk And the model type affects, when there is no noise θ jk =0, the reasoning accuracy of lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is θ jk =1 means the data is noisy, and the reasoning accuracy of lightweight DNN (LDNN) is The inference accuracy of medium-scale DNN (MDNN) is The reasoning accuracy of high-dimensional DNN (HDNN) is The total accuracy is expressed as:

4. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The task queue model IGW queue length calculation setting Represents time slot U on IGW jk The queue length is: t∈T, where represents the amount of tasks processed by the IGW in time slot t, represents the amount of tasks unloaded from IGW to ECS in time slot t. The ECS queue length is calculated as Indicates the U processed by ECS jk Amount of tasks, set Indicates the U processed by ECS jk Task volume, set For U jk The task queue: in Indicates the amount of data that was overloaded during transmission.

5. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: When tasks are offloaded to the IGW, ECS, or CS, the IGW processing delay and ECS / CS processing delay models are as follows: IGW: ECS: CS: The total service time U to complete the inference task i,k The formula is as follows:

6. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The multi-stage MINLP problem and target optimization model, the optimization model uses an objective function to quantify the system performance at any given time, defined as: Where (1-μ jk ) and (μ jk ) are the weights of inference accuracy and service time in the objective function, where (0≤μ jk ≤1), let X={X t } t∈T represents the combination of all optimization variables changing over time, X t represents the optimization variable combination vector for time period t. The multi-stage MINLP problem is expressed as: At the same time, set the constraints for P1.

7. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The virtual queue definition is: The quadratic Lyapunov function is defined as: Single slot drift definition: Drift Penalty: V is a non-negative control parameter that is used to balance the average system benefit and the average queuing delay.

8. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The interaction process of the state, action and reward modules is as follows: 1) The initial state consists of task and model information. The state at time slot $t$ is defined as follows: 2) Determine task offloading and resource allocation. The specific operations are as follows: 3) The system reward is represented by r(t), and its formula is:

9. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The model-free actor module needs to distinguish between discrete variables and continuous variables during the optimization process. The model-based critic module scores and evaluates the quantified offloading decisions by solving the remaining continuous variable resources.

10. The EdgeManager framework and AIoT system resource management method based on Lyapunov optimization according to claim 1, characterized in that: The strategy update module updates the action strategy based on the critic evaluation results through a reinforcement learning algorithm to optimize the long-term system benefits. The problem of each time slot is decomposed into 1) bandwidth allocation optimization of IGW, ECS and CS; 2) Optimization of computing resource allocation among IGW, ECS, and CS; 3). Optimize the amount of data offloaded by IGW.

Images