Multi-task edge computing method for Internet of Things

Abstract

The invention provides a multi-task edge computing method for the Internet of Things, and belongs to the technical field of edge computing, and the method comprises the steps: collecting task state information of a to-be-processed task generated by a task collector in a monitoring region of the Internet of Things, and a transmission queue and a computing queue reported by an edge server; inputting the task state information into an intelligent agent, and outputting a scheduling decision of the to-be-processed task; determining tasks needing to be unloaded and an unloaded target server according to the scheduling decision, creating a virtual task in a calculation queue of the target server, determining an expected arrival sequence of the tasks needing to be unloaded in the target server according to the generation time of the tasks needing to be unloaded and the waiting time of the virtual task, and calculating to obtain total delay and total energy consumption; and based on the queue perception TD3, updating the intelligent agent to obtain an optimal scheduling decision. The problem that in the prior art, due to the fact that the influence of transmission delay of task unloading between servers on the sequence of the calculation queues is ignored, task scheduling is inaccurate is solved.

Description

Technical Field

[0001] This invention belongs to the field of edge computing technology, and in particular relates to a multi-task edge computing method for the Internet of Things. Background Technology

[0002] With the widespread deployment of IoT and AI technologies, the long-term service status monitoring of various infrastructures is gradually becoming automated and real-time. Critical infrastructures distributed across different regions are susceptible to various factors such as corrosion, fatigue, and impact during long-term operation, thus requiring high-frequency, multi-sensor monitoring to continuously assess their health status. Edge computing technology, which deploys computing power to edge nodes close to the data source, has become a key technological path in the construction of intelligent IoT monitoring systems.

[0003] However, edge computing environments exhibit high heterogeneity and dynamism. On the one hand, monitoring terminals in different locations differ significantly in terms of task generation frequency, data size, and computational intensity; on the other hand, the computing power, number of connected terminals, and bandwidth of each edge server also vary. This heterogeneity and dynamism easily lead to severe load imbalances in task scheduling and resource allocation in multi-server edge computing architectures, which not only prolongs task processing latency but also increases overall energy consumption, negatively impacting the continuity and stability of monitoring tasks.

[0004] Most current studies considering queuing latency assume that the computation order of tasks in the computation queue is consistent with the task generation order, ignoring the impact of different task offloading and transmission delays from the original computing server to the target server on the computation queue order. This leads to inaccurate queuing latency estimation, affecting system performance evaluation and optimization. At the same time, there is a problem of state dimension fluctuation caused by dynamic changes in the number of tasks, which cannot adapt to the actual needs of dynamic task flow in IoT monitoring. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a multi-task edge computing method for the Internet of Things, which solves the technical problem in existing technologies where the impact of task offloading transmission delays between servers on the computing queue order is ignored, leading to inaccurate queuing delay estimation and degraded task scheduling performance.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a multi-task edge computing method for the Internet of Things, comprising the following steps: Collect task status information of pending tasks generated by task collectors deployed in the IoT monitoring area, as well as task transmission queues and task calculation queues reported by all edge servers; The task status information is input to the intelligent agent deployed in the control center, and the scheduling decision for the task to be processed is output; the scheduling decision includes the target server identifier and the amount of computing resources allocated. Based on the target server identifier in the scheduling decision, determine the task to be unloaded and the target server to be unloaded, create a corresponding virtual task for the task to be unloaded in the computing queue of the target server, and calculate the waiting time of the virtual task. Based on the generation time of the required unloaded tasks and the waiting time of the virtual tasks, determine the expected arrival order of the unloaded tasks on the target server, and calculate the total latency and total energy consumption of the unloaded tasks based on the expected arrival order. Based on the total latency and energy consumption of the unloaded tasks as needed, the parameters of the agent in the control center are updated according to queue awareness TD3 to obtain the optimal scheduling decision.

[0007] Furthermore: the expression of the intelligent agent deployed in the control center is as follows:

[0008]

[0009]

[0010]

[0011] in, For time slots state, For time slots The status of tasks awaiting decision-making. For time slots Server status, For time slots The action, For the target server, For the decision of unloading, For the allocated computing resources, For time slots The reward The parameter is used to normalize the time cost of task completion. The parameter is used to normalize energy costs. The parameter is used to normalize the returns. To the benefit of successfully completing the task within the maximum tolerable latency, For time slots Total latency, For time slots Total energy consumption For the preset delay threshold, For the collector's index, An index of pending tasks generated by the collector.

[0012] The further beneficial effects mentioned above are as follows: the intelligent agent perceives the status of tasks and servers in real time, outputs task scheduling decisions, and combines rewards including latency, energy consumption and timeout penalties to guide the strategy to achieve joint optimization of latency and energy consumption in a dynamic environment, thereby improving system energy efficiency and scheduling reliability while meeting task deadline constraints.

[0013] Furthermore: the waiting time for obtaining the virtual task specifically includes: The task to be unloaded is determined based on the target server identifier in the scheduling decision, and the edge server where the task to be unloaded is located is recorded as the default server. The target server to be unloaded is determined based on the target server identifier. Obtain the uplink and downlink transfer rates of the task to be unloaded from the default server to the target server; Calculate the transmission time required for the unloaded task to be transferred from the default server to the target server, based on the amount of data to be unloaded, the transmission queue of the target server, the uplink transmission rate, and the downlink transmission rate. Use the transmission time as the waiting time for the virtual task.

[0014] The further beneficial effects mentioned above are as follows: This invention creates virtual tasks and quantifies the transmission time of tasks into the waiting time of virtual tasks. It can predict in advance the actual arrival order of tasks that need to be unloaded in the computing queue, thereby accurately estimating the queuing delay caused by network transmission differences. This avoids the misjudgment of queue status caused by ignoring transmission delay in traditional technologies, and improves the accuracy and reliability of task scheduling decisions.

[0015] Furthermore, the expression for the uplink transmission rate is as follows:

[0016] in, This is the default uplink transmission rate from the server to the base station. Set as the default server. For uplink transmission bandwidth, For uplink transmission power, This is the default channel gain from the server to the base station. Noise power; The expression for the downlink transmission rate is as follows:

[0017] in, This refers to the downlink transmission rate from the base station to the target server. For the target server, For downlink transmission bandwidth, For downlink transmission power, The channel gain from the base station to the target server; The expression for the waiting time of the virtual task is as follows:

[0018]

[0019] in, For time slots The waiting time for virtual tasks. This refers to the time required for the unloading task to be transferred from the default server to the target server. Generate the size of the tasks to be processed for the collector. For the mission, The collector generated for it, For this is the first One task, Server at time t On the transmission queue, for The generation time, for The generation time, for Uplink transmission time, For time slots The waiting time for virtual tasks. For time slots Length, For the collector's index, An index of pending tasks generated by the collector.

[0020] The further beneficial effects mentioned above are as follows: This invention calculates the channel state and transmission queue of the uplink and downlink, and realizes accurate estimation of task offloading transmission time; it quantifies the transmission time of the task into a dynamically updated virtual task waiting time, which can reflect the impact of network congestion and transmission delay on task scheduling, provide the agent with real and continuous system state information, and improve the prediction accuracy of task arrival order and queuing delay.

[0021] Furthermore: obtaining the total latency and total energy consumption of the tasks to be unloaded specifically includes: The computing queue in the target server is divided into an arrival queue and a virtual queue. The arrival queue consists of tasks that have been transferred to the target server from the tasks to be unloaded, and the virtual queue consists of tasks that are being transferred to the target server from the tasks to be unloaded. Calculate the expected arrival time of the task to be uninstalled based on the generation time of the task to be uninstalled and the waiting time of the virtual task. Determine the expected arrival order of the uninstallation tasks to be transmitted to the target server based on the expected arrival time of the uninstallation tasks. Based on the expected arrival order, expected arrival time, and processing time of the tasks to be unloaded, the total delay and total energy consumption of the tasks to be unloaded are calculated.

[0022] The further beneficial effects mentioned above are: dividing the target server's computing queue into an actual arrival queue and a virtual queue, the execution order can be dynamically reconstructed based on the expected arrival time of the task, the queuing waiting time caused by the transmission delay between the tasks and the server can be accurately calculated, the problem of underestimating the queuing delay due to ignoring the transmission difference in the traditional model can be alleviated, the total delay and total energy consumption of the task can be calculated more accurately, and the agent can make better scheduling decisions.

[0023] Furthermore, the expression for the total delay of the task to be unloaded is as follows:

[0024] in, The total delay for the task that needs to be uninstalled. For tasks existing in the subsequent condition set, the tasks in the virtual queue are, in order, compared to the tasks in the virtual queue. The earlier arriving tasks, the more tasks are currently in the arrival queue compared to the number of tasks in the virtual queue. The task of arriving early For virtual queues, For the task The waiting time For the task The waiting time For the task Processing time, To reach the queue, For the collector's index, An index of pending tasks generated by the collector; The expression for the total energy consumption is as follows:

[0025]

[0026]

[0027] in, Total energy consumption, For the task unloading strategy, Energy consumption during task transmission. The computational energy consumption for the task. As an energy conversion factor, For the target server Computational resources allocated to the task Generate the size of the tasks to be processed for the collector. The number of CPU cycles required for the collector to generate a unit of data for the tasks to be processed. For uplink transmission power, This refers to the uplink transmission rate.

[0028] The further beneficial effects mentioned above are as follows: by calculating the total latency and total energy consumption of the task on the target server, a basis is provided for the reward and optimization of the agent; the total latency calculation integrates transmission waiting time, queue queuing time and task processing time, and by distinguishing between virtual queues and actual arrival queues and introducing minimum waiting time judgment, the accurate evaluation of the dynamic queuing process of tasks is realized, enabling the agent to achieve an effective trade-off between latency and energy consumption, and improve energy efficiency and computing resource utilization while meeting latency requirements.

[0029] Furthermore: the queue-aware TD3 algorithm includes a main network and a target network, and both the main network and the target network include an actor network and two critic networks; The input to the actor network in the main network is the current state of the agent, and the output is the deterministic action of the agent. The inputs to the two commentator networks in the main network are the agent's current state and deterministic actions, and the outputs are the first action value estimate and the second action value estimate, respectively. The input to the actor network of the target network is the next state of the agent, and the output is the target action of the agent; The inputs to the two critic networks of the target network are the next state and the target action, and the outputs are the third action value estimate and the fourth action value estimate.

[0030] Furthermore, the expressions for the Q-values ​​of the two critic networks in the main network are as follows:

[0031] in, The Q-values ​​of the two critic networks in the main network. As a reward, As a discount factor, For the valuation of the first action, For the valuation of the second action; The expressions for the Q-values ​​of the two critic networks in the target network are as follows:

[0032] in, Let Q-values ​​be the two critic networks in the target network. For the valuation of the third action, This is for the valuation of the fourth action.

[0033] The further beneficial effects mentioned above are as follows: by introducing a dual critic network into the actor-critic framework, the queue-aware TD3 algorithm alleviates the problem of overestimation in the continuous action space of traditional reinforcement learning, enabling the algorithm to adapt to the dynamically changing queue state dimension and achieve precise optimization of task unloading and resource allocation strategies.

[0034] Furthermore, the critic network is frequently updated using a loss function, the expression of which is as follows:

[0035]

[0036] in, The mean squared error loss of the first critic network in queue-aware TD3. This represents the number of samples in a batch obtained from the experience replay pool. To iterate through each sample data in this batch, For the first The state of the sample For the first The actions of the sample Given the neural network parameters The critics' network estimates the current Q-value of the state-action pair output. The target Q value is calculated for the target network. For instant rewards, As a discount factor, To minimize the bi-objective critic, This represents the parameters of the neural network.

[0037] The actor network is updated occasionally by optimizing the policy gradient, and the expression for optimizing the policy gradient is as follows:

[0038] in, To represent the policy objective function, For the policy network in state The action of outputting below, For state and actions Q value, For the action The partial derivative operator, Indicates the network parameters of the policy The gradient operator.

[0039] The further beneficial effects mentioned above are as follows: Through the policy gradient update mechanism, the actor network can gradually optimize the scheduling strategy based on the value gradient direction provided by the critic network. This invention combines deterministic policy gradient with a dual critic structure, which improves the stability and convergence speed of policy updates, enabling the agent to efficiently learn the optimal unloading and computational resource allocation strategy in a dynamic task queue environment.

[0040] The beneficial effects of this invention are: This invention designs an edge computing system architecture that supports task forwarding and collaborative scheduling between servers. Monitoring tasks collected by sensors can be forwarded to target servers through a default server. Combined with the global state scheduling of the central controller, the resource allocation for task computing is dynamically allocated, thereby improving the overall load balancing capability of the system.

[0041] This invention addresses the task requirements of multi-server collaborative edge computing scenarios by constructing an optimization problem that jointly optimizes latency and energy consumption under latency constraints. Considering the dynamic characteristics of the scenario, the problem is transformed into an MDP for subsequent solution.

[0042] Unlike traditional modeling based on generation order, this invention proposes a computation queue model construction method based on task arrival attributes. It introduces virtual tasks and waiting time to reconstruct the state, thereby achieving accurate modeling of computation queues and queuing delays.

[0043] This invention combines Markov decision modeling, introduces a reward function that considers the feedback of task completion status, and proposes the QATD3 algorithm. Through state redundancy removal and adaptive state filling, it effectively improves decision performance and realizes joint decision-making for task unloading and resource allocation. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a multi-task edge computing method for the Internet of Things. Detailed Implementation

[0045] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0046] Example 1 like Figure 1The diagram shown is a flowchart of a multi-task edge computing method for the Internet of Things (IoT) provided by the present invention. The multi-task edge computing method for the Internet of Things includes the following steps: Collect task status information of tasks to be processed generated by task collectors deployed in the IoT monitoring area, as well as task transmission queues and task computation queues reported by all edge servers; task status information includes task generation device, task data size, task computation amount, and maximum tolerable latency of the task; The task status information is input to the agent deployed in the control center, which then outputs a scheduling decision for the task to be processed. The scheduling decision includes the target server identifier and the amount of computing resources allocated. The amount of computing resources allocated is the computing resources required by the server to process the task. Based on the target server identifier in the scheduling decision, determine the task to be unloaded and the target server to be unloaded, create a corresponding virtual task for the task to be unloaded in the computing queue of the target server, and calculate the waiting time of the virtual task. Based on the generation time of the required unloaded tasks and the waiting time of the virtual tasks, determine the expected arrival order of the unloaded tasks on the target server, and calculate the total latency and total energy consumption of the unloaded tasks based on the expected arrival order. Based on the total latency and energy consumption of the unloaded tasks as needed, the parameters of the agent in the control center are updated according to queue awareness TD3 to obtain the optimal scheduling decision.

[0047] Existing technologies still have the following key shortcomings, especially in dynamic and high-frequency scenarios: Most current studies assume that the execution order of tasks in the computation queue is consistent with their generation order, ignoring the network latency experienced by unloading tasks from the original server to the target server, thus leading to inaccurate computation order modeling; due to the inaccurate computation order modeling, further errors in queuing delay estimation are introduced, ultimately affecting the system optimization performance of the entire task scheduling scheme; most reinforcement learning scheduling methods fail to fully consider the coupling relationship between dynamic task transmission and queue state changes, lacking a refined state reconstruction mechanism and policy feedback support.

[0048] This invention addresses the task scheduling and resource allocation problems in dynamic multi-server edge computing systems, focusing on enhancing the system's ability to perceive and predict task queuing behavior. The core objective is to propose an Arrival Order Based Queue State (AOBQS) model to accurately model the computation queue and task queuing time, considering the impact of task transmission time on the computation queue order. By introducing the concepts of virtual tasks and waiting time to reconstruct the task queue state, the system's Markov property is guaranteed, and a Markov decision model jointly optimizing latency and energy consumption is established based on this. Furthermore, a queue-aware, latency-aware, dual-deterministic policy gradient (Queue-Aware TD3, QATD3) algorithm is proposed to obtain efficient offloading and allocation strategies, achieving adaptive optimization of system performance.

[0049] In one embodiment of the present invention, in response to the high-frequency, high-timeliness, and computationally intensive data processing needs in the Internet of Things and smart monitoring processes, the present invention provides a multi-task edge computing method for the Internet of Things, and designs a multi-server collaborative edge computing architecture to achieve efficient task offloading and computing resource allocation, including: a task collector, an edge server, and a control center. Several task collectors are deployed at key structural locations within the IoT monitoring area to collect diverse and heterogeneous monitoring data such as vibration, temperature, stress, and potential. Several edge servers are deployed at different locations within the IoT monitoring area. Each task collector has a default edge server for temporary task storage and control command forwarding; this default edge server is designated as the default server for tasks generated by the task collector, and the edge server processing the tasks is designated as the target server. The system collects task status information for tasks generated by the task collectors deployed within the IoT monitoring area, as well as task transmission queues and task computation queues reported by all edge servers deployed within the IoT monitoring area. Task status information includes the task generating device, task data size, task computation load, and maximum tolerable latency. When a task is unloaded, it is first transmitted and temporarily stored on the default server, then enters the default server's task transmission queue. After transmission according to the queue, it enters the target server's task computation queue for processing.

[0050] In a specific embodiment of the present invention, in a given time slot There exists a set of task collectors. and edge servers ,in and These represent the total number of task collectors and edge servers, respectively. and These are the indexes for the task collector and the edge server, respectively; the... The status of each edge server is represented as follows: ,in and They represent the first The task transmission queue and task computation queue of each edge server This indicates the maximum CPU frequency, which can determine the target server based on the task to be processed and the server status, and allocate CPU frequency to the task.

[0051] The tasks generated by the task collector follow a Poisson arrival process; the transmission time between the task collector and the default server is negligible; the... The characteristics of a task collector are represented as follows: ,in , , and The first The generation rate, task size, number of CPU cycles required per unit of data, and maximum tolerable latency of each task collector are specified. The generated first Each task is represented as Defined as ,in This refers to the task generation time; the generated tasks are executed according to the computational strategy determined by the control center, and the computational decision is represented as follows: ,in For the target server's index, The amount of computing resources allocated to the task. Set an unload strategy for the task; if ,but This indicates that the task to be processed is computed locally, i.e., computed by the default server.

[0052] In one embodiment of the present invention, after collecting task status information from the task collector and edge server, considering the dynamic arrival time and waiting time of each task to be processed from the default server to the target server, which makes conventional decision-making methods difficult to adapt, the present invention models the task using a Markov decision process to establish an intelligent agent. By defining the state space, action space, and reward, the optimal task scheduling decision and computing resource allocation are performed. Therefore, the task status information is input to the intelligent agent deployed in the control center, and the scheduling decision for each task is output. The scheduling decision includes the target server identifier, the unloading flag, and the amount of computing resources allocated. In a specific embodiment of the present invention, the intelligent agent, based on the current state... Make an action Output the next state and rewards , where state That is, time slot The status of the tasks to be processed and the status of the edge servers are considered. The actions are the unloading flag of the task to be processed, the target server identifier, and the amount of computing resources allocated. The unloading flag indicates whether the task is to be unloaded, and the target server identifier determines the target server to which the task will be transferred. The expression of the agent deployed in the control center is as follows:

[0053]

[0054]

[0055]

[0056] in, For time slots state, For time slots The status of tasks awaiting decision-making. For time slots Server status, For time slots The action, For the target server, For the decision of unloading, For the allocated computing resources, For time slots The reward The parameter is used to normalize the time cost of task completion. The parameter is used to normalize energy costs. The parameter is used to normalize the returns. To the benefit of successfully completing the task within the maximum tolerable latency, For time slots Total latency, For time slots Total energy consumption For the preset delay threshold, For the collector's index, An index for the tasks to be processed generated by the collector; through the reward function, the agent can be continuously optimized to obtain better task scheduling decisions.

[0057] In one embodiment of the present invention, the task to be processed first enters the task transmission queue of the default server. After transmission is completed, the task to be processed enters the computation queue of the target server. This process leads to two different system state changes: one is a real-time transmission queue state change, and the other is a computation queue state change due to transmission delay. The time difference caused by the transmission delay undermines the Markov property of the agent, and unreasonable task scheduling decisions are easily made due to the difference in the time of task transmission to the target server. Therefore, the present invention proposes virtual tasks to accurately estimate the queuing delay of edge servers. When a task needs to be unloaded by an unloading flag, the task is added to the transmission queue of the default server, and a corresponding virtual task is generated and placed in the computation queue of the target server. That is, the present invention determines the task to be unloaded and the target server to be unloaded based on the unloading flag and the target server flag, creates a corresponding virtual task for the task to be unloaded in the computation queue of the target server, and calculates the waiting time of the virtual task. Specifically, this includes: The task to be unloaded is determined based on the target server identifier in the scheduling decision, and the edge server where the task to be unloaded is located is recorded as the default server. The target server to be unloaded is determined based on the target server identifier. Obtain the uplink and downlink transmission rates of the task to be unloaded from the default server to the target server; the expression for the uplink transmission rate of the task to be unloaded from the default server to the base station is as follows:

[0058] in, This is the default uplink transmission rate from the server to the base station. Set as the default server. For uplink transmission bandwidth, For uplink transmission power, This is the default channel gain from the server to the base station. Noise power; The expression for the transfer rate from the base station to the target server that needs to be offloaded is as follows:

[0059] in, This refers to the downlink transmission rate from the base station to the target server. For the target server, For downlink transmission bandwidth, For downlink transmission power, The channel gain from the base station to the target server; Calculate the transmission time required for the unloaded task to be transferred from the default server to the target server, based on the amount of data to be unloaded, the transmission queue of the target server, the uplink transmission rate, and the downlink transmission rate. The transmission time is used as the waiting time for the virtual task; the expression for the waiting time is as follows:

[0060]

[0061] in, For time slots The waiting time for virtual tasks. This refers to the time required for the unloading task to be transferred from the default server to the target server. Generate the size of the tasks to be processed for the collector. For the mission, The collector generated for it, For this is the first One task, Server at time t On the transmission queue, for The generation time, for The generation time, for Uplink transmission time, For time slots The waiting time for virtual tasks. For time slots Length, For the collector's index, This is the index of the pending tasks generated by the collector. This expression represents the total latency of the task, including transmission time, queuing time after the task arrives in the computation queue, and computation time. If the computation queue is empty when the task arrives, the task is computed directly; otherwise, it must continue waiting. Indicates task Compared to tasks Arrive first, if the task Compare If it arrives first but has not yet arrived, then the task... The queuing time also needs to be taken into account. Due to the influence of arrival time, the time required before task calculation is:

[0062] In a specific embodiment of the present invention, the computing queue in the target server can be represented as follows: The virtual tasks in the computation queue are represented as ,in, for, and These represent the processing time required for the task and the time slot, respectively. The remaining transmission time to the target server, before the task is completed. When making the uninstallation decision, Set as Then the task waiting time is updated, as shown in the following expression:

[0063] in, For time slots Length, For time slots Task waiting time, For time slots Task waiting time, when When the time is specified, it indicates that the task to be uninstalled has been transferred from the default server to the target server, and the virtual task is replaced by the task to be uninstalled.

[0064] This invention determines the expected arrival order of tasks to be unloaded on the target server based on the generation time of the unloaded tasks and the waiting time of the virtual tasks, and calculates the total latency and total energy consumption of the tasks to be unloaded based on the expected arrival order; specifically including: The computing queue in the target server is divided into an arrival queue and a virtual queue. The arrival queue consists of tasks that have been transferred to the target server from the tasks to be unloaded, and the virtual queue consists of tasks that are being transferred to the target server from the tasks to be unloaded. Calculate the expected arrival time of the task to be uninstalled based on the generation time of the task to be uninstalled and the waiting time of the virtual task. Determine the expected arrival order of the uninstallation tasks to be transmitted to the target server based on the expected arrival time of the uninstallation tasks. Based on the expected arrival order, expected arrival time, and processing time of the tasks to be unloaded, the total delay and total energy consumption of the tasks to be unloaded are calculated.

[0065] The task is processed when it arrives at the target server and all tasks that arrived before it in the computation queue have been completed. However, considering that the computation queue may initially be empty, processing can only begin after at least one task arrives; therefore, the task needs to be unloaded. The computational delay should include the task's transmission time. Therefore, the computation time can be recalculated, and the expression for the total delay requiring task unloading is as follows:

[0066] in, The total delay for the task that needs to be uninstalled. For tasks existing in the subsequent condition set, the tasks in the virtual queue are, in order, compared to the tasks in the virtual queue. The earlier arriving tasks, the more tasks are currently in the arrival queue compared to the number of tasks in the virtual queue. The task of arriving early For virtual queues, For the task The waiting time For the task The waiting time For the task Processing time, To reach the queue, For the collector's index, An index of pending tasks generated by the collector; The expression for the total energy consumption required to unload tasks is as follows:

[0067]

[0068]

[0069] in, Total energy consumption, For the task unloading strategy, Energy consumption during task transmission. The computational energy consumption for the task. As an energy conversion factor, For the target server Computational resources allocated to the task Generate the size of the tasks to be processed for the collector. The number of CPU cycles required for the collector to generate a unit of data for the tasks to be processed. For uplink transmission power, As tasks are processed, the waiting time for each task is updated to improve the uplink transmission rate. By capturing changes in the task execution order during task transmission, the state and latency of the multi-task edge computing architecture can be accurately estimated, thereby supporting reliable decision-making by the learning agent.

[0070] In one embodiment of this invention, the design of the agent often assumes that the state dimension is static. However, in the IoT monitoring area and edge server of this invention, the task queue lengths of the default server and edge server change over time, leading to instability in the state dimension and reducing the learning performance of the agent. Although existing methods use truncation or fixed zero-padding to mitigate this, truncation may lose important information, and fixed zero-padding introduces redundant data. Therefore, this invention designs an enhanced TD3 variant specifically for dynamic task queue environments, namely Queue-Aware TD3 (QATD3), short for Queue-Aware Delay-Deterministic Policy Gradient (QATD3). Based on the total latency and total energy consumption of each task, the agent in the control center is updated with parameters according to Queue-Aware TD3 to obtain the optimal scheduling decision.

[0071] In a specific embodiment of the present invention, the queue-aware TD3 algorithm includes a main network and a target network. The main network is used for decision-making and evaluation, and the target network is a delayed-updated copy of the main network, used to generate stable training signals. Both the main network and the target network include an actor network and two critic networks. The input to the actor network in the main network is the agent's current state. The output is the deterministic action of the agent. ,in, For policy networks.

[0072] The input to both commentator networks in the main network is the agent's current state. and deterministic actions Output and The first Critic network pair states are respectively and actions Action value estimation and the second Critic network on the state and actions Action value estimation; the dual-commentator design can reduce the overestimation of the target Q-value by minimizing the deviation between the two action value estimates, thus improving the stability and accuracy of the two commentator networks in the main network; the expression for the target Q-value is as follows:

[0073] in, For the target Q value, As a reward, Discount factor; The input to the actor network of the target network is the next state of the agent. The output is the target action of the agent. ,in, The strategy is to smooth out noise and sample from a Gaussian distribution to encourage exploration; Synchronization between the target network and the main network is maintained through a soft update mechanism to ensure the parameters of the target network are synchronized. Updated to:

[0074] in, For the target network parameters, Main network parameters, For soft update learning rate, .

[0075] In a specific embodiment of the present invention, the commentator network is frequently updated using a loss function, the expression of which is as follows:

[0076] in, The loss function of the critic network, This represents the number of samples in a batch obtained from the experience replay pool. To iterate through each sample data in this batch, For the first The state of the sample For the first The actions of the sample Given the neural network parameters The critics' network estimates the current Q-value of the state-action pair output. The target Q value is calculated for the target network. For instant rewards, As a discount factor, To minimize the bi-objective critic, This represents the parameters of the neural network.

[0077] The actor network updates occasionally by optimizing the policy gradient and employs TD3's unique delayed policy updates mechanism. Specifically, the critic network updates at every training step, while the actor network updates only once every fixed number of steps (usually 2 steps). The expression for optimizing the policy gradient is as follows:

[0078] in, To represent the policy objective function, For the policy network in state The action of outputting below, For state and actions Q value, For the action The partial derivative operator, Indicates the network parameters of the policy The gradient operator.

[0079] Furthermore, policy smoothing noise can be introduced during the target action generation process to enhance the robustness of the algorithm. By adding noise to the actions generated by the target actor network, the QATD3 algorithm ensures better exploration and reduces the risk of getting trapped in local optima, enabling QATD3 to effectively handle dynamic and continuous action spaces.

[0080] The beneficial effects of this invention are as follows: This invention designs an edge computing system architecture that supports task forwarding and collaborative scheduling between servers. Monitoring tasks collected by sensors can be forwarded to target servers through a default server. Combined with the global state scheduling of the central controller, the resource allocation for task computing is dynamically allocated, thereby improving the overall load balancing capability of the system.

[0081] This invention addresses the task requirements of multi-server collaborative edge computing scenarios by constructing an optimization problem that jointly optimizes latency and energy consumption under latency constraints. Considering the dynamic characteristics of the scenario, the problem is transformed into an MDP for subsequent solution.

[0082] Unlike traditional modeling based on generation order, this invention proposes a computation queue model construction method based on task arrival attributes. It introduces virtual tasks and waiting time to reconstruct the state, thereby achieving accurate modeling of computation queues and queuing delays.

[0083] This invention combines Markov decision modeling, introduces a reward function that considers the feedback of task completion status, and proposes the QATD3 algorithm. Through state redundancy removal and adaptive state filling, it effectively improves decision performance and realizes joint decision-making for task unloading and resource allocation.

Claims

1. A multi-task edge computing method for the Internet of Things, characterized in that, Includes the following steps: Collect task status information of pending tasks generated by task collectors deployed in the IoT monitoring area, as well as task transmission queues and task calculation queues reported by all edge servers; The task status information is input to the intelligent agent deployed in the control center, and the scheduling decision for the task to be processed is output; the scheduling decision includes the target server identifier and the amount of computing resources allocated. Based on the target server identifier in the scheduling decision, determine the task to be unloaded and the target server to be unloaded, create a corresponding virtual task for the task to be unloaded in the computing queue of the target server, and calculate the waiting time of the virtual task. Based on the generation time of the required unloaded tasks and the waiting time of the virtual tasks, determine the expected arrival order of the unloaded tasks on the target server, and calculate the total latency and total energy consumption of the unloaded tasks based on the expected arrival order. Based on the total latency and energy consumption of the unloaded tasks, the parameters of the agent in the control center are updated according to the queue-aware TD3 to obtain the optimal scheduling decision.

2. The multi-task edge computing method for the Internet of Things according to claim 1, characterized in that, The expression of the intelligent agent deployed in the control center is as follows: in, For time slots state, For time slots The status of tasks awaiting decision-making. For time slots Server status, For time slots The action, For the target server, For the decision of unloading, For the allocated computing resources, For time slots The reward The parameter is used to normalize the time cost of task completion. The parameter is used to normalize energy costs. The parameter is used to normalize the returns. To the benefit of successfully completing the task within the maximum tolerable latency, For time slots Total latency, For time slots Total energy consumption For the preset delay threshold, For the collector's index, An index of pending tasks generated by the collector.

3. The multi-task edge computing method for the Internet of Things according to claim 1, characterized in that, The waiting time for obtaining the virtual task specifically includes: The task to be unloaded is determined based on the target server identifier in the scheduling decision, and the edge server where the task to be unloaded is located is recorded as the default server. The target server to be unloaded is determined based on the target server identifier. Obtain the uplink and downlink transfer rates of the task to be unloaded from the default server to the target server; Calculate the transmission time required for the unloaded task to be transferred from the default server to the target server, based on the amount of data to be unloaded, the transmission queue of the target server, the uplink transmission rate, and the downlink transmission rate. Use the transmission time as the waiting time for the virtual task.

4. The multi-task edge computing method for the Internet of Things according to claim 3, characterized in that, The expression for the uplink transmission rate is as follows: in, This is the default uplink transmission rate from the server to the base station. Set as the default server. For uplink transmission bandwidth, For uplink transmission power, This is the default channel gain from the server to the base station. Noise power; The expression for the downlink transmission rate is as follows: in, This refers to the downlink transmission rate from the base station to the target server. For the target server, For downlink transmission bandwidth, For downlink transmission power, The channel gain from the base station to the target server; The expression for the waiting time of the virtual task is as follows: in, For time slots The waiting time for virtual tasks. This refers to the time required for the unloading task to be transferred from the default server to the target server. Generate the size of the tasks to be processed for the collector. For the mission, The collector generated for it, For this is the first One task, Server at time t On the transmission queue, for The generation time, for The generation time, for Uplink transmission time, For time slots The waiting time for virtual tasks. For time slots Length, For the collector's index, An index of pending tasks generated by the collector.

5. The multi-task edge computing method for the Internet of Things according to claim 1, characterized in that, The total latency and total energy consumption of the tasks to be unloaded are obtained, specifically including: The computing queue in the target server is divided into an arrival queue and a virtual queue. The arrival queue consists of tasks that have been transferred to the target server from the tasks to be unloaded, and the virtual queue consists of tasks that are being transferred to the target server from the tasks to be unloaded. Calculate the expected arrival time of the task to be uninstalled based on the generation time of the required uninstallation task and the waiting time of the virtual task. Determine the expected arrival order of the tasks to be uninstalled to the target server based on the expected arrival time of the uninstalled tasks. Based on the expected arrival order, expected arrival time, and processing time of the tasks to be unloaded, the total delay and total energy consumption of the tasks to be unloaded are calculated.

6. The multi-task edge computing method for the Internet of Things according to claim 5, characterized in that, The expression for the total delay required to unload the tasks is as follows: in, The total delay for the task that needs to be uninstalled. For tasks existing in the subsequent condition set, the tasks in the virtual queue are, in order, compared to the tasks in the virtual queue. The earlier arriving tasks, the more tasks are currently in the arrival queue compared to the number of tasks in the virtual queue. The task of arriving early For virtual queues, For the task The waiting time For the task The waiting time For the task Processing time, To reach the queue, For the collector's index, An index of pending tasks generated by the collector; The expression for the total energy consumption of the tasks that need to be unloaded is as follows: in, Total energy consumption, For the task unloading strategy, Energy consumption during task transmission. The computational energy consumption for the task. As an energy conversion factor, For the target server Computational resources allocated to the task Generate the size of the tasks to be processed for the collector. The number of CPU cycles required for the collector to generate a unit of data for the tasks to be processed. For uplink transmission power, This refers to the uplink transmission rate.

7. The multi-task edge computing method for the Internet of Things according to claim 1, characterized in that, The queue-aware TD3 algorithm includes a main network and a target network, and each of the main network and the target network includes an actor network and two critic networks. The input to the actor network in the main network is the current state of the agent, and the output is the deterministic action of the agent. The inputs to the two commentator networks in the main network are the agent's current state and deterministic actions, and the outputs are a first action value estimate and a second action value estimate, respectively. The Q-values ​​of the two commentator networks in the main network are calculated based on the first action value estimate and the second action value estimate. The input to the actor network of the target network is the next state of the agent, and the output is the target action of the agent; The inputs to the two commentator networks of the target network are the next state and the target action, and the outputs are the third action value estimate and the fourth action value estimate. The Q-values ​​of the two commentator networks in the target network are calculated using the third action value estimate and the fourth action value estimate.

8. The multi-task edge computing method for the Internet of Things according to claim 7, characterized in that, The expressions for the Q-values ​​of the two critic networks in the main network are as follows: in, The Q-values ​​of the two critic networks in the main network. As a reward, As a discount factor, For the valuation of the first action, For the valuation of the second action; The expressions for the Q-values ​​of the two critic networks in the target network are as follows: in, Let Q-values ​​be the two critic networks in the target network. For the valuation of the third action, This is for the valuation of the fourth action.

9. The multi-task edge computing method for the Internet of Things according to claim 7, characterized in that, The commentator network is frequently updated using a loss function, the expression of which is as follows: in, The mean squared error loss of the first critic network in queue-aware TD3. This represents the number of samples in a batch obtained from the experience replay pool. To iterate through each sample data in this batch, For the first The state of the sample For the first The actions of the sample Given the neural network parameters The critics' network estimates the current Q-value of the state-action pair output. The target Q value is calculated for the target network. For instant rewards, As a discount factor, To minimize the bi-objective critic, This represents the parameters of the neural network. The actor network is updated occasionally by optimizing the policy gradient, and the expression for optimizing the policy gradient is as follows: in, To represent the policy objective function, For the policy network in state The action of outputting below, For state and actions Q value, For the action The partial derivative operator, Indicates the network parameters of the policy The gradient operator.

Images