Task scheduling method and system based on edge computing

Abstract

The invention relates to the field of edge computing scheduling, in particular to a task scheduling method and system based on edge computing, and the method comprises the steps: carrying out the optimization of multi-scale knowledge distillation and weighted model aggregation on a local task model of each edge node through employing a dynamic knowledge transfer mechanism, and updating a global model; an energy consumption perception strategy optimization module is introduced, a Markov decision process model is constructed, the model is optimized by using a reinforcement learning algorithm, and system performance and energy consumption are dynamically balanced; inputting a historical task load into the pre-training model, and predicting a future load through a self-supervised learning single-mode time sequence prediction module; and optimizing the Markov decision process model according to the future load predicted in the previous step, and dynamically adjusting the computing resource allocation of the edge node. The method has the beneficial effects that the task scheduling efficiency, the task precision and the real-time response capability are improved, the importance of efficient nodes is ensured, the generalization capability and the stability of a global model are improved, and the method has good robustness.

Description

Technical Field

[0001] The present invention relates to the field of edge computing scheduling, and in particular to a task scheduling method and system based on edge computing. Background Art

[0002] With the rapid development of technologies such as the Internet of Things (IoT), smart cities, industrial automation, autonomous driving, and intelligent transportation, edge computing, as an emerging computing architecture, has shown great potential in processing large-scale data and achieving low-latency responses.

[0003] However, the edge computing environment also brings new challenges, especially in task scheduling. In a typical edge computing environment, network nodes are usually widely distributed and have different computing power, storage capacity, bandwidth and energy consumption constraints. Therefore, how to reasonably and efficiently schedule these heterogeneous resources to meet dynamically changing task requirements is still a problem that needs to be solved. Specifically, the following issues are common challenges in the current edge computing task scheduling: 1) Edge computing nodes are usually deployed in a variety of environments, and the differences in resources such as computing power, storage capacity, bandwidth and energy consumption make resource scheduling complicated. How to maximize resource utilization through reasonable scheduling strategies and dynamically adjust scheduling schemes according to the load and capacity of different nodes has become an important research topic. 2) The workload of edge computing systems is usually highly dynamic, the generation, processing and end time of tasks are unstable, and the resources and processing power required vary greatly. In this environment, how to accurately predict future task loads and dynamically adjust scheduling strategies based on the prediction results has become a problem that needs to be solved. In particular, how to understand the changing trend of task loads in advance through data analysis and machine learning algorithms to more accurately allocate resources and achieve efficient processing of tasks. 3) In edge computing environments, many edge nodes are power-constrained devices, such as IoT sensors and low-power computing devices. Therefore, how to minimize energy consumption while ensuring computing performance has become an important goal in task scheduling. Especially in mobile devices and battery-powered nodes, energy consumption is one of the core constraints in system design. When optimizing task scheduling, how to balance the performance requirements of computing tasks and the energy consumption constraints of nodes, and reduce energy consumption while ensuring that tasks are completed on time, is a major problem in current edge computing systems. 4) In edge computing, although computing resources are close to data sources, communication between edge nodes may still be limited by network bandwidth. 5) In many edge computing applications, task offloading is a key operation. Due to computing resource constraints, some computing-intensive tasks need to be offloaded to more powerful central computing nodes or the cloud for processing. How to reasonably select which tasks to offload and when to offload, and how to coordinate the allocation and scheduling of tasks between edge nodes and the cloud, depends on dynamic load prediction and intelligent scheduling algorithms. This requires edge computing systems to have intelligent decision-making capabilities and be able to make optimal scheduling decisions in complex environments. 6) With the popularity of smart devices, an important application area of ​​edge computing is personal privacy and data security protection. In many task scheduling scenarios, especially when user data is involved, how to ensure data privacy and security is an important challenge. Since edge nodes are distributed and some data is processed locally, how to ensure data encryption, access control and user privacy protection has become a key task in system design.

[0004] Although some existing task scheduling methods have attempted to optimize task scheduling and resource allocation through reinforcement learning, deep learning, model prediction and other means, these methods often have the following problems: First, due to the complexity of the training model, it is difficult to quickly adapt to the changing environment in practical applications; second, most methods rely on global data and fail to fully consider the heterogeneity of nodes in edge computing; third, in a complex dynamic environment, how to balance multiple objectives (such as latency, energy consumption, resource utilization, etc.) is still a problem that has not been fully solved.

[0005] Therefore, the intelligent task scheduling system based on edge computing still faces many challenges. How to efficiently utilize the computing power of edge nodes, accurately predict the load changes of tasks, and flexibly optimize based on the real-time feedback of task scheduling is the key to improving the performance of edge computing systems. Summary of the invention

[0006] In order to overcome the above-mentioned shortcomings, the present invention aims to provide a task scheduling method and system based on edge computing, which can efficiently schedule tasks in an edge computing environment by introducing a dynamic knowledge transfer mechanism, energy consumption perception strategy optimization and time series prediction technology.

[0007] The present invention achieves the above object through the following scheme: A task scheduling method based on edge computing comprises the following steps:

[0008] S1: A dynamic knowledge transfer mechanism is used to optimize the local task model of each edge node through multi-scale knowledge distillation and weighted model aggregation, and the global model is updated;

[0009] S2: Introduce the energy consumption perception strategy optimization module, build a Markov decision process model, use the reinforcement learning algorithm to optimize the model, and dynamically balance system performance and energy consumption;

[0010] S3: Input the historical task load into the pre-trained model and predict the future load through the self-supervised learning single-modal time series prediction module;

[0011] S4: Optimize the Markov decision process model according to the future load predicted in the previous step and dynamically adjust the computing resource allocation of edge nodes.

[0012] As a preferred method, S1 adopts a lightweight method to extract low-, medium-, and high-level feature representations in hierarchical order to generate multi-scale knowledge representation:

[0013]

[0014] Among them, ω i represents the task model of edge node i, and T is the temperature parameter in the distillation process.

[0015] As a preference, high-quality nodes have a greater weight in the weighted model aggregation process. High-quality nodes are evaluated by calculating the quality scores of edge nodes. The evaluation content includes: the node's computing power, communication delay and energy consumption.

[0016] As a preference, the global model calculation formula is as follows:

[0017]

[0018] in, is the model weight of the i-th node, β i is the resource quality weight of the node;

[0019] During the distillation process, the loss function of the local model of the edge node is defined as:

[0020] L distill =|w i -φ i | 2 .

[0021] Preferably, knowledge sharing and synchronization is achieved between edge nodes through a regular synchronization mechanism.

[0022] Preferably, the distillation temperature T is passively adjusted, and the adjustment formula is:

[0023]

[0024] Among them, α and β are adjustment coefficients, load i is the current load of node i.

[0025] As a preferred method, the core formula of the reinforcement learning algorithm is:

[0026] L(θ)=E t [min(r t (θ)A t ,clip(r t (θ), 1-∈, 1+∈)A t )]

[0027] r t =-(αL t +βE t )

[0028] A t =Q π (s t , a t )-V π (s t )

[0029] Among them, P iis the power of the ith node, T i is the computation time of the i-th node, and the delay is L t Indicates delay, E t represents energy consumption, Q π (s t , a t ) is the expected return of taking an action in the current state, V π (s t ) is the value function in the current state, and the advantage function A t Represents the gain of policy improvement.

[0030] Preferably, the specific steps of S3 include:

[0031] S3.1: Time series data is encoded using a generative pre-trained model, and the prediction capability is improved through a mask mechanism of self-supervised learning. The training objectives are:

[0032]

[0033] Among them, x τ and Represent the true value and predicted value respectively, M is the set of masked time steps; S3.2: Input the historical task load into the trained pre-trained model to output the predicted future load.

[0034] Preferably, a multi-task learning framework is introduced to enhance the robustness of the pre-trained model.

[0035] Preferably, dynamically adjusting the computing resource allocation of edge nodes also includes task priority.

[0036] The task scheduling system based on edge computing includes: a plurality of edge nodes and a central coordination module. The edge nodes include a dynamic knowledge transfer module, an energy consumption perception strategy optimization module and a time series prediction module. The central coordination module includes a model management unit and a dynamic resource scheduling module. The central coordination module globally optimizes load tasks and dynamically adjusts edge node resource configuration.

[0037] The beneficial effects of the present invention are as follows: by introducing a dynamic knowledge transfer mechanism, an energy consumption-aware strategy optimization module, and a time series prediction module, the present invention can effectively improve the efficiency of task scheduling in an edge computing environment; by controlling the number of learnable parameters, the efficiency of the scheduling model is maintained, and at the same time, through weighted aggregation strategies and resource quality scores, the importance of efficient nodes in global task scheduling is ensured; the designed adaptive resource allocation mechanism can dynamically adjust the allocation of computing resources according to the priority of the task and the real-time load of the node, effectively improving the accuracy and real-time response capability of task scheduling; through knowledge sharing and synchronization between edge nodes, the generalization ability and stability of the global model are improved. In multiple intelligent transportation and Internet of Things application scenarios, the present invention has demonstrated excellent performance, verifying its effectiveness and robustness in processing dynamic task loads, optimizing energy consumption, and balancing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the steps of the method of the present invention;

[0039] Figure 2 It is a schematic diagram of system connection for implementing the method of the present invention;

[0040] Figure 3 Schematic diagram of an integrated block diagram for realizing effective integration of global and task-related regional features in the present invention; wherein (a) is an integrated block diagram of a global model and a feature aggregation module; (b) is a block diagram of a resource quality scoring and weighted aggregation module; (c) is a block diagram of a dynamic resource scheduling module;

[0041] Figure 4 is a schematic diagram of global and local feature balance and fusion according to an embodiment of the present invention;

[0042] Figure 5 It is a visualization diagram of task scheduling and resource allocation scores according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with specific implementation examples, but the protection scope of the present invention is not limited thereto:

[0044] Example: Figure 1 As shown, the task scheduling method based on edge computing includes the following steps:

[0045] S1: A dynamic knowledge transfer mechanism is used to optimize the local task model of each edge node through multi-scale knowledge distillation and weighted model aggregation, and the global model is updated;

[0046] S2: Introduce the energy consumption perception strategy optimization module, build a Markov decision process model, use the reinforcement learning algorithm to optimize the model, and dynamically balance system performance and energy consumption;

[0047] S3: Input the historical task load into the pre-trained model and predict the future load through the self-supervised learning single-modal time series prediction module;

[0048] S4: Optimize the Markov decision process model according to the future load predicted in the previous step and dynamically adjust the computing resource allocation of edge nodes.

[0049] This method has been verified in multiple intelligent transportation and IoT application scenarios, and the accuracy and efficiency of task scheduling have been significantly improved. In the intelligent transportation hub test environment, the edge node-based task scheduling system has achieved significant optimization in low-latency and high-efficiency task processing, and the system performance has been significantly improved. By combining dynamic knowledge transfer, energy-aware optimization strategy and time series prediction module, this method successfully achieves low-latency real-time task scheduling on multiple complex transportation and IoT data sets, and effectively balances the system's energy consumption and performance, demonstrating strong task scheduling capabilities and system stability.

[0050] The following is a more detailed description of each step:

[0051] S1: A dynamic knowledge transfer mechanism is used to optimize the local task model of each edge node through multi-scale knowledge distillation and weighted model aggregation, and the global model is updated to solve the problems of node resource heterogeneity and non-independent and identically distributed (Non-IID) data distribution. The specific implementation process is as follows:

[0052] S1.1: The edge node extracts and distills multi-scale features of the local task model to generate lightweight knowledge representation; the dynamic knowledge transfer mechanism optimizes the task model of each edge node, extracts low-, medium-, and high-level features to generate multi-scale knowledge representation, which is processed using a lightweight method to facilitate subsequent aggregation in the global model. The mathematical description of multi-scale features is as follows:

[0053]

[0054] Among them, ω i represents the task model of edge node i, and T is the temperature parameter in the distillation process. Each node extracts low, medium, and high-level feature representations in hierarchical order through a multi-scale feature extraction method.

[0055] S1.2: Calculate the resource quality score of the edge nodes and use a weighted aggregation strategy to optimize the model; the resource quality score is based on comprehensive parameters such as the computing power, communication delay, and energy consumption of each node to ensure that high-quality nodes have a greater weight in the model aggregation. The calculation formula is:

[0056]

[0057] Among them, ∈ is a smoothing term to prevent division by zero errors, comp i is the computing power of the i-th node, latency i is the communication delay of node i. This step makes the weight of each node proportional to its computing power and inversely proportional to the impact of delay and energy consumption, thereby improving the aggregate influence of efficient nodes.

[0058] S1.3: In the distillation process, multi-layer feature fusion is used to improve the generalization ability of the model. The multi-scale knowledge representation extracted from the edge nodes is weighted and aggregated to generate a global task model, which improves the generalization ability and consistency of the model. Figure 3 As shown in (a), the global model and feature aggregation module realize efficient knowledge integration through the interaction and fusion of multi-layer features. This module combines the resource quality score of the node to aggregate and optimize the task model of the edge node, further improving the performance and stability of the model. The global model is updated through the following formula to achieve feature fusion and aggregation:

[0059]

[0060] in, is the model weight of the i-th node, β i is the resource quality weight of the node.

[0061] During the distillation process, the loss function of the local model of the edge node is defined as:

[0062] L distill =|w i -φ i | 2

[0063] This step ensures that the model has stronger generalization capabilities in diverse task scenarios, especially under the environmental conditions of different edge nodes, ensuring the consistency of the global model on different nodes.

[0064] S1.4: Implement knowledge sharing and synchronization between edge nodes; during the knowledge synchronization process, multiple edge nodes maintain model consistency through feature representation exchange mechanism. Figure 3 As shown in (b), the resource quality scoring and weighted aggregation module performs weighted aggregation on the model according to the computing power and communication delay of the node, thereby ensuring the priority contribution of efficient nodes. The edge nodes exchange the distilled knowledge representation through a regular synchronization mechanism to ensure the knowledge flow and update between the global model and the local model of each edge node. The synchronization mechanism adopts the following strategies:

[0065]

[0066] Among them, K is the number of nodes participating in synchronization, is the feature representation of node i after synchronization.

[0067] S1.5: Use an adaptive update strategy to adjust the distillation temperature. In order to improve the stability of the distillation process, the distillation temperature T is dynamically adjusted. The temperature parameters of the distillation process are adjusted according to the load condition, computing power, and resource usage of each node, thereby improving the stability of learning. The adjustment formula is:

[0068]

[0069] Among them, α and β are adjustment coefficients, load i is the current load of node i.

[0070] S2: Introduce the energy consumption awareness strategy optimization module, build a Markov decision process model, use the reinforcement learning algorithm to optimize the model, and dynamically balance system performance and energy consumption; this step models the resource allocation problem as a Markov decision process.

[0071] The specific steps are as follows:

[0072] S2.1: Define the state space, action space and reward function for task scheduling;

[0073] In the edge computing environment, the state space S t Including the current resource utilization, energy consumption estimation and predicted workload of each node, the action space A t It represents the task migration strategy and computing resource allocation scheme. The reward function combines delay and energy consumption and is defined as:

[0074] r t =-(αL t +βE t );

[0075] Delay L t and energy consumption E t The specific calculation formula is as follows:

[0076]

[0077] Among them, P i is the power of the ith node, T i is the computation time of the i-th node.

[0078] S2.2: Use reinforcement learning algorithm to optimize strategy and dynamically balance task delay and energy consumption; reinforcement learning algorithm achieves strategy optimization through Proximal Policy Optimization (PPO), and its core formula is:

[0079] L(θ)=E t [min(rt (θ)A t ,clip(r t (θ), 1-∈, 1+∈)A t )]

[0080] Among them, the advantage function A t It represents the gain of strategy improvement, and the calculation formula is:

[0081] A t =Q π (s t , a t )-V π (s t )

[0082] Q π (s t , a t ) is the expected return of taking an action in the current state, V π (s t ) is the value function in the current state.

[0083] S2.3: Design a dynamic resource allocation mechanism based on task priority; introduce task priority and adjust the resource allocation ratio according to the urgency and importance of the task. The task priority calculation formula is: Among them, priority i is the priority of task i, T deadline is the deadline of task i.

[0084] S3: Input the historical task load into the pre-trained model and predict the future load through the self-supervised learning single-modal time series prediction module. The specific steps are as follows:

[0085] S3.1: Use self-supervised learning methods to pre-train the GPT model; time series data is encoded using the Generative Pre-Trained Transformer (GPT) model, and the prediction ability is improved through the mask mechanism of self-supervised learning. The training objectives are:

[0086]

[0087] Among them, x τ and Represent the true value and predicted value respectively, and M is the set of masked time steps.

[0088] S3.2: Predicting future load based on historical task load data. In the time series prediction module, the GPT model predicts future resource requirements based on historical task load data. Figure 4As shown in the figure, the balance and fusion of global and local features optimizes the task allocation strategy by combining the time series prediction results, which significantly improves the accuracy and efficiency of task scheduling. t Generate future forecasts

[0089]

[0090] S3.3: Introduce multi-task learning to enhance the robustness of the prediction model; through the multi-task learning framework, combine the prediction information of multiple related tasks, share the feature space to improve the generalization ability of the model. The multi-task loss function is defined as:

[0091]

[0092] Among them, λ i For each task’s weight, L i is the prediction error of task i.

[0093] S4: Optimize the Markov decision process model according to the future load predicted in the previous step and dynamically adjust the computing resource allocation of edge nodes.

[0094] Use the prediction results to optimize task scheduling. The prediction results are used as input to update the state in the reinforcement learning module to optimize the task scheduling scheme. The time series prediction module and the energy consumption perception strategy optimization module form a closed loop to improve the overall system performance.

[0095] like Figure 3 As shown in (c), the dynamic resource scheduling module dynamically allocates computing resources according to the priority of each task and the real-time status of the node to ensure the real-time performance of the system and task response. Combined with the output of the time series prediction module, it adjusts the computing resource usage ratio of the node C i :

[0096] This step dynamically adjusts the resource allocation plan to ensure that high-priority tasks receive priority computing rights.

[0097] like Figure 2 As shown, a task scheduling system based on edge computing is proposed, and the method of the present invention is implemented on the system. The specific contents of the system include: a number of edge nodes and a central coordination module. The edge nodes include a dynamic knowledge transfer module, an energy consumption perception strategy optimization module and a time series prediction module. The central coordination module includes a model management unit and a dynamic resource scheduling module.

[0098] The central coordination module is used for global task scheduling optimization to ensure that the system can maintain high performance under high load conditions. The model update formula is:

[0099]

[0100] The model management unit is used for global model aggregation, and the dynamic resource scheduling module is used to dynamically adjust the resource configuration of edge nodes.

[0101] The distributed task scheduling algorithm used is used to dynamically distribute the load. The tasks are globally optimized and distributedly scheduled through the central coordination module. The task load ratio and priority of each node are dynamically adjusted:

[0102]

[0103] like Figure 5 As shown, the visualization of task scheduling and resource allocation scores shows how the system allocates resources based on the computing power, energy consumption and task priority of the nodes, verifying the effectiveness of the scheduling strategy.

[0104] The method of the present invention can achieve efficient collaboration between the global task model and the local task model of the edge node in the edge computing environment by introducing a dynamic knowledge transfer mechanism and an energy consumption perception strategy optimization module, combined with a time series prediction module based on self-supervised learning. Relying on weighted aggregation and knowledge synchronization mechanisms, the features of the global and local models are fully integrated. At the same time, through dynamic resource allocation and priority scheduling strategies, the efficiency and accuracy of task scheduling are significantly improved, while reducing computing overhead and energy consumption, ensuring high real-time and high-efficiency task execution, and ultimately achieving excellent performance in a variety of complex task scenarios.

[0105] The above description is a specific embodiment of the present invention and the technical principles used. If the changes made according to the concept of the present invention do not exceed the spirit covered by the description and drawings, they should still fall within the scope of protection of the present invention.

Claims

1. The task scheduling method based on edge computing is characterized by The following steps are involved: S1: A dynamic knowledge transfer mechanism is used to optimize the local task model of each edge node through multi-scale knowledge distillation and weighted model aggregation, and the global model is updated; S2: Introduce the energy consumption perception strategy optimization module, build a Markov decision process model, use the reinforcement learning algorithm to optimize the model, and dynamically balance system performance and energy consumption; S3: Input the historical task load into the pre-trained model and predict the future load through the self-supervised learning single-modal time series prediction module; S4: Optimize the Markov decision process model according to the future load predicted in the previous step and dynamically adjust the computing resource allocation of edge nodes.

2. The task scheduling method based on edge computing according to claim 1 is characterized in that S1 adopts a lightweight method to extract low-, medium-, and high-level feature representations in hierarchical order to generate multi-scale knowledge representations: Among them, ω i represents the task model of edge node i, and T is the temperature parameter in the distillation process.

3. The task scheduling method based on edge computing according to claim 1 is characterized in that: High-quality nodes have a greater weight in the weighted model aggregation process. High-quality nodes are evaluated by calculating the quality scores of edge nodes. The evaluation content includes: the node's computing power, communication delay and energy consumption.

4. The task scheduling method based on edge computing according to claim 1 is characterized in that the global The model calculation formula is as follows: in, is the model weight of the i-th node, β i is the resource quality weight of the node; During the distillation process, the loss function of the local model of the edge node is defined as: L distill =|w i -f i | 2 。 5. The task scheduling method based on edge computing according to any one of claims 1 to 4, characterized in that Knowledge sharing and synchronization are achieved between edge nodes through regular synchronization mechanisms.

6. The task scheduling method based on edge computing according to claim 2 is characterized in that: The distillation temperature T is passively adjusted, and the adjustment formula is: Among them, α and β are adjustment coefficients, load i is the current load of node i.

7. The task scheduling method based on edge computing according to claim 1 is characterized in that The core formula of the reinforcement learning algorithm is: L(θ)=E t [min(r t (i)A t ,clip(r t (θ), 1-∈, 1+∈)A t )] r t =-(αL t +βE t ) A t =Q π (s t ,a t )-V π (s t ) Among them, P i is the power of the ith node, T i is the computation time of the i-th node, and the delay is L t Indicates delay, E t represents energy consumption, Q π (s t , a t ) is the expected return of taking an action in the current state, V π (s t ) is the value function in the current state, and the advantage function A t Represents the gain of policy improvement.

8. The task scheduling method based on edge computing according to claim 1 is characterized in that The specific steps of S3 include: S3.1: Time series data is encoded using a generative pre-trained model, and the prediction capability is improved through a mask mechanism of self-supervised learning. The training objectives are: Among them, x τ and Represent the true value and predicted value respectively, and M is the set of masked time steps; S3.2: Input the historical task load into the trained pre-trained model to output the predicted future load.

9. The task scheduling method based on edge computing according to claim 8 is characterized in that A multi-task learning framework is introduced to enhance the robustness of the pre-trained model.

10. The task scheduling system based on edge computing is characterized by include: Several edge nodes and a central coordination module. The edge node includes a dynamic knowledge transfer module, an energy consumption perception strategy optimization module and a time series prediction module. The central coordination module includes a model management unit and a dynamic scheduling resource module. The central coordination module globally optimizes the load tasks and dynamically adjusts the edge node resource configuration.

Images