A cloud edge end computing power scheduling and task unloading management method and system

Abstract

This invention relates to the field of Internet of Things (IoT) technology, specifically to a method and system for cloud-edge-device computing power scheduling and task offloading management. The technical solution includes: real-time data collection of resource and network status, and preprocessing the collected data to obtain a standardized dataset. Predictive values ​​are obtained for resource load and network status in the cloud-edge-device environment. Then, model training is performed to obtain the current optimal model parameters. Based on the current optimal model, predictions are made, and a multi-objective function vector is constructed using a non-dominated sorting algorithm to obtain the optimal decision scheme. Based on the optimal decision scheme, a three-level scheduling mechanism (global scheduling, regional scheduling, and local scheduling) is implemented to execute computing power scheduling and task allocation. The entire lifecycle of tasks is monitored and managed. This invention effectively solves the core defects of existing cloud-edge-device collaborative systems, such as low prediction accuracy, decision lag, low resource utilization, poor reliability, and insufficient privacy protection.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) technology, specifically to a method and system for cloud-edge-device computing power scheduling and task offloading management. Background Technology

[0002] With the deep integration of cloud computing, edge computing, and IoT technologies, the "cloud-edge-device" collaborative architecture has been widely applied in smart cities, industrial internet, vehicle-to-everything (V2X) and other fields, becoming the core architecture supporting the collaborative work of massive heterogeneous terminal devices and meeting real-time business needs. Currently, the number of various terminal devices (sensors, industrial robots, smart cameras, etc.) is growing explosively, generating massive amounts of data that require real-time processing and efficient scheduling. At the same time, different business scenarios have increasingly stringent requirements for task latency, system energy consumption, operating costs, and data privacy. Cloud-edge-device computing power scheduling and task offloading have become crucial to ensuring the efficient operation of the system.

[0003] However, existing cloud-edge-device collaborative systems still have many bottlenecks in computing power scheduling and task offloading technologies, making it difficult to adapt to actual application needs. They suffer from low accuracy in predicting resource and network status, high inference latency, and an inability to accurately predict future load fluctuations, resulting in delayed scheduling decisions. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a cloud-edge-device computing power scheduling and task offloading management method for constructing a closed-loop system of "prediction-decision-scheduling-execution-monitoring-feedback". The cloud-edge-device computing power scheduling and task offloading management method includes: S1. Collect resource and network status data in real time at a preset frequency, and preprocess the collected data to obtain a standardized dataset.

[0005] S2. Based on the Long Short-Term Memory Network-Gated Recurrent Unit Hybrid Neural Network Prediction Model, the predicted value x' is obtained by predicting the resource load and network status in the cloud-edge-device environment.

[0006] S3. Based on the standardized dataset in S1 and the predicted values ​​in S2, train the model to obtain the current optimal model parameters.

[0007] S4. Based on the current optimal model, predictions are made using four objective evaluation items: delay, energy consumption, cost, and reliability. A multi-objective function vector is constructed using a non-dominated sorting algorithm, and then the optimal decision scheme is obtained.

[0008] S5. Based on the optimal decision-making scheme, complete the three-level scheduling mechanism of global scheduling, regional scheduling, and local scheduling to perform computing power scheduling and task allocation.

[0009] S6. Monitor and manage the entire lifecycle of tasks.

[0010] Preferred data includes: CPU utilization, memory utilization, GPU utilization, disk I / O speed, disk utilization, task queue length, number of tasks being executed, uplink bandwidth, downlink bandwidth, network latency, network jitter and / or packet loss rate.

[0011] Preferred prediction models include: an input layer, a data normalization layer, a long short-term memory network layer, a random deactivation layer, a two-layer gated recurrent unit layer, a multi-head self-attention layer, a two-layer fully connected layer, and an output layer.

[0012] Preferred configuration: Input layer: Receives three-dimensional time series data, with dimensions including batch size, historical time window size, and number of features; Data normalization layer: Uses a min-max normalization method to linearly map all feature values ​​to the [0,1] interval; Long Short-Term Memory (LSTM) network layer: Contains 128 hidden units, effectively capturing long-term dependencies in the time series through three gating structures: input gate, forget gate, and output gate. Each gate weights the input information with the hidden state of the previous time step and outputs the result through a non-linear activation function; Random deactivation layer: Randomly discards 20% of neuron outputs during training; Two-layer gated recurrent unit (ROU) layer: Each layer contains 64 hidden units, merging the input gate and forget gate into a single layer. The system includes an update gate and a reset gate, resulting in a simpler structure and faster inference speed. It is responsible for extracting short-term features from time series data. A multi-head self-attention layer employs a 4-head self-attention mechanism, automatically learning the importance weights of different historical time steps for future predictions, focusing on time steps with significant impact on the prediction results. A two-layer fully connected layer consists of 64 neurons in the first layer and 32 neurons in the second, both using linear rectified functions as activation functions. This layer maps the high-dimensional features output by the attention mechanism to a low-dimensional space. The output layer has the batch size and prediction step size as its output dimensions, where the prediction step size represents the number of future time steps to predict (a recommended value is 5). It uses a linear activation function and directly outputs the predicted value.

[0013] Preferred: Obtaining the current optimal model parameters includes: time series construction, model training, mixed prediction, and online updates.

[0014] Preferred approach: Time series construction: A sliding window method is used to convert a one-dimensional time series into a two-dimensional dataset suitable for supervised learning; Model training: An adaptive moment estimation optimizer is used, with an initial learning rate set to 0.001, and mean squared error is used as the loss function; Hybrid prediction: Combining the advantages of direct prediction and recursive prediction, a direct prediction method is used for short-term predictions (1-3 steps), i.e., training a model to directly output the prediction results for multiple time steps; a recursive prediction method is used for long-term predictions (4 steps or more), i.e., the prediction result of the previous step is used as the input for the next prediction; Online update: A sliding window of size N is maintained to store historical data from the most recent N time steps; whenever new data is collected, it is added to the sliding window, while the oldest data in the window is removed; every M time steps, the model is fine-tuned using the data in the sliding window, with a small learning rate and only the parameters of the last few layers of the model are updated.

[0015] The preferred method for constructing a multi-objective function vector using a non-dominated sorting algorithm includes the following steps: Step 1: Initialize the population; Step 2: Calculate the objective function; Step 3: Non-dominated sorting; Step 4: Adaptive adjustment of reference points; Step 5: Association operation and niching selection; Step 6: Adaptive crossover and mutation operation; Step 7: Population merging and iteration; Step 8: Selection of the optimal decision scheme.

[0016] Preferred initialization includes: randomly generating an initial population of a preset size, with each individual in the population corresponding to a complete task unloading decision scheme, using integer encoding, where each gene in the encoding corresponds to the execution node of a single task, and the encoding length is consistent with the total number of tasks to be unloaded in the system; at the same time, algorithm termination conditions are set, including the maximum number of iterations and the population convergence accuracy threshold. It is recommended that the maximum number of iterations be set to 100-200 times and the convergence accuracy threshold be set to 1e-4.

[0017] Preferred method: The objective function calculation includes: constructing an objective function vector for each individual in the population, wherein the objective function vector is: Where y is the instruction task, f latency (y) represents the total task delay; f energy (y) represents the total energy consumption of the system; f cost (y) represents the computation and transmission costs; f reliability (y) represents the task execution failure rate.

[0018] Preferred non-dominated sorting includes: performing non-dominated sorting on all individuals in the population according to four objective function values, dividing the population into different frontiers, where the first frontier F1 is the non-dominated solution set, the second frontier F2 is the solution set dominated by F1 but not dominated by other frontiers, and so on, to complete the hierarchical sorting of all individuals, providing a basis for subsequent elite selection and population update.

[0019] The preferred method for adaptive adjustment of reference points includes: generating an initial set of reference points using the boundary cross method, with the total number of reference points set according to the optimization requirements of the four objectives; then, dynamically adjusting the position and number of reference points based on the current population distribution: splitting reference points with more than a threshold of associated individuals and deleting reference points with less than a threshold of associated individuals, ensuring that reference points can accurately match the population distribution and guiding the algorithm to evolve towards the global optimum.

[0020] Preferred method: Association operation and niching selection include: using a weighted distance association metric to associate individuals in each front surface with the nearest reference point, and calculating the number of individuals associated with each reference point; then using a niching selection method to prioritize individuals in the first front surface. When the number of individuals in the first front surface is insufficient, individuals corresponding to reference points with fewer associated individuals are selected from subsequent front surfaces. This ensures that the selected individuals have both excellent objective function values ​​and uniform distribution of the Pareto optimal solution set, forming the next generation of parent population.

[0021] Preferred: Adaptive crossover and mutation operations include: dynamically calculating the crossover probability and mutation probability based on the current population's convergence (CD) and diversity (DV), where the convergence (CD) reflects the population's concentration and the diversity (DV) reflects the population's dispersion; performing crossover and mutation operations on the parent population to generate the offspring population, with single-point crossover and random mutation operations.

[0022] Preferred method: Population merging and iteration includes: merging the parent population and the offspring population to form a new population of twice the size, repeating steps 2 to 6 until a preset termination condition is met.

[0023] Preferred: The optimal decision scheme selection includes: after the algorithm terminates, outputting the final Pareto optimal solution set (i.e., all individuals in the first frontier); using the approximation ideal solution sorting method, calculating the relative proximity of each individual in the Pareto optimal solution set to the positive ideal solution and the negative ideal solution, and selecting the individual with the largest relative proximity as the final optimal decision scheme for task unloading.

[0024] Preferred scheduling methods: Global scheduling: Executed by the cloud center, responsible for large-scale task scheduling and resource allocation across regions. Resource reservation and load balancing are performed based on long-term forecasts, optimizing the overall system's resource utilization and energy consumption. Regional scheduling: Executed by edge nodes, responsible for real-time task scheduling and load balancing within their respective regions. Task allocation is dynamically adjusted based on short-term forecasts to meet the low-latency requirements of real-time applications. Local scheduling: Executed by terminal devices, responsible for executing local tasks and making simple decisions. For tasks with low computational load and extremely high real-time requirements, processing is done directly on the terminal device, avoiding latency and bandwidth consumption caused by data transmission. When resources at a certain level are insufficient, resource support is automatically requested from the next higher level; when resources at a certain level are abundant, tasks are proactively distributed to the next lower level, achieving load balancing across the entire system.

[0025] The preferred approach involves full lifecycle monitoring and management of tasks, including: task classification and grading, task status monitoring, task migration mechanisms, checkpoint-based task recovery, and multi-replica execution strategies. Task classification and grading: Tasks are categorized into three levels—real-time, near-real-time, and non-real-time—based on their priority, latency requirements, computational complexity, and data sensitivity. Task status monitoring: The execution status of tasks is monitored in real-time, including five states: waiting, running, paused, failed, and completed. Task migration mechanism: When a node failure, network interruption, or excessive load is detected, task migration is immediately triggered, transferring the task to another healthy node. Checkpoint-based task recovery: The execution status checkpoints of tasks are periodically saved, allowing execution to resume from the point of interruption after migration, avoiding redundant calculations and improving system efficiency. Multi-replica execution strategy: For critical tasks, replicas are executed simultaneously on multiple nodes. The task succeeds as long as one replica completes successfully, significantly improving system reliability.

[0026] This invention also proposes a cloud-edge-device computing power scheduling and task offloading management system, which includes: The resource status awareness module is the foundation for the system to make forward-looking scheduling and offloading decisions. It is deployed at various levels of cloud center, edge nodes and terminal devices to complete the real-time collection and preprocessing of the system's resource and network status to obtain a standardized dataset.

[0027] The state prediction module, based on a hybrid neural network prediction model of long short-term memory network-gated recurrent unit, predicts resource load and network state in cloud-edge-device environments to obtain predicted values ​​x'.

[0028] The model selection module is used to train the model based on the standardized dataset and predicted values, obtain the current optimal model parameters, and construct the current optimal model based on the current optimal model parameters.

[0029] The multi-objective optimization task offloading decision module is used to make predictions based on the current optimal model using four objective evaluation items: delay, energy consumption, cost, and reliability. It then uses a non-dominated sorting algorithm to construct a multi-objective function vector and obtain the optimal decision scheme.

[0030] The hierarchical dynamic computing power scheduling module is used to execute a three-level scheduling mechanism of "global scheduling + regional scheduling + local scheduling" based on the optimal decision scheme output by the multi-objective optimization task unloading decision module, so as to realize the dynamic allocation and load balancing of computing power resources of the entire system.

[0031] The task lifecycle management and fault tolerance module is deployed on edge nodes and cloud centers to monitor and manage all tasks throughout the entire process, thereby improving system reliability and task execution success rate.

[0032] The privacy-protected data processing module is deployed at all levels of the system to protect the privacy of sensitive data without affecting system performance, and to meet data security regulations.

[0033] The technical effects and advantages of this invention are as follows: This invention effectively solves the core defects of existing cloud-edge-device collaborative systems, such as low prediction accuracy, decision lag, low resource utilization, poor reliability, and insufficient privacy protection, which are described in detail below: 1. It achieves high-precision and low-latency prediction of resource and network status. Its average absolute error for CPU utilization prediction is only 3.89%, and the average absolute percentage error for network latency prediction is 7.89%. The inference time is as low as 1.42 milliseconds / sample. Compared with a single LSTM or GRU model, both accuracy and speed are significantly improved, providing reliable support for forward-looking scheduling.

[0034] 2. Balancing the four objectives of latency, energy consumption, cost, and reliability, it breaks through the limitations of single-objective optimization in existing technologies, achieves global optimization of task offloading decisions, reduces the average task response latency of the system by 40%-60%, meets the requirements of millisecond-level real-time applications, and reduces the total energy consumption of the system by 25%-40%, significantly reducing operating costs.

[0035] 3. The "global + regional + local" three-level scheduling mechanism takes into account both global resource optimization and local real-time requirements, fully taps into idle computing power in the cloud, edge and terminal, and increases the overall system resource utilization rate from 35% to over 72%, achieving efficient allocation and load balancing of computing resources and adapting to the collaborative work needs of heterogeneous devices.

[0036] 4. A robust task lifecycle management and fault tolerance mechanism, combined with checkpoint recovery and multi-objective execution strategies, increases the system task execution success rate to over 99.9%, significantly enhancing system reliability and preventing task failures caused by node failures or network interruptions. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating a cloud-edge-device computing power scheduling and task unloading management method proposed in this invention.

[0038] Figure 2 This is a flowchart illustrating the prediction model in a cloud-edge-device computing power scheduling and task offloading management method proposed in this invention.

[0039] Figure 3 This is a flowchart illustrating the method for obtaining the current optimal model parameters in a cloud-edge-device computing power scheduling and task unloading management method proposed in this invention.

[0040] Figure 4 This is a flowchart illustrating a method for constructing multi-objective function vectors using a non-dominated sorting algorithm in a cloud-edge-device computing power scheduling and task unloading management method proposed in this invention.

[0041] Figure 5 This is a structural block diagram of a cloud-edge-device computing power scheduling and task unloading management system proposed in this invention. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the invention, and should not be construed as limiting the invention. Rather, embodiments of the invention include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0043] Example 1 refer to Figure 1 This embodiment proposes a cloud-edge-device computing power scheduling and task offloading management method to construct a closed-loop system of "prediction-decision-scheduling-execution-monitoring-feedback". The cloud-edge-device computing power scheduling and task offloading management method includes: S1. Real-time data collection of resource and network status is performed at a preset frequency, and the collected data is preprocessed to obtain a standardized dataset. The preset frequency can be set to collect data once every 1 second to 1 minute according to actual needs, and other frequency settings are not excluded, but will not be elaborated here. Data collection can be carried out through resource monitoring agents deployed on various nodes. The collected data may include: CPU utilization, memory utilization, GPU utilization, disk I / O speed, disk utilization, task queue length, number of tasks being executed, uplink bandwidth, downlink bandwidth, network latency, network jitter, and packet loss rate, etc. For example, deployed in 10 production workshops, each workshop is equipped with 2 edge servers, responsible for monitoring the production process, real-time equipment control, and product quality inspection within the workshop. Each production equipment collects production data in real time at a frequency of 1 second, including temperature, pressure, rotation speed, vibration, displacement, etc. Data preprocessing may include: missing value handling: missing values ​​are filled using linear interpolation, and data segments that are missing for more than 3 consecutive time steps are deleted. outlier handling: outliers are identified using the 3σ principle, and data exceeding the mean ± 3 times the standard deviation are replaced with the median of the feature. Data normalization: Perform min-max standardization on all features to obtain a standardized dataset.

[0044] S2. A prediction model based on a hybrid neural network of long short-term memory network and gated recurrent unit is used to predict resource load and network status in a cloud-edge-device environment to obtain the predicted value x'. (Reference) Figure 2This model employs a hierarchical structure, consisting of the following layers from input to output: input layer, data normalization layer, long short-term memory network layer, random deactivation layer, two-layer gated recurrent unit layer, multi-head self-attention layer, two-layer fully connected layer, and output layer. The input layer receives three-dimensional time series data, with dimensions including batch size, historical time window size, and number of features. The historical time window size represents the time step of the historical data used for prediction, with a recommended value of 30; the number of features represents the number of resources and network state indicators collected, typically 8-16. The data normalization layer uses a min-max normalization method to linearly map all feature values ​​to the [0,1] interval, eliminating dimensional differences between different features. The long short-term memory network layer contains 128 hidden units and effectively captures long-term dependencies in the time series, such as daily and weekly cycles, through three gating structures: input gate, forget gate, and output gate. Each gate weights the input information with the hidden state of the previous time step and outputs the result through a non-linear activation function. Random deactivation layer: Randomly discards 20% of neuron outputs during training to prevent overfitting and improve generalization ability. Two-layer gated recurrent unit layer: Each layer contains 64 hidden units, merging the input gate and forget gate into an update gate, while also introducing a reset gate. This simplifies the structure and speeds up inference, extracting short-term features from the time series, such as burst loads and transient network fluctuations. The first layer returns the complete sequence output, while the second layer only returns the output of the last time step. Multi-head self-attention layer: Employs a 4-head self-attention mechanism, automatically learning the importance weights of different historical time steps for future predictions, focusing on time steps with a significant impact on the prediction result. Its mathematical expression is: Where T is the matrix transpose symbol, q is the query vector, k is the key vector, and v is the value vector, which is calculated by linearly projecting the basic feature vectors using three independent learnable linear transformation matrices (also called weight matrices), the details of which will not be elaborated here. k Let Sm be the dimension of the key vector. Sm (soft maximization function) is a normalization calculation method whose core function is to transform the "raw association score" obtained from matrix multiplication in the multi-head self-attention layer. The input is mapped to the [0,1] interval, and the sum of the scores of all historical time steps is 1. The final output is the normalized weight of each historical moment for the future prediction. Multi-head attention linearly projects the input to multiple different subspaces, calculates the attention separately, and then concatenates the results. Two fully connected layers: The first layer contains 64 neurons, and the second layer contains 32 neurons. Both use the linear rectified function as the activation function and are responsible for mapping the high-dimensional features output by the attention mechanism to the low-dimensional space, preparing for the final prediction. Output layer: The output dimension is (batch size, prediction step size), where the prediction step size represents the number of time steps to predict the future, and the recommended value is 5. A linear activation function is used to directly output the predicted value.

[0045] S3. Based on the standardized dataset in S1 and the predicted values ​​in S2, train the model to obtain the current optimal model parameters. (Reference) Figure 3 Obtaining the current optimal model parameters can include: time series construction, model training, fusion prediction, and online updates. Time series construction: A one-dimensional time series is transformed into a two-dimensional dataset suitable for supervised learning using a sliding window method. Specifically, for each time step t, the parameters are calculated using a time series model from tt... w Historical data from t+1 to t is used as input, and data from t+1 to t+t is used as input. p The future data will be used as the output. w t represents the length of the historical time window. p Future prediction step size. Metrics such as computing load, CPU usage, memory load, network latency, bandwidth traffic, and link jitter are all time-dependent nonlinear time series: the current node load is determined by the concurrent business, task backlog, and traffic surges of the previous few seconds / minutes; network congestion and link fluctuations exhibit continuity, periodicity, and short-term abrupt changes; single-point data at a single moment cannot reflect the trend of change, only the instantaneous static value. Therefore, we cannot predict the future using only single-point data at time t; we must introduce a continuous historical window to capture: short-term fluctuation trends, periodic change patterns, and precursory features of sudden traffic / computing load surges. Model training: An adaptive moment estimation optimizer is used, with an initial learning rate set to 0.001, and then the mean squared error is used as the loss function, the mathematical expression of which is: Where i is the training sample ID, I is the total number of training samples, i = 1, 2, ..., I; x i x represents the normalized true value of the data in S1. i 'The predicted value calculated in S2; an early stopping mechanism is adopted, when the validation set loss does not decrease for 10 consecutive training epochs, training is stopped and the current optimal model parameters are saved; this method is fast and accurate, but large data biases are easily masked by historical data and become distorted. The loss function can also be...' Where i is the training sample number, I is the total number of training samples, i=1,2,…,I; x i To standardize the true value of the data, x i ' is the predicted value calculated in S2; t i This refers to the time interval between the sampling time point numbered i and the standard point t0. The standard point t0 is earlier than the sampling time point of the training samples, i.e., t i <t i+1'a' represents the time effect base, ranging from 1 to 5, though other values ​​are not excluded. The specific calculation process is not detailed here. The loss function constructed using this method avoids the distortion caused by historical data masking large-scale data biases. It also adds a time attribute to historical data, weakens older data, and highlights the weight of recent data, improving calculation accuracy. Hybrid Prediction: Combining the advantages of direct and recursive prediction, direct prediction is used for short-term predictions (1-3 steps), where a model is trained and outputs predictions for multiple time steps. Recursive prediction is used for long-term predictions (4 steps or more), where the prediction result of the previous step is used as the input for the next step. Online Update: A sliding window of size N is maintained, storing historical data from the most recent N time steps. Whenever new data is collected, it is added to the sliding window, while the oldest data in the window is removed. Every M time steps, the model is fine-tuned using data from the sliding window. During fine-tuning, a small learning rate (e.g., 0.0001) is used, and only the parameters of the last few layers of the model are updated to avoid destroying the learned general features.

[0046] S4. Based on the current optimal model, predictions are made using four evaluation criteria: delay, energy consumption, cost, and reliability. A non-dominated sorting algorithm is then used to construct a multi-objective function vector to obtain the optimal decision scheme, thereby achieving global optimum in task offloading decisions. Specifically, the predictions based on the current optimal model are obtained by inputting the corresponding data into the parameters of the current optimal model. Details are not elaborated here. (Reference) Figure 4 The construction of a multi-objective function vector using a non-dominated sorting algorithm can specifically include: Step 1: Population initialization. A pre-defined initial population is randomly generated. Each individual in the population corresponds to a complete task unloading decision scheme, using integer encoding. Each gene in the encoding corresponds to the execution node (terminal device, edge node, or cloud center) of a single task. The encoding length is consistent with the total number of tasks to be unloaded in the system. Simultaneously, algorithm termination conditions are set, including the maximum number of iterations and the population convergence accuracy threshold. It is recommended to set the maximum number of iterations to 100-200 and the convergence accuracy threshold to 1e-4. Step 2: Objective function calculation. For each individual (task unloading decision scheme) in the population, an objective function vector is constructed. The objective function vector is: Where y is the instruction task, f latency (y) represents the total task latency, indicating the total time from submission to completion of all tasks, including data transmission latency, computation latency, and queuing latency. f energy (y) represents the total system energy consumption: This indicates the total energy consumed by the system to execute all tasks, including energy consumption of terminal devices, edge nodes, and the cloud. f cost(y) represents the computation and transmission costs, indicating the total economic cost of executing all tasks, including cloud resource rental costs and network bandwidth costs. f reliability(y) represents the task failure rate, which is the sum of the probabilities of all task failures and is related to the node failure rate and network outage rate. Its individual values ​​need to be calculated by summing their components, which will not be elaborated here. Step 3: Non-dominated sorting. All individuals in the population are non-dominated according to the four objective function values, dividing the population into different frontiers. The first frontier, F1, is the non-dominated solution set (i.e., no other individual is superior to this individual in all four objectives). The second frontier, F2, is the solution set dominated by F1 but not by other frontiers, and so on, completing the hierarchical sorting of all individuals to provide a basis for subsequent elite selection and population updates. Step 4: Reference point adaptive adjustment. An initial set of reference points is generated using the boundary crossover method, with the total number of reference points set according to the optimization requirements of the four objectives. Then, based on the current population distribution, the position and number of reference points are dynamically adjusted: reference points with more than a threshold of associated individuals are split, and reference points with fewer than a threshold are deleted, ensuring that reference points accurately match the population distribution and guiding the algorithm towards the global optimum. The threshold is set according to specific circumstances and will not be elaborated here. Step 5: Association operation and niching selection. A weighted distance association metric can be used to associate individuals in each front surface with the nearest reference point, calculating the number of individuals associated with each reference point. Then, a niching selection method is used, prioritizing individuals in the first front surface. When the number of individuals in the first front surface is insufficient, individuals corresponding to reference points with fewer associated individuals are selected from subsequent front surfaces, ensuring that the selected individuals possess both excellent objective function values ​​and maintain the uniformity of the Pareto optimal solution set, forming the next generation of parent population. Step 6: Adaptive crossover and mutation operation. Based on the current population's convergence (CD) and diversity (DV), the crossover and mutation probabilities are dynamically calculated. CD reflects the population's concentration, and DV reflects its dispersion. Crossover and mutation operations are performed on the parent population to generate the offspring population. Single-point crossover and random mutation are used. The crossover and mutation probabilities are adaptively adjusted to avoid premature convergence or excessively slow convergence. Step 7: Population Merging and Iteration. The parent and offspring populations are merged to form a new population twice the size. Steps 2 through 6 are repeated until the preset termination condition is met (reaching the maximum number of iterations or a population convergence accuracy threshold). Step 8: Optimal Decision Scheme Selection. After the algorithm terminates, it outputs the final Pareto optimal solution set (i.e., all individuals on the first frontier). Using the approximation-ideal-solution sorting method, it calculates the relative proximity of each individual in the Pareto optimal solution set to the positive and negative ideal solutions, selecting the individual with the highest relative proximity as the optimal decision scheme for final task unloading. This scheme can simultaneously achieve a globally optimal balance of four objectives, adapting to the current cloud-edge-device resource status and business requirements. The crossover and mutation probabilities are dynamically adjusted based on the population's evolutionary state.When the population convergence speed is slow, the crossover and mutation probabilities are increased to enhance global search capabilities; when the population converges to a certain extent, the crossover and mutation probabilities are decreased to strengthen local search capabilities. Not only are non-dominated solutions retained, but also some dominant solutions with good diversity are preserved to avoid the algorithm getting trapped in local optima while ensuring population diversity. Uniformly distributed reference points are used to guide the evolutionary direction of the population, ensuring that the Pareto optimal solution set has good distribution and uniformity. An approximation-ideal-solution sorting method is used to select the decision scheme that best meets the current business needs from the Pareto optimal solution set. This method calculates the distance of each solution to the positive and negative ideal solutions to obtain the relative proximity of each solution, and selects the solution with the highest relative proximity as the optimal decision scheme. The weights of each objective can be dynamically adjusted according to the priority of different application scenarios.

[0047] S5. Based on the optimal decision-making scheme, a three-level scheduling mechanism—global scheduling, regional scheduling, and local scheduling—is implemented to perform computing power scheduling and task allocation, ensuring both global optimization and meeting real-time requirements. Global scheduling: Executed by the cloud center, responsible for large-scale task scheduling and resource allocation across regions. Resource reservation and load balancing are performed based on long-term prediction results, optimizing the overall system's resource utilization and energy consumption. Regional scheduling: Executed by edge nodes, responsible for real-time task scheduling and load balancing within their respective regions. Task allocation is dynamically adjusted based on short-term prediction results to meet the low-latency requirements of real-time applications. Local scheduling: Executed by terminal devices, responsible for executing local tasks and making simple decisions. For tasks with low computational load and extremely high real-time requirements, processing is done directly on the terminal device, avoiding latency and bandwidth consumption caused by data transmission. When resources at a certain level are insufficient, resource support is automatically requested from the next higher level; when resources at a certain level are abundant, tasks are proactively distributed to the next lower level, achieving load balancing across the entire system.

[0048] S6. Full lifecycle monitoring and management of tasks. This invention establishes a comprehensive task lifecycle management mechanism, enabling full monitoring and management of tasks from submission to completion. Specifically, this includes: task classification and grading, task status monitoring, task migration mechanism, checkpoint-based task recovery, and multi-replica execution strategy. Task classification and grading: Tasks are categorized into three levels—real-time, near-real-time, and non-real-time—based on their priority, latency requirements, computational complexity, and data sensitivity. Task status monitoring: Real-time monitoring of task execution status, including five states: waiting, running, paused, failed, and completed. Task migration mechanism: When a node failure, network interruption, or excessive load is detected, task migration is immediately triggered, transferring the task to another normal node. Checkpoint-based task recovery: Task execution status checkpoints are periodically saved, allowing tasks to resume execution from the point of interruption after migration, avoiding redundant calculations and improving system efficiency. Multi-replica execution strategy: For critical tasks, replicas are executed simultaneously on multiple nodes. As long as one replica completes successfully, the task is considered successful, significantly improving system reliability.

[0049] Example 2 refer to Figure 5 This invention also proposes a cloud-edge-device computing power scheduling and task offloading management system. This invention adopts a three-tiered collaborative architecture of "cloud-edge-device," with clearly defined functions at each level and mutual cooperation. The cloud center layer is responsible for global resource monitoring, large-scale data processing, model training and updating, and cross-regional long-term task scheduling decisions. The edge node layer includes edge servers, base stations, gateways, etc., and is responsible for regional resource management, real-time task processing, and short-term task scheduling decisions. The terminal device layer includes various sensors, smart terminals, industrial equipment, etc., and is responsible for data acquisition and local lightweight task processing. The cloud-edge-device computing power scheduling and task offloading management system includes: The resource status awareness module is the foundation for the system's proactive scheduling and offloading decisions. Deployed at various levels, including the cloud center, edge nodes, and terminal devices, its core function is to collect and preprocess the real-time resource and network status of the entire system to obtain a standardized dataset. Specifically, it collects multi-dimensional status data from the cloud, edge nodes, and terminal devices, such as CPU utilization, memory usage, network latency, bandwidth consumption, and task queue length. It also performs preprocessing operations on the collected data, including missing value imputation, outlier handling, and data normalization, to eliminate data interference.

[0050] The state prediction module, based on a hybrid neural network prediction model of long short-term memory network-gated recurrent unit, predicts resource load and network state in cloud-edge-device environments to obtain predicted values ​​x'. This provides accurate and reliable upfront data support for subsequent task offloading decisions and computing power scheduling, addressing the problems of low prediction accuracy and poor adaptability in existing technologies.

[0051] The model selection module is used to train the model based on the standardized dataset and predicted values, obtain the current optimal model parameters, and construct the current optimal model based on the current optimal model parameters.

[0052] The multi-objective optimization task offloading decision module, deployed in a cloud center, plays a crucial role in predicting four objective evaluation items—latency, energy consumption, cost, and reliability—based on the current optimal model. It then constructs a multi-objective function vector using a non-dominated sorting algorithm to obtain the optimal decision scheme, thereby achieving global optimization of task offloading decisions. Specifically, it constructs a multi-objective optimization model for task offloading, simultaneously considering four optimization objectives: total task latency, total system energy consumption, computation and transmission costs, and task execution failure rate, to solve the multi-objective optimization problem and obtain a uniformly distributed Pareto optimal solution set. From this Pareto optimal solution set, it selects the optimal task offloading scheme that best meets the current business priority, clearly defining the execution node (terminal, edge, or cloud) for each task, achieving precise matching of tasks and resources, and overcoming the shortcomings of existing technologies such as lagging decision-making and singular optimization objectives.

[0053] The hierarchical dynamic computing power scheduling module is deployed across three levels: cloud center, edge nodes, and terminal devices. Its core function is to execute a three-tiered scheduling mechanism of "global scheduling + regional scheduling + local scheduling" based on the optimal decision scheme output by the multi-objective optimization task offloading decision module, achieving dynamic allocation and load balancing of computing resources across the entire system. Specifically, the cloud center is responsible for large-scale cross-regional task scheduling, resource reservation, and long-term planning, optimizing global resource utilization based on long-term prediction results; edge nodes are responsible for real-time task scheduling and load balancing within their respective regions, dynamically adjusting task allocation based on short-term prediction results to meet real-time requirements; terminal devices are responsible for executing local lightweight tasks and making simple scheduling decisions, avoiding latency losses caused by data transmission; when resources at a certain level are insufficient or overloaded, it automatically implements cross-level resource support and task migration, ensuring efficient utilization of system computing resources while meeting the real-time requirements of different tasks.

[0054] The task lifecycle management and fault tolerance module, deployed on edge nodes and the cloud center, is designed to monitor and manage all tasks throughout their entire lifecycle, improving system reliability and task execution success rate. Specific functions include: classifying and grading tasks based on priority, latency requirements, computational complexity, etc., into real-time, near-real-time, and non-real-time tasks; monitoring the entire lifecycle of tasks from submission, waiting, execution to completion, and promptly detecting task execution anomalies (node ​​failure, network interruption, excessive load); triggering a task migration mechanism to transfer abnormal tasks to normal nodes and resuming task execution based on a checkpoint mechanism to avoid redundant calculations; and employing a multi-replica execution strategy for critical tasks to ensure at least one replica completes successfully, increasing the system's task execution success rate to over 99.9% and addressing the existing system's weak fault tolerance and low reliability.

[0055] The privacy-preserving data processing module is deployed across all levels of the system. Its core function is to protect the privacy of sensitive data and meet data security regulations without impacting system performance. Specific functions include: performing local data preprocessing and feature extraction on terminal devices and edge nodes to ensure sensitive data does not leave the local machine, reducing the risk of data leakage at the source; employing federated learning technology, allowing each node to train a model based on local data, uploading only the model parameters to the cloud for aggregation and updates; adding Gaussian noise to the uploaded model parameters using differential privacy technology to prevent the inference of original sensitive data from model parameters; and introducing a blockchain-based trusted computing unit to record the task execution and model update processes, ensuring all operations are traceable and tamper-proof, further enhancing the security and reliability of data privacy protection.

[0056] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0057] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cloud-edge-device computing power scheduling and task offloading management method, characterized in that, The cloud-edge-device computing power scheduling and task offloading management method includes: S1. Collect resource and network status data in real time at a preset frequency, and preprocess the collected data to obtain a standardized dataset; S2. A prediction model based on a hybrid neural network of long short-term memory network and gated recurrent unit is used to predict resource load and network status in the cloud-edge-device environment. S3. Train the model based on the standardized dataset and predicted values ​​to obtain the current optimal model parameters; S4. Based on the current optimal model, predictions are made using four objective evaluation items: delay, energy consumption, cost, and reliability. A multi-objective function vector is constructed using a non-dominated sorting algorithm, and then the optimal decision scheme is obtained. S5. Based on the optimal decision-making scheme, complete the three-level scheduling mechanism of global scheduling, regional scheduling, and local scheduling to perform computing power scheduling and task allocation; S6. Monitor and manage the entire lifecycle of tasks.

2. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, The collected data includes: CPU utilization, memory utilization, GPU utilization, disk I / O speed, disk utilization, task queue length, number of tasks being executed, uplink bandwidth, downlink bandwidth, network latency, network jitter and / or packet loss rate.

3. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, The prediction model includes: an input layer, a data normalization layer, a long short-term memory network layer, a random deactivation layer, a two-layer gated recurrent unit layer, a multi-head self-attention layer, a two-layer fully connected layer, and an output layer.

4. The cloud-edge-device computing power scheduling and task offloading management method according to claim 3, characterized in that, Input layer: Receives three-dimensional time series data, with dimensions including batch size, historical time window size, and number of features; Data normalization layer: Uses a min-max normalization method to linearly map all feature values ​​to the [0,1] interval; Long Short-Term Memory (LSTM) network layer: Contains 128 hidden units, effectively capturing long-term dependencies in the time series through three gating structures: input gate, forget gate, and output gate. Each gate weights the input information with the hidden state of the previous time step and outputs the result through a non-linear activation function; Random deactivation layer: Randomly discards 20% of the neuron outputs during training; Two-layer gated recurrent unit. Layers: Each layer contains 64 hidden units, merging the input gate and forget gate into an update gate, while also introducing a reset gate; Multi-head self-attention layer: Employing a 4-head self-attention mechanism, it can automatically learn the importance weights of different historical time steps for future predictions, focusing on time steps that have a significant impact on the prediction results; Two-layer fully connected layer: The first layer contains 64 neurons, and the second layer contains 32 neurons, both using linear rectified functions as activation functions, responsible for mapping the high-dimensional features output by the attention mechanism to a low-dimensional space; Output layer: The output dimension is the batch size and prediction step size, using a linear activation function to directly output the predicted value.

5. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, The current optimal model parameters are obtained through: time series construction, model training, mixed prediction, and online updates.

6. The cloud-edge-device computing power scheduling and task offloading management method according to claim 5, characterized in that, Time Series Construction: A sliding window method is used to convert a one-dimensional time series into a two-dimensional dataset suitable for supervised learning; Model Training: An adaptive moment estimation optimizer is used, with an initial learning rate set to 0.001, and mean squared error is used as the loss function; Hybrid Prediction: The advantages of direct prediction and recursive prediction are combined. For short-term prediction, a direct prediction method is used, i.e., training a model to directly output the prediction results of multiple time steps; for long-term prediction, a recursive prediction method is used, i.e., the prediction result of the previous step is used as the input of the next prediction; Online Update: A sliding window of size N is maintained to store historical data from the most recent N time steps; whenever new data is collected, it is added to the sliding window, while the oldest data in the window is removed; every M time steps, the model is fine-tuned using the data in the sliding window, with a small learning rate and only the parameters of the last few layers of the model are updated.

7. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, The construction of a multi-objective function vector using a non-dominated sorting algorithm specifically includes the following steps: Step 1: Initialize the population; Step 2: Calculate the objective function; Step 3: Non-dominated sorting; Step 4: Adaptive adjustment of reference points; Step 5: Association operation and niching selection; Step 6: Adaptive crossover and mutation operation; Step 7: Population merging and iteration; Step 8: Selection of the optimal decision scheme.

8. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, Global scheduling is executed by the cloud center, responsible for large-scale task scheduling and resource allocation across regions. It performs resource reservation and load balancing based on long-term prediction results, optimizing the resource utilization and energy consumption of the entire system. Regional scheduling is executed by edge nodes, responsible for real-time task scheduling and load balancing within the region. It dynamically adjusts task allocation based on short-term prediction results to meet the low latency requirements of real-time applications. Local scheduling is executed by terminal devices, responsible for the execution of local tasks and simple decision-making. For tasks with small computational load and extremely high real-time requirements, they are processed directly on the terminal device. When resources are insufficient at a certain level, it automatically requests resource support from the next higher level; when resources are excessive at a certain level, it proactively distributes tasks to the next lower level.

9. The cloud-edge-device computing power scheduling and task offloading management method according to claim 1, characterized in that, The full lifecycle monitoring and management of tasks includes: task classification and grading, task status monitoring, task migration mechanism, checkpoint-based task recovery, and multi-copy execution strategy.

10. A cloud-edge-device computing power scheduling and task offloading management system, wherein the cloud-edge-device computing power scheduling and task offloading management system comprises: The resource status awareness module is used to collect and preprocess the real-time data of the system's resources and network status to obtain a standardized dataset. The state prediction module, based on a hybrid neural network prediction model of long short-term memory network-gated recurrent unit, predicts resource load and network state in cloud-edge-device environments to obtain predicted values. The model optimization module is used to train the model based on the standardized dataset and predicted values, obtain the current optimal model parameters, and construct the current optimal model based on the current optimal model parameters. The multi-objective optimization task offloading decision module is used to make predictions based on the current optimal model using four objective evaluation items: delay, energy consumption, cost, and reliability. It also uses a non-dominated sorting algorithm to construct a multi-objective function vector and then obtains the optimal decision scheme. The hierarchical dynamic computing power scheduling module is used to execute a three-level scheduling mechanism of global scheduling + regional scheduling + local scheduling based on the optimal decision scheme output by the multi-objective optimization task offloading decision module, so as to realize the dynamic allocation and load balancing of computing power resources of the entire system. The task lifecycle management and fault tolerance module is used for full-process monitoring and management of all tasks; The privacy-protected data processing module is used to protect the privacy of sensitive data without affecting system performance.

Images