A method and system for task scheduling optimization of an edge system based on reinforcement learning

Abstract

The present application relates to the technical field of edge task scheduling, in particular to a task scheduling optimization method and system of an edge system based on reinforcement learning. The method comprises expressing and learning a task node state data representation vector and a heterogeneous graph by using a joint graph attention network model to generate task node embedding features; time embedding and trend modeling of a task node offloading frequency based on an offloading intention encoding mechanism to generate offloading intention features through an attention network; generating a task offloading probability through a prediction layer based on the offloading intention features, the task node embedding features and the offloading frequency change rate, and calculating the total power required for a task node to complete a computing task; calculating a task node carbon emission threshold based on the proportion of the total power required for the task node to complete the computing task; training a reward function of a carbon emission driven power self-consistent optimization algorithm through reinforcement learning; and generating a task scheduling scheme.

Description

Technical Field

[0001] This invention relates to the field of edge task scheduling technology, and in particular to a task scheduling optimization method and system for edge systems based on reinforcement learning. Background Technology

[0002] With the deep integration of edge computing and green energy technologies, edge systems are evolving towards high-density computing power deployment and low-carbon energy supply. Task scheduling optimization, as a cross-domain collaborative technology, aims to break the independent decision-making model of computing resource scheduling and energy management. Through multi-dimensional coupling optimization of computing power, electricity, and carbon emissions, it achieves a balance between task execution efficiency and green energy consumption at edge nodes. Currently, computing task nodes (hereinafter referred to as task nodes) in edge systems face uncertain task offloading demands and dynamic changes in power supply and demand. Against this backdrop, achieving collaborative control of task scheduling and power supply has become a key issue in building sustainable intelligent edge systems.

[0003] The existing task scheduling optimization methods currently have the following problems: (1) Insufficient modeling of heterogeneous resource relationships. Traditional methods often ignore the complex semantic relationships between the computing power, electricity, and task unloading behavior of task nodes, and fail to fully characterize the collaboration and resource accessibility features between nodes using graph structures, which affects the quality of scheduling strategies; (2) Decoupling modeling of tasks and energy scheduling. Most current methods separate the computation task scheduling from the power allocation, lack cross-domain feature fusion and decision linkage mechanisms, resulting in a mismatch between task unloading strategies and actual power supply capacity; (3) Decoupling of scheduling strategies from carbon emission constraints. Some existing task scheduling optimization strategies ignore carbon emission control targets and cannot take into account both task completion rate and total carbon emission control, resulting in poor applicability in actual low-carbon deployment.

[0004] Therefore, there is an urgent need for a task scheduling optimization method for edge systems based on reinforcement learning. This method can be developed by constructing a heterogeneous graph that integrates computing power, electricity, and offloading behavior, utilizing a joint graph attention network, introducing an offloading intention encoding module, modeling the task offloading probability using the joint offloading frequency change rate, and combining a carbon emission-driven reinforcement learning strategy to achieve closed-loop self-consistent optimization of computing task scheduling and green energy allocation. This will improve the overall operating efficiency and intelligent control level of edge systems under low-carbon goals. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a task scheduling optimization method and system for edge systems based on reinforcement learning.

[0006] Firstly, the present invention provides a task scheduling optimization method for edge systems based on reinforcement learning, which adopts the following technical solution:

[0007] A task scheduling optimization method for edge systems based on reinforcement learning includes:

[0008] Acquire task node status data and various semantic edge types between nodes in the edge system, and complete the construction of heterogeneous graphs;

[0009] A joint graph attention network model is used to learn the representation vectors of task node state data and heterogeneous graphs to generate task node embedding features.

[0010] Based on the unloading intent encoding mechanism, the unloading frequency of task nodes is embedded in time and trend modeled, and unloading intent features are generated through an attention network.

[0011] Based on the unloading intention features, task node embedding features, and unloading frequency change rate, the prediction layer generates the task unloading probability and calculates the total electrical energy required for the task node to complete the computation task.

[0012] Calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete a computation task.

[0013] The reward function of a carbon emission-driven self-consistent power optimization algorithm is trained using reinforcement learning.

[0014] Generate a task scheduling scheme.

[0015] Furthermore, the acquisition of task node status data and various semantic edge types between nodes in the edge system, and the completion of heterogeneous graph construction, includes collecting task node data from the edge system. During the scheduling period The state data within the edge system, assuming the edge system is in the scheduling cycle The set of task nodes inside is , Indicates the task node index. Indicates the first Task nodes This indicates the total number of task nodes. Task node status data includes: available computing power. Unit: FLOPS, representing task nodes. During the scheduling period Available computing power within the system is obtained through the edge system's computing resource management module; remaining power... Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining power within the storage is obtained through the battery management module of the edge system; remaining energy storage capacity Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining local energy storage capacity is obtained through the edge system's battery management module; task offloading frequency. Unit: number of times / cycle, representing task nodes. During the scheduling period The task offloading frequency within the system is obtained through the edge system's scheduling log module; green electricity availability. Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The predicted availability of green electricity is dynamically estimated based on photovoltaic or wind power prediction models. Based on the scheduling cycle of task nodes in the edge system By combining the state data within the node with various semantic edge types such as inter-node communication connections, energy scheduling, task migration, and carbon collaborative constraints, heterogeneous graph construction can be achieved.

[0016] Furthermore, the process of using a joint graph attention network model to learn the representation of task node state data vectors and heterogeneous graphs to generate task node embedding features includes obtaining the Joint-GAT joint graph attention network by improving the graph neural network structure. The Joint-GAT model will manage task nodes during the scheduling cycle. The state data within the task node is constructed into a task node state data set. Min-max normalization is performed on each dimension of the state data in the task node state data set to obtain a dimensionless task node state data representation vector. Then, the task node state data representation vector is mapped through a non-linear activation function to generate the initial node embedding features of the task node.

[0017] Furthermore, the step of using a joint graph attention network model to learn the task node state data representation vector and the heterogeneous graph to generate task node embedding features also includes, during the training of the joint graph attention network model, denoting the first... Task nodes under the edge planting type The set of adjacent nodes is , No. Each pair of task nodes in the layer and The embedding features are and The attention weights for updating the current task node's embedding features using the existing attention mechanism are calculated and expressed as follows:

[0018] ,

[0019] in, Indicates the first Each pair of task nodes under the seed edge type and Attention weights, including the embedding features of adjacent task nodes. Embed features for the current task node The impact weight of the update; It is a non-linear activation function; For the first Transpose of the attention parameter vector under the seeded edge type; For the first The linear transformation matrix corresponding to the seed edge type is used to embed the features of the task node. and Mapped to the In the feature space of seed edge type; This represents a vector concatenation operation; normalization is performed using the softmax function to obtain normalized attention weights, expressed as:

[0020] ,

[0021] in, For the first Each pair of task nodes under the seed edge type and Normalized attention weights; Indicates the first Each pair of task nodes under the seed edge type and Attention weights; This indicates that all adjacent nodes are traversed.

[0022] Furthermore, the step of using a joint graph attention network model to learn the representation of task node state data vectors and heterogeneous graphs to generate task node embedding features also includes, after calculating the normalized attention weights for each type of semantic edge, weighted aggregation of all neighbor node embedding features under their respective normalized attention weights, as shown below:

[0023] ,

[0024] in, Represents task node In the Layer-updated embedded features; It is a non-linear activation function; For all semantic edge types A set; For the first Each pair of task nodes under the seed edge type and Normalized attention weights; For the first The linear transformation matrix corresponding to the edge type; For task nodes In the Layer embedding features; after several layers of attention aggregation in the joint graph attention network model, task nodes are generated. During the scheduling period Embedding features , is represented as:

[0025] ,

[0026] in, For task nodes During the scheduling period Embedded features; For joint graph attention network models; A vector representing the state features of the task node; This is a heterogeneous diagram.

[0027] Furthermore, the unloading intention encoding mechanism performs temporal embedding and trend modeling of the task node unloading frequency, and generates unloading intention features through an attention network. This includes, after obtaining the task node embedding vector, introducing an unloading intention encoding mechanism to enhance the predictive ability of the task node's future computational task scheduling behavior. First, a length of [missing information] is constructed... The task node's historical unloading behavior sequence will be used to schedule the task node during the scheduling period. The task unloading frequency within the time embedding module is input, and a positional encoding mechanism is used to perform time-aware modeling of the task unloading frequency through a time embedding function, obtaining the task node's time within the scheduling cycle. The time-embedded features within the time frame are used to form the task node's time within the scheduling cycle. The time-embedded sequence is input into an existing Transformer-based attention network to extract deep semantic features of the task node's historical unloading behavior sequence, and finally outputs the task node's behavior within the scheduling cycle. Unload intent characteristics within.

[0028] Furthermore, the step of generating the task unloading probability through the prediction layer based on unloading intention features, task node embedding features, and unloading frequency change rate, and calculating the total electrical energy required for the task node to complete the computation task, includes inputting the task node embedding features, unloading intention features, and unloading frequency change rate into the prediction layer to generate a task unloading probability prediction result:

[0029] ,

[0030] in, Represents task node During the scheduling period Task uninstallation probability; This represents the Sigmoid activation function; This represents the transpose of the output layer weight vector; Represents the hyperbolic tangent function; , , These represent the weight matrices for the task node embedding vector, the unloading intention vector, and the rate of change of the unloading frequency, respectively. Represents task node During the scheduling period The embedding vector; Represents task node During the scheduling period The unloading intent vector; , Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period and The rate of change in the frequency of task unloading between; This represents the bias term; it is calculated based on the task node's task offloading probability and unit task energy consumption parameters during the scheduling cycle. Total electrical energy required to complete the computation task The node unit task energy consumption parameter is obtained by constructing an empirical model based on statistical information of power consumption and task duration during the historical task execution of the node.

[0031] Furthermore, the calculation of the carbon emission threshold based on the proportion of total electrical energy required for a task node to complete a computational task includes introducing a carbon emission-driven power self-consistent optimization algorithm, based on the total electrical energy required by the task node in task scheduling. The proportion of carbon emission thresholds Distribute tasks proportionally to each task node to form task nodes. During the scheduling period Carbon emission threshold within ,in The carbon emission threshold was obtained through expert experience.

[0032] Furthermore, the reward function for training the carbon emission-driven self-consistent power optimization algorithm through reinforcement learning includes constructing a reward function by jointly considering the proportion of green electricity use, carbon emissions, task delay, and task offloading ratio indicators. This guides the reinforcement learning scheduling strategy to generate decision schemes that link tasks and energy. The reward function for training the carbon emission-driven self-consistent power optimization algorithm through the reinforcement learning framework is expressed as follows:

[0033] ,

[0034] in, Represents task node During the scheduling period The reward function value within; , , , These are the weighting coefficients for green electricity priority, carbon emission penalty, task delay penalty, and task unloading penalty, respectively. Represents task node During the scheduling period The proportion of green electricity used within the country; Represents task node During the scheduling period The predicted amount of green electricity available; Represents task node During the scheduling period The total electrical energy required to complete the computation task; Represents task node During the scheduling period Electricity purchased from external power grids and its carbon emission factor Multiply them to get the carbon emissions; Represents task node During the scheduling period The upper limit of the internally available energy storage power supply; Represents task node During the scheduling period Remaining energy storage capacity within; Represents task node During the scheduling period The carbon emission threshold within; Represents task node During the scheduling period The probability of task uninstallation. The maximum total electrical energy required for a task node to complete its computational task can be obtained from the historical data of the task node.

[0035] Secondly, a task scheduling optimization system for edge systems based on reinforcement learning includes:

[0036] The data acquisition module is configured to acquire task node status data and various semantic edge types between nodes in the edge system, and to complete the construction of heterogeneous graphs.

[0037] The embedding feature module is configured to use a joint graph attention network model to learn the representation of the task node state data vector and the heterogeneous graph to generate task node embedding features.

[0038] The intent vector module is configured to embed and model the unloading frequency of task nodes over time based on the unloading intent encoding mechanism, and generate unloading intent features through an attention network.

[0039] The unloading probability module is configured to generate the task unloading probability through the prediction layer based on the unloading intention feature, task node embedding feature and unloading frequency change rate, and calculate the total power required for the task node to complete the computing task.

[0040] The carbon emission threshold module is configured to calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete the computation task.

[0041] The reward module is configured to train the reward function of the carbon emission-driven self-consistent power optimization algorithm through reinforcement learning;

[0042] The scheduling module is configured to generate task scheduling schemes.

[0043] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the task scheduling optimization method for an edge system based on reinforcement learning.

[0044] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide a task scheduling optimization method for an edge system based on reinforcement learning.

[0045] In summary, the present invention has the following beneficial technical effects:

[0046] 1. Based on multi-source information such as computing power, power and offloading behavior of task nodes in the edge system, this invention constructs a heterogeneous graph containing various semantic edge types such as communication connection, energy scheduling, task migration and carbon cooperative constraints. It uses a joint graph attention network to capture complex dependencies between task nodes, breaking through the limitations of traditional single-dimensional modeling.

[0047] 2. The task unloading probability prediction model constructed in this invention effectively improves the node task unloading probability prediction capability by integrating the task node embedding features generated by the joint graph attention network, the unloading intention features generated by the unloading intention encoding mechanism, and the unloading frequency change rate.

[0048] 3. The carbon emission-driven self-consistent power optimization algorithm constructed in this invention comprehensively considers green electricity priority, carbon emission penalty, task delay penalty, and task unloading penalty, and dynamically adjusts the task scheduling strategy of task nodes. It can generate a power supply scheme that prioritizes the use of green electricity and minimizes grid power purchase and carbon emissions, while ensuring timely task completion. This scheme includes a combination of available green electricity, remaining energy storage capacity, and external grid power purchase, thus achieving closed-loop control of computing power, electricity, and carbon emissions. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of a task scheduling optimization method for an edge system based on reinforcement learning, according to Embodiment 1 of the present invention. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the accompanying drawings.

[0051] Example 1

[0052] Reference Figure 1 This embodiment of a task scheduling optimization method for an edge system based on reinforcement learning includes:

[0053] Acquire task node status data and various semantic edge types between nodes in the edge system, and complete the construction of heterogeneous graphs;

[0054] A joint graph attention network model is used to learn the representation vectors of task node state data and heterogeneous graphs to generate task node embedding features.

[0055] Based on the unloading intent encoding mechanism, the unloading frequency of task nodes is embedded in time and trend modeled, and unloading intent features are generated through an attention network.

[0056] Based on the unloading intention features, task node embedding features, and unloading frequency change rate, the prediction layer generates the task unloading probability and calculates the total electrical energy required for the task node to complete the computation task.

[0057] Calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete a computation task.

[0058] The reward function of a carbon emission-driven self-consistent power optimization algorithm is trained using reinforcement learning.

[0059] Generate a task scheduling scheme.

[0060] Specifically:

[0061] S1. Data Acquisition and Heterogeneous Graph Construction

[0062] Task nodes in the edge acquisition system During the scheduling period The state data within the edge system, assuming the edge system is in the scheduling cycle The set of task nodes inside is , Indicates the task node index. Indicates the first Task nodes This indicates the total number of task nodes. Task node status data includes: available computing power. Unit: FLOPS, representing task nodes. During the scheduling period Available computing power within the system is obtained through the edge system's computing resource management module; remaining power... Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining power within the storage is obtained through the battery management module of the edge system; remaining energy storage capacity Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining local energy storage capacity is obtained through the edge system's battery management module; task offloading frequency. Unit: number of times / cycle, representing task nodes. During the scheduling period The task offloading frequency within the system is obtained through the edge system's scheduling log module; green electricity availability. Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The available green electricity is predicted internally and dynamically estimated based on photovoltaic or wind power prediction models.

[0063] Based on task nodes in the edge system During the scheduling period By combining internal state data with various semantic edge types such as inter-node communication connections, energy scheduling, task migration, and carbon coordination constraints, a heterogeneous graph can be realized. Build.

[0064] S2. Task Node Embedding Features

[0065] To fully model the state features of multi-source task nodes and the types of multiple semantic edges in heterogeneous graphs, an improved graph neural network structure, distinct from the traditional Graph Attention Network (GAT), is designed—the Joint Graph Attention Network (Joint-GAT) model. The Joint-GAT model represents task node state data as vectors. and heterogeneous graphs As input, initial embedded features are first generated through a feedforward layer. Then, it interacts and merges with neighboring nodes under a multi-semantic attention mechanism, and finally outputs the task node. During the scheduling period Embedded features The specific process is as follows:

[0066] First, set the task nodes During the scheduling period The internal state data is constructed into a task node state data set. ,right Each dimension of the state data is normalized using a min-max method to obtain a dimensionless vector representing the task node state data. Then on Task nodes are generated through mapping using a non-linear activation function. Initial node embedding features , It is a non-linear activation function. This is the weight matrix. For bias vectors, , All were obtained through training a joint graph attention network model;

[0067] Heterogeneous graph There are various semantic edge types, such as communication connections, energy scheduling, task migration, and carbon cooperation constraints, representing different semantic interaction relationships between task nodes. In the training of the joint graph attention network model, let the first edge be... Task nodes under the edge planting type The set of adjacent nodes is , No. Each pair of task nodes in the layer and The embedding features are and To distinguish feature interactions under different edge types, the existing attention mechanism is used to calculate the attention weights of the embedded features of adjacent task nodes on the update of the embedded features of the current task node, as shown in the following formula:

[0068] ,

[0069] in, Indicates the first Each pair of task nodes under the seed edge type and Attention weights, i.e., the embedding features of adjacent task nodes. Embed features for the current task node The impact weight of the update; It is a non-linear activation function used to enhance feature representation capabilities; For the first The transpose of the attention parameter vector under the edge type can be obtained by training a joint graph attention network model; For the first The linear transformation matrix corresponding to the seed edge type is used to embed the features of the task node. and Mapped to the In the feature space of edge types, it can be obtained by training a joint graph attention network model; This represents a vector concatenation operation;

[0070] Next, the attention weights are normalized using the softmax function, as shown in the following formula:

[0071] ,

[0072] in, For the first Each pair of task nodes under the seed edge type and Normalized attention weights; Indicates the first Each pair of task nodes under the seed edge type and Attention weights; This indicates that all adjacent nodes have been traversed.

[0073] After calculating the normalized attention weights for each type of semantic edge, the embedded features of all neighbor nodes will be weighted and aggregated under their respective normalized attention weights, as shown in the following formula:

[0074] ,

[0075] in, Represents task node In the Layer-updated embedded features; It is a non-linear activation function used to enhance feature representation capabilities; For all semantic edge types A set; For the first Each pair of task nodes under the seed edge type and Normalized attention weights; For the first The linear transformation matrix corresponding to the edge type can be obtained by training a joint graph attention network model. For task nodes In the Layer embedding features;

[0076] After several layers of attention aggregation in the joint graph attention network model, task nodes are generated. During the scheduling period Embedding features The formula is as follows:

[0077] ,

[0078] in, For task nodes During the scheduling period Embedded features; For joint graph attention network models; A vector representing the state data of the task node; It is a heterogeneous graph;

[0079] S3. Unload Intent Characteristics

[0080] In obtaining task node embedding features Subsequently, to enhance the predictive ability of future task scheduling behavior of task nodes, an unloading intent encoding mechanism is further introduced. First, a code with a length of [length missing] is constructed. Task node history unloading behavior sequence ,in Represents task node During the scheduling period Internal task unloading frequency, Indicates the periodic window length. ;Will The input time embedding module uses a positional encoding mechanism to determine the task offloading frequency through a time embedding function. Perform time-aware modeling to obtain task nodes. During the scheduling period The temporal embedding features within the time frame are defined by the following formula:

[0081] ,

[0082] in, For task nodes During the scheduling period Temporal embedding features within; For time embedding functions; Represents task node During the scheduling period Internal task unloading frequency; scheduling period Time location encoding;

[0083] Form task nodes During the scheduling period Time-embedded sequences within

[0084] The above time-embedded sequence Input an existing Transformer-based attention network to extract deep semantic features of the historical unloading behavior sequence of task nodes, and finally output the task nodes. During the scheduling period Uninstallation intent characteristics The formula is as follows:

[0085]

[0086] S4. Task unloading probability, total electrical energy required to complete the computation task.

[0087] Embedding task nodes with features Uninstallation intent characteristics Rate of change with unloading frequency The input is fed into the prediction layer to generate the task unloading probability prediction result:

[0088] ,

[0089] in, Represents task node During the scheduling period The probability of task unloading, with a value range of [value missing]. This is used to guide the generation of task scheduling strategies; This represents the Sigmoid activation function, used to map predicted values ​​to probabilistic forms; This represents the transpose of the output layer weight vector, which is learned during training using the gradient descent algorithm. This represents the hyperbolic tangent function, used to perform nonlinear transformations on the fused feature vectors to enhance the model's expressive power. , , These are weight matrices representing the task node embedding features, unloading intention features, and unloading frequency change rate, respectively. They have the same dimension and are all learned through the gradient descent algorithm during training. Represents task node During the scheduling period Embedded features; Represents task node During the scheduling period The uninstallation intent characteristics; , Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period and The rate of change in the frequency of task unloading between; This represents the bias term, which is learned during training using the gradient descent algorithm.

[0090] The above formula effectively improves the ability to predict the probability of task unloading by integrating the task node embedding vector generated by the joint graph attention network, the unloading intention vector generated by the unloading intention encoding module, and the unloading frequency change rate.

[0091] Ultimately, based on the task node's task unloading probability Energy consumption parameters per unit task The calculations show that each task node has a scheduling period of [time period]. Total electrical energy required to complete the computation task Unit: kilowatt-hour (kWh), where the node unit task energy consumption parameter An empirical model can be built based on statistical information about power consumption and task duration during the historical execution of tasks at a node. The formula is as follows:

[0092] ,

[0093] in, Energy consumption per unit task, unit: kilowatt-hour (kWh); Indicates the length of the periodic window; For task nodes During the scheduling period The average computing power within the edge system, in kilowatts (kW), can be obtained through the edge system's computing resource management module. For task nodes During the scheduling period The average task computation time within the edge system, in hours (h), can be obtained through the edge system's computing resource management module.

[0094] S5. Carbon emission threshold

[0095] Introducing a carbon emission-driven self-consistent power optimization algorithm, based on task nodes. Total electrical energy required in task scheduling The proportion of carbon emission thresholds Distribute tasks proportionally to each task node to form task nodes. During the scheduling period Carbon emission threshold within ,in The carbon emission threshold, in kgCO2, was obtained through expert experience.

[0096] S6. A carbon emission-driven self-consistent power optimization algorithm based on reinforcement learning is developed by training the reward function of the algorithm using a reinforcement learning framework. This reward function, by combining indicators such as the proportion of green electricity use, carbon emissions, task delay, and task offloading ratio, guides the reinforcement learning scheduling strategy to generate a decision scheme that links "tasks and energy," reflecting the optimization goal of integrating computing power and low-carbon power. The reward function formula is as follows:

[0097] ,

[0098] in, Represents task node During the scheduling period The reward function value within the range is used to evaluate the effectiveness of power dispatching strategies; , , , The weight coefficients for green electricity priority, carbon emission penalty, task delay penalty, and task unloading penalty are obtained through reinforcement learning training. Represents task node During the scheduling period The percentage of green electricity used within the premises is within the range of [value range missing]. Unitless; the closer to 1, the higher the priority for using green electricity. Represents task node During the scheduling period Forecasted green electricity availability, in kilowatt-hours (kWh). Represents task node During the scheduling period Total electrical energy required to complete the computation task, in kilowatt-hours (kWh). Represents task node During the scheduling period Electricity purchased from the external power grid, unit: kilowatt-hour (kWh). This indicates that the available green electricity and the available energy storage have met the requirements for completing the task, and there is no need to purchase electricity from the external grid. At this time, carbon emissions are 0. This indicates that some electricity needs to be purchased from external grids, which is related to the carbon emission factor of electricity. Multiplying them together yields the carbon emissions, the electricity carbon emission factor. Unit: kgCO2 / kWh, which can be obtained from relevant national standards; Represents task node During the scheduling period The upper limit of internally available energy storage power supply, in kilowatt-hours (kWh). Represents task node During the scheduling period Remaining energy storage capacity within the unit: kilowatt-hours (kWh); Represents task node During the scheduling period Carbon emission threshold within the region, unit: kgCO2; Represents task node During the scheduling period The probability of task unloading, without units; The maximum total electrical energy required for a task node to complete its computational task can be obtained from the historical data of the task node, in kilowatt-hours (kWh).

[0099] S7. Generate a task scheduling scheme

[0100] To achieve optimal task scheduling and power supply strategies, a reward function value is selected. The largest energy supply option is taken as the final task scheduling optimization result, including the availability of green electricity. Remaining energy storage capacity Electricity purchased from external power grids .

[0101] By introducing a carbon emission-driven self-consistent power optimization algorithm and combining it with a reinforcement learning framework, the task scheduling strategy of task nodes can be dynamically adjusted to prioritize the use of green electricity and minimize grid power purchases and carbon emissions, while ensuring that tasks are completed on time, thereby improving the system's energy efficiency and environmental sustainability.

[0102] Example 2

[0103] This embodiment provides a task scheduling optimization system for edge systems based on reinforcement learning, including:

[0104] The data acquisition module is configured to acquire task node status data and various semantic edge types between nodes in the edge system, and to complete the construction of heterogeneous graphs.

[0105] The embedding feature module is configured to use a joint graph attention network model to learn the representation of the task node state data vector and the heterogeneous graph to generate task node embedding features.

[0106] The intent vector module is configured to embed and model the unloading frequency of task nodes over time based on the unloading intent encoding mechanism, and generate unloading intent features through an attention network.

[0107] The unloading probability module is configured to generate the task unloading probability through the prediction layer based on the unloading intention feature, task node embedding feature and unloading frequency change rate, and calculate the total power required for the task node to complete the computing task.

[0108] The carbon emission threshold module is configured to calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete the computation task.

[0109] The reward module is configured to train the reward function of the carbon emission-driven self-consistent power optimization algorithm through reinforcement learning;

[0110] The scheduling module is configured to generate task scheduling schemes.

[0111] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device, the aforementioned task scheduling optimization method for an edge system based on reinforcement learning.

[0112] A terminal device includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions adapted for loading and execution by the processor of the task scheduling optimization method for a reinforcement learning-based edge system.

[0113] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A task scheduling optimization method for edge systems based on reinforcement learning, characterized in that, include: Acquire task node status data and various semantic edge types between nodes in the edge system, and complete the construction of heterogeneous graphs; A joint graph attention network model is used to learn the representation vectors of task node state data and heterogeneous graphs to generate task node embedding features. Based on the unloading intent encoding mechanism, the unloading frequency of task nodes is embedded in time and trend modeled, and unloading intent features are generated through an attention network. Based on the unloading intention features, task node embedding features, and unloading frequency change rate, the prediction layer generates the task unloading probability and calculates the total electrical energy required for the task node to complete the computation task. Calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete a computation task. The reward function of a carbon emission-driven self-consistent power optimization algorithm is trained using reinforcement learning. Generate a task scheduling scheme; The process of acquiring task node status data and various semantic edge types between nodes in the edge system, and constructing a heterogeneous graph, includes collecting data on task nodes in the edge system. During the scheduling period The state data within the edge system, assuming the edge system is in the scheduling cycle The set of task nodes inside is , Indicates the task node index. Indicates the first Task nodes Indicates the total number of task nodes; Task node status data includes: available computing power Unit: FLOPS, representing task nodes. During the scheduling period Available computing power within the system is obtained through the edge system's computing resource management module; remaining power... Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining power within the storage is obtained through the battery management module of the edge system; remaining energy storage capacity Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The remaining local energy storage capacity is obtained through the edge system's battery management module; task offloading frequency. Unit: number of times / cycle, representing task nodes. During the scheduling period The task offloading frequency within the system is obtained through the edge system's scheduling log module; green electricity availability. Unit: kilowatt-hour (kWh), representing a task node. During the scheduling period The available green electricity is predicted internally and dynamically estimated based on photovoltaic or wind power prediction models; Based on the scheduling cycle of task nodes in the edge system By combining the state data within the node with various semantic edge types such as inter-node communication connections, energy scheduling, task migration, and carbon collaborative constraints, heterogeneous graph construction can be achieved.

2. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 1, characterized in that, The process involves using a joint graph attention network model to learn the representation vectors of task node state data and heterogeneous graphs, generating task node embedding features. This includes obtaining the Joint Graph Attention Network (Joint-GAT) by improving the graph neural network structure. The Joint-GAT model manages task nodes during the scheduling cycle. The state data within the task node is constructed into a task node state data set. Min-max normalization is performed on each dimension of the state data in the task node state data set to obtain a dimensionless task node state data representation vector. Then, the task node state data representation vector is mapped through a non-linear activation function to generate the initial node embedding features of the task node.

3. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 2, characterized in that, The method of using a joint graph attention network model to learn the representation vector of task node state data and the heterogeneous graph to generate task node embedding features also includes, during the training of the joint graph attention network model, denoting the first... Task nodes under the edge planting type The set of adjacent nodes is Each pair of task nodes in the first layer and The embedding features are and The attention weights for updating the current task node's embedding features using the existing attention mechanism are calculated and expressed as follows: ， in, Indicates the first Each pair of task nodes under the edge type and Attention weights, including the embedding features of adjacent task nodes. Embed features for the current task node The impact weight of the update; It is a non-linear activation function; For the first Transpose of the attention parameter vector under the seeded edge type; For the first The linear transformation matrix corresponding to the seed edge type is used to embed the features of the task node. and Mapped to the In the feature space of seed edge type; This represents a vector concatenation operation; normalization is performed using the softmax function to obtain normalized attention weights, expressed as: ， in, For the first Each pair of task nodes under the edge type and Normalized attention weights; Indicates the first Each pair of task nodes under the edge type and Attention weights; This indicates that all adjacent nodes are traversed.

4. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 3, characterized in that, The method of using a joint graph attention network model to learn the representation of task node state data vectors and heterogeneous graphs to generate task node embedding features also includes, after calculating the normalized attention weights for each type of semantic edge, weighted aggregation of all neighbor node embedding features under their respective normalized attention weights, as shown below: ， in, Represents task node In the Layer-updated embedded features; It is a non-linear activation function; For all semantic edge types A set; For the first Each pair of task nodes under the edge type and Normalized attention weights; For the first The linear transformation matrix corresponding to the edge type; For task nodes In the Layer embedding features; after several layers of attention aggregation in the joint graph attention network model, task nodes are generated. During the scheduling period Embedding features , represented as: ， in, For task nodes During the scheduling period Embedded features; For joint graph attention network models; A vector representing the state data of the task node; This is a heterogeneous diagram.

5. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 4, characterized in that, The unloading intention encoding mechanism performs temporal embedding and trend modeling of task node unloading frequency, and generates unloading intention features through an attention network. This includes, after obtaining the task node embedding vector, introducing an unloading intention encoding mechanism to enhance the predictive ability of future task scheduling behavior of task nodes. First, a length of [missing information] is constructed. The task node's historical unloading behavior sequence will be used to schedule the task node during the scheduling period. The task unloading frequency within the time embedding module is input, and a positional encoding mechanism is used to perform time-aware modeling of the task unloading frequency through a time embedding function, obtaining the task node's time within the scheduling cycle. The time-embedded features within the time frame are used to form the task node's time within the scheduling cycle. The time-embedded sequence is input into an existing Transformer-based attention network to extract deep semantic features of the task node's historical unloading behavior sequence, and finally outputs the task node's behavior within the scheduling cycle. Uninstallation intent characteristics within.

6. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 5, characterized in that, The process involves generating a task unloading probability through a prediction layer based on unloading intention features, task node embedding features, and unloading frequency change rate, and calculating the total electrical energy required for the task node to complete the computation task. This includes inputting the task node embedding features, unloading intention features, and unloading frequency change rate into the prediction layer to generate a task unloading probability prediction result. ， in, Represents task node During the scheduling period Task uninstallation probability; This represents the Sigmoid activation function; This represents the transpose of the output layer weight vector; Represents the hyperbolic tangent function; , , These represent the weight matrices for the task node embedding vector, the unloading intention vector, and the rate of change of the unloading frequency, respectively. Represents task node During the scheduling period The embedding vector; Represents task node During the scheduling period The unloading intent vector; , Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period Internal task unloading frequency, Represents task node During the scheduling period and The rate of change in the frequency of task unloading between; This represents the bias term; it is calculated based on the task node's task offloading probability and unit task energy consumption parameters during the scheduling cycle. Total electrical energy required to complete the computation task The node unit task energy consumption parameter is obtained by constructing an empirical model based on statistical information of power consumption and task duration during the historical task execution of the node.

7. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 6, characterized in that, The calculation of carbon emission thresholds based on the proportion of total power required for task nodes to complete computational tasks includes introducing a carbon emission-driven power self-consistent optimization algorithm, based on the total power required by task nodes in task scheduling. The proportion of carbon emission thresholds Distribute tasks proportionally to each task node to form task nodes. During the scheduling period Carbon emission threshold within ,in The carbon emission threshold was obtained through expert experience.

8. The task scheduling optimization method for an edge system based on reinforcement learning according to claim 7, characterized in that, The reward function of the carbon emission-driven self-consistent power optimization algorithm trained by reinforcement learning includes constructing a reward function by combining indicators such as the proportion of green electricity use, carbon emissions, task delay, and task offloading ratio. This guides the reinforcement learning scheduling strategy to generate decision schemes that link tasks and energy. The reward function of the carbon emission-driven self-consistent power optimization algorithm trained by the reinforcement learning framework is expressed as follows: ， in, Represents task node During the scheduling period The reward function value within; , , , These are the weighting coefficients for green electricity priority, carbon emission penalty, task delay penalty, and task unloading penalty, respectively. Represents task node During the scheduling period The proportion of green electricity used domestically; Represents task node During the scheduling period The predicted amount of green electricity available; Represents task node During the scheduling period The total electrical energy required to complete the computation task; Represents task node During the scheduling period Electricity purchased from external power grids and its carbon emission factor Multiply them to get the carbon emissions; Represents task node During the scheduling period The upper limit of internally available energy storage power supply; Represents task node During the scheduling period Remaining energy storage capacity within; Represents task node During the scheduling period The carbon emission threshold within; Represents task node During the scheduling period The probability of task uninstallation. The maximum total electrical energy required for a task node to complete its computational task can be obtained from the historical data of the task node.

9. A task scheduling optimization system for an edge system based on reinforcement learning, executing the task scheduling optimization method for an edge system based on reinforcement learning as described in claim 1, characterized in that, include: The data acquisition module is configured to acquire task node status data and various semantic edge types between nodes in the edge system, and to complete the construction of heterogeneous graphs. The embedding feature module is configured to use a joint graph attention network model to learn the representation of the task node state data vector and the heterogeneous graph to generate task node embedding features. The intent vector module is configured to embed and model the unloading frequency of task nodes over time based on the unloading intent encoding mechanism, and generate unloading intent features through an attention network. The unloading probability module is configured to generate the task unloading probability through the prediction layer based on the unloading intention feature, task node embedding feature and unloading frequency change rate, and calculate the total power required for the task node to complete the computing task. The carbon emission threshold module is configured to calculate the carbon emission threshold of a task node based on the proportion of total electrical energy required to complete the computation task. The reward module is configured to train the reward function of the carbon emission-driven self-consistent power optimization algorithm through reinforcement learning; The scheduling module is configured to generate task scheduling schemes.

Images