A Multi-Objective Task Scheduling Method and Device in an Edge Computing Environment

Abstract

The present invention relates to a multi-objective task scheduling method and device in an edge computing environment, characterized in that it includes the steps of: constructing a pointer network model for making decisions on the execution order of multiple service requests in an edge server, and making the result sampling strategy of the pointer network model use the average waiting time of the service request as an optimization indicator; integrating the resource utilization and running time of the edge server into an optimization goal as a reward function of reinforcement learning to train the pointer network model. The embodiment of the present invention integrates the waiting time optimization goal into the model decision, thereby achieving the effect of multi-objective optimization, and does not require a large amount of experimental analysis of the weight values, and the model training of the pointer network is more effective. At the same time, the pointer network model based on reinforcement learning does not require long-term iteration like traditional heuristic algorithms, and can make rapid decisions, which meets the requirements of edge environments that are sensitive to delays.

Description

Technical Field

[0001] The present invention relates to the field of edge computing application technology, and in particular to a multi-objective task scheduling method and device in an edge computing environment. Background Art

[0002] Edge computing has become an increasingly popular computing paradigm in recent years. Due to the inherent resource constraints of edge servers, a key issue in edge computing research is how to rationally schedule and utilize service resources in edge environments to meet as many service requests as possible. The multi-objective task scheduling problem in edge environments involves the situation where, given an edge server, multiple users typically request its services simultaneously. When the query rate (QPS) increases to a certain level, the edge server's limited resources often prevent it from running a large number of tasks simultaneously. Consequently, these tasks must be queued after arriving at the edge server.

[0003] Related technologies for multi-objective task scheduling in edge computing environments use a weighted summation approach to optimize multi-objective solutions, making it difficult to achieve the optimal combination of multiple objective weights. Furthermore, setting these weights requires extensive and lengthy experimental verification and relies heavily on the experimenter's expertise. Furthermore, some technologies employ heuristic algorithms, requiring lengthy iterations and failing to meet the latency-sensitive requirements of edge environments. Summary of the Invention

[0004] The embodiments of the present invention provide a multi-objective task scheduling method and device in an edge computing environment to solve the problems existing in the related art.

[0005] An embodiment of the present invention provides a multi-objective task scheduling method in an edge computing environment, characterized in that it includes the steps of:

[0006] Constructing a pointer network model for deciding the execution order of multiple service requests in the edge server, and making the result sampling strategy of the pointer network model use the average waiting time of the service requests as the optimization indicator;

[0007] The resource utilization and running time of the edge server are integrated into an optimization objective as the reward function of reinforcement learning to train the pointer network model.

[0008] In some embodiments, the pointer network model includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to help the Actor network perform the training. The Actor network includes an encoder and a decoder, and the encoder and decoder both include a recurrent neural network composed of multiple long short-term memory networks. The decision on the execution order of multiple service requests in the edge server includes the following steps:

[0009] The queued service request sequence is used as an input sequence and converted into a first intermediate vector and input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder;

[0010] Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by the attention mechanism of the pointer network model for the state of each hidden layer of the decoder;

[0011] Obtaining, based on the second intermediate vector, a probability that the decoder selects each service request in a hidden layer as an output of the layer;

[0012] Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server.

[0013] In some embodiments, each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, including the steps of:

[0014] For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so,

[0015] Among the second intermediate vectors corresponding to the service requests that have not yet been selected, service requests corresponding to a portion of second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

[0016] In some embodiments, each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, including the steps of:

[0017] If the number of service requests that have not been selected exceeds a first preset threshold, probability sampling is performed in the second intermediate vectors corresponding to the service requests that have not been selected, and the output of the current layer is determined according to the sampling result.

[0018] In some embodiments, integrating the resource utilization and runtime of the edge server into an optimization objective as a reward function for reinforcement learning to train the pointer network model includes the following steps:

[0019] based on Determine the reward function reward, where m is the total number of edge servers, T_server j The total time required for edge server j to complete all service requests;

[0020] Training is performed using reinforcement learning with stochastic gradient descent.

[0021] On the other hand, an embodiment of the present invention provides a multi-objective task scheduling device in an edge computing environment, characterized in that it includes:

[0022] A multi-service request decision module constructs a pointer network model for deciding the execution order of multiple service requests in the edge server, and uses the average waiting time of service requests as an optimization indicator in the result sampling strategy of the pointer network model;

[0023] The reinforcement learning module integrates the resource utilization and running time of the edge server into an optimization objective as the reward function of reinforcement learning to train the pointer network model.

[0024] In some embodiments, the pointer network model includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to help the Actor network perform the training, and the Actor network includes an encoder and a decoder, and the encoder and decoder both include a recurrent neural network composed of multiple long short-term memory networks;

[0025] The multi-service request decision module is further configured to:

[0026] The queued service request sequence is used as an input sequence and converted into a first intermediate vector and input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder;

[0027] Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by the attention mechanism of the pointer network model for the state of each hidden layer of the decoder;

[0028] Obtaining, based on the second intermediate vector, a probability that the decoder selects each service request in a hidden layer as an output of the layer;

[0029] Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server.

[0030] In some embodiments, the multi-service request decision module is further configured to:

[0031] For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so,

[0032] Among the second intermediate vectors corresponding to the service requests that have not yet been selected, service requests corresponding to a portion of second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

[0033] In some embodiments, the multi-service request decision module is further configured to:

[0034] If the number of service requests that have not been selected exceeds a first preset threshold, probability sampling is performed in the second intermediate vectors corresponding to the service requests that have not been selected, and the output of the current layer is determined according to the sampling result.

[0035] In some embodiments, the reinforcement learning module is used to:

[0036] based on Determine the reward function reward, where m is the total number of edge servers, T_server j The total time required for edge server j to complete all service requests;

[0037] Training is performed using reinforcement learning with stochastic gradient descent.

[0038] The beneficial effects brought about by the technical solution provided by the present invention include:

[0039] The embodiment of the present invention provides a multi-objective task scheduling method and device in an edge computing environment, which integrates resource utilization and running time into one optimization goal, and proposes a new pointer network sampling strategy to integrate the waiting time optimization goal into the model decision, thereby achieving the effect of multi-objective optimization. It does not require a large amount of experimental analysis on the weight values, and the model training of the pointer network is more effective. At the same time, the pointer network model based on reinforcement learning does not require long iterations like traditional heuristic algorithms, and can make quick decisions, which meets the requirements of edge environments that are sensitive to delays. Offline training before deployment to the edge environment avoids the problem that the untrained model will make poor decisions with a certain probability after direct deployment, resulting in serious fluctuations in the overall service quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A flowchart of a multi-objective task scheduling method in an edge computing environment provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the structure of a pointer network model provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the internal structure of a decoder in a pointer network model provided by an embodiment of the present invention;

[0044] Figure 4 A flowchart of a multi-objective task scheduling method in an edge computing environment provided by an embodiment of the present invention;

[0045] Figure 5 A data comparison diagram of experimental result 1 provided in an embodiment of the present invention;

[0046] Figure 6 A data comparison diagram of Experimental Result 2 provided in an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the structure of a multi-objective task scheduling device in an edge computing environment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a multi-objective task scheduling method in an edge computing environment, which includes the steps of:

[0050] S100: Constructing a pointer network model for deciding the execution order of multiple service requests in an edge server, and making the result sampling strategy of the pointer network model use the average waiting time of the service requests as an optimization indicator;

[0051] S200: Integrate the resource utilization and running time of the edge server into an optimization objective as a reward function of reinforcement learning to train the pointer network model.

[0052] It should be noted that resource utilization indicates the average usage of resources in the process of edge server processing tasks, which may specifically include the utilization of the edge server's CPU capacity, I / O capacity, bandwidth capacity, and memory capacity; the service request running time can refer to the total time required for the edge server to execute all queued service requests; the service request waiting time can refer to the time interval from the service request reaching the edge server to the completion of processing.

[0053] This embodiment of the present invention addresses the need to balance resource utilization and latency sensitivity in edge environments. By building a model based on pointer networks and reinforcement learning, this approach combines three metrics: resource utilization, runtime, and latency during task execution. This method proposes a multi-objective task scheduling method for edge environments based on pointer networks and reinforcement learning. Experiments have demonstrated that this method effectively improves resource utilization on edge servers, shortens the time required to complete task sequences, and reduces average task latency.

[0054] It is understandable that before constructing the pointer network model (PtNet) for offline training in S100, it is necessary to obtain standardized edge server and task features and assign tasks to specific edge servers for execution. In some embodiments, the steps include:

[0055] S010: Extract edge server and task data in a real environment, preprocess the data, and obtain standardized edge server and task features.

[0056] S020: Based on step S010, according to the geographical location and coverage of the edge server and the geographical location where the task is initiated, calculate which edge servers can receive the task, and assign the task to a specific edge server for execution.

[0057] Specifically, S010 also includes:

[0058] Step S011: Analyze the edge server data in the real environment and use a four-dimensional vector to represent the edge server server j for:

[0059] server j =(M j ,B j ,O j ,C j ),

[0060] Mj Indicates the total memory size of the edge server; B j Indicates the total bandwidth of the network where the edge server is located; j Indicates the input / output capability of the edge server; C j Represents the computing power of the edge server CPU. Define the set S of m edge servers as:

[0061] S={s1,s2,…,s m}.

[0062] Step S012: Analyze the task data in the real environment and use the seven-dimensional vector to represent the task i for:

[0063] task i =(m i ,b i ,o i ,c i ,T i ,t i ,π i ).

[0064] (m i ,b i ,o i ,c i ) is used to indicate the execution of a task i Required memory, bandwidth, I / O, and CPU; task i The timestamp of initiation is T i Indicates task i The time required for running is t i Represents; π i Indicates the set of all edge servers that can run the task, which is calculated using the geographical location and coverage of the edge servers combined with the geographical location when the task is initiated. i The coordinates at the time of initiation are P i Indicates that the edge server s j The coordinates of P j Indicates that the edge server s j The coverage radius is R j Indicates. i Expressed as:

[0065] π i ={s j |R j ≥||P i -P j ||2,R j ∈S},

[0066] where ||Pi -P j ||2 means P i With P j The Euclidean distance between them. Define the N task set Q as:

[0067] TASK={task1,task2,…,task N}.

[0068] Specifically, S020 also includes:

[0069] S021: Based on step S012, calculate each task i The corresponding π i .

[0070] S022: For each task i , from π i Select an edge server to process the task i The selection strategy can adopt classic load balancing algorithms, including weighted randomness, consistent hashing, etc. When multiple tasks are assigned to the same edge server, due to the limited resources of the edge server, it cannot meet the running requirements of all tasks at the same time. The unrunning tasks need to wait in the edge server to be scheduled.

[0071] The embodiment of the present invention extracts edge server data and task data in a real environment for PtNet decision-making and training, making the model's operating environment closer to the real environment.

[0072] In some embodiments, the pointer network model constructed in S100 includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to help the Actor network perform the training. The Actor network includes an encoder and a decoder, and the encoder and decoder both include a recurrent neural network composed of multiple long short-term memory networks. The pointer network model decides the execution order of multiple service requests in the edge server, including the steps of:

[0073] S110: The queued service request sequence is used as an input sequence and converted into a first intermediate vector, which is input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder;

[0074] S120: Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by using the attention mechanism of the pointer network model for the state of each hidden layer of the decoder;

[0075] S130: Obtaining, based on the second intermediate vector, the probability that the decoder selects each service request in a hidden layer as the output of the current layer;

[0076] S140: Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server.

[0077] It should be noted that in S110, the first intermediate vector can be understood as the vector form converted after the input sequence is embedded (Embedding) operation. This first intermediate vector is passed through a recurrent neural network (RNN) composed of multiple long short-term memory (LSTM) networks to obtain the state of each hidden layer of the corresponding encoder. The number of hidden layers of the encoder and decoder depends on the length of the input sequence. In S120, the second intermediate vector can be expressed as: where e j represents the state of the jth hidden layer of the encoder, d i represents the hidden layer state of the decoder layer i. In S130, the second intermediate vector can be subjected to a softmax operation to obtain: p(C i |C1,…,C i-1 ,Q)=softmax(u i ), which represents the probability vector of the input sequence in the i-th hidden layer of the decoder. Its dimension is equal to the length of the input sequence, which means the probability of the decoder selecting each task as the output of this layer in the i-th hidden layer.

[0078] It is understandable that the result sampling strategy in S140 uses the average waiting time of service requests as an optimization indicator, so that the pointer network model tends to execute tasks that require less time to run first to a certain extent, thereby achieving the effect of reducing the average waiting time.

[0079] Preferably, each hidden layer in S140 selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, including the steps of:

[0080] S141: For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so,

[0081] Among the second intermediate vectors corresponding to the service requests that have not yet been selected, the service requests corresponding to a portion of the second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

[0082] Furthermore, if the number of service requests that have not been selected exceeds the first preset threshold, the output of this layer can be determined according to the conventional decision-making method, that is, probability sampling is performed in the second intermediate vector corresponding to the service requests that have not been selected and the output of this layer is determined according to the sampling results.

[0083] In some embodiments, S200 is used to use the extracted data to train the PtNet offline using reinforcement learning. When training the PtNet using reinforcement learning, it is first necessary to define a reward function. The three indicators of resource utilization, running time, and average waiting time can be selected as evaluation indicators of service quality.

[0084] Resource utilization can be expressed as:

[0085]

[0086] Use (M j ,B j ,O j ,C j ) represents the average memory, bandwidth, I / O, and CPU usage of the jth edge server during parallel processing;

[0087] The running time can be expressed as:

[0088]

[0089] Among them, T_server j It represents the time required for the jth edge server to complete all tasks;

[0090] The average waiting time can be expressed as:

[0091]

[0092] Among them, WAIT i Indicates task task i waiting time.

[0093] In some embodiments, S200 includes the steps of:

[0094] S210: Based on Determine the reward function reward, where m is the total number of edge servers, T_server j The total time required for edge server j to complete all service requests;

[0095] S220: Use reinforcement learning with stochastic gradient descent for training.

[0096] It is understandable that since reinforcement learning training requires the design of a reward function, the three evaluation indicators of resource utilization, runtime, and average waiting time can be weighted and summed to serve as the reward. However, in this embodiment, considering further efficiency improvements, this multi-objective optimization problem is converted into a single-objective optimization problem at the training level. Taking advantage of the strong linear correlation between the two optimization objectives, resource utilization and runtime, the resource utilization evaluation indicator and runtime evaluation indicator are integrated into one during PtNet training, that is, runtime is used to represent them uniformly.

[0097] It should be noted that in S220, the Actor network can be trained using a policy gradient-based reinforcement learning method, and the Critic network can be trained using a stochastic gradient descent reinforcement learning method. The specific training process can adopt the Actor-Critic (A3C) algorithm.

[0098] Therefore, the embodiment of the present invention integrates resource utilization and runtime into one optimization objective, and proposes a new pointer network sampling strategy to integrate the waiting time optimization objective into the model decision, thereby achieving the effect of multi-objective optimization. Compared with the weighted summation method, the advantage of this method is that it does not require a large amount of experimental analysis on the weight values, and is more effective for pointer network model training.

[0099] Furthermore, the pointer network model can be trained offline first, and the real data accumulated historically can be used to train the model. Then, the trained model can be deployed in a real edge computing environment, and the generated real-time data can be input into the pointer network model after offline training for online training.

[0100] Online training involves deploying a model in an edge environment and using the data from tasks arriving in real time on the edge server to train the model in real time. Its unique feature is that the model can self-adjust based on the characteristics of the real-time tasks, optimizing the model's performance and adapting to dynamic changes in task characteristics.

[0101] In this embodiment of the present invention, the pointer network model undergoes offline training before deployment to the edge environment. Compared to deploying the model without training and then training it online, this approach avoids the problem of an untrained model making poor decisions after direct deployment, which can lead to severe fluctuations in overall service quality. Furthermore, through online training, the model continuously adapts to the dynamic changes in task characteristics, preventing degradation in scheduling quality caused by changes in task characteristics over time.

[0102] like Figure 4 As shown, in a specific embodiment, the multi-objective task scheduling method in the edge computing environment has the following specific steps:

[0103] Step S1: Extract edge server and task data in a real environment, pre-process the data, and obtain standardized edge server and task features. Step S1 includes:

[0104] Step S1.1: Analyze the edge server data in the real environment and use a four-dimensional vector to represent the edge server server j For: server j =(M j ,B j ,O j ,C j ), where M j Indicates the total memory size of the edge server, B j Indicates the total bandwidth of the network where the edge server is located, O j represents the input / output capacity of the edge server, C j Indicates the computing power of the edge server CPU; define the set of m edge servers S as: S = {s1,s2,…,s m}.

[0105] Step S1.2: Analyze the task data in the real environment and use the seven-dimensional vector to represent the task i For: task i =(m i ,b i ,o i ,c i ,T i ,t i ,π i ), where (m i ,b i ,o i ,c i ) is used to indicate the execution of a task i Required memory, bandwidth, I / O, and CPU, T i Indicates task task i Timestamp of initiation, t i Indicates task task i The time required for running, π i represents the set of all edge servers that can run the task (which can be calculated using the geographical location and coverage of the edge server combined with the geographical location when the task is initiated), P i Indicates that the task i Coordinates at the time of initiation, P j Indicates that the edge server s j The coordinates of R j Indicates that the edge server s j Coverage radius;

[0106] πi Expressed as: π i ={s j |R j ≥||P i -P j ||2,R j ∈S}, where ||P i -P j ||2 means P i With P j The Euclidean distance between

[0107] Define N task sets Q as: TASK = {task1,task2,…,task N}.

[0108] Step S2: Based on the geographical location and coverage of the edge server, combined with the geographical location of the task initiation, calculate which edge servers can receive the task and assign the task to a specific edge server for execution. S2 includes:

[0109] Step S2.1: Calculate each task i The corresponding π i .

[0110] Step S2.2: For each task i , from π i Select an edge server to process the task i The selection strategy adopts classic load balancing algorithms, including weighted random and consistent hashing. When multiple tasks are assigned to the same edge server, the limited resources of the edge server make it impossible to meet the running requirements of all tasks at the same time. The unrunning tasks need to wait in the edge server to be scheduled.

[0111] Step S3: Build a pointer network model PtNet, use the extracted data, and use reinforcement learning to train PtNet offline, and deploy the trained model in the edge environment. S3 includes:

[0112] Step S3.1: Establish the pointer network model PtNet. The overall structure of PtNet is divided into two parts, the Actor network and the Critic network. The Actor network as a whole is composed of two recurrent neural networks (RNNs), called encoders and decoders, which are used to decide the execution order of the task sequence. Each recurrent neural network is composed of multiple long short-term memory network (LSTM) units; the role of the Critic network is to help the Actor network train, and it is composed of a recurrent neural network and a deep neural network (DNN). The overall structure of the PtNet model is as follows Figure 2 shown.

[0113] Step S3.2: Train PtNet using reinforcement learning. First, define the reward function. In this method, the evaluation indicators of service quality include resource utilization, running time, and average waiting time. Specifically:

[0114] Resource utilization refers to the average usage of resources in the process of edge server processing tasks, using (M j ,B j ,O j ,C j ) represents the average memory, bandwidth, I / O, and CPU usage of the jth edge server during parallel processing. The resource utilization index is defined using reward1 as:

[0115]

[0116] The running time represents the total time required for all tasks to be processed by the edge server. The running time indicator is defined using reward2 as:

[0117]

[0118] Where T_server j It represents the time required for the jth edge server to complete all tasks.

[0119] The average waiting time represents the average time from the time when all tasks arrive at the edge server to the time when they are processed by the edge server. i Indicates task task i The waiting time is defined using reward3 as the average waiting time indicator:

[0120]

[0121] Step S3.3: Use policy gradient-based reinforcement learning to train the Actor network, and use stochastic gradient descent reinforcement learning to train the Critic network. The specific training process adopts the Actor-Critic (A3C) algorithm.

[0122] Step S3.4: Integrate the resource utilization evaluation index and the runtime evaluation index into one, and uniformly use runtime to represent them.

[0123] Step S3.5: Propose a new pointer network sampling strategy, so that in this strategy, the pointer network will tend to let the tasks that require less running time be executed first to a certain extent, thereby achieving the effect of reducing the average waiting time.

[0124] S3.5 specifically includes:

[0125] S3.5_1: In a certain step of the decoder, there are p1~p n Sequence, corresponding to q1~q n Input sequence, excluding the input q that has been selected before to avoid repeated sampling;

[0126] It should be noted that if Figure 3 As shown, in the decoder of the pointer network, each step will generate such a p1~p n Sequences, corresponding to q1~q n Input sequence, p1~p n The meaning of the sequence is the probability of each input corresponding to that step.

[0127] S3.5_2: Determine the number of remaining inputs q. If it is less than or equal to 5, execute S3.5_3; otherwise, execute S3.5_4.

[0128] S3.5_3: Select the final sampling result q* according to the common sampling strategy and jump to S3.5_6. The common sampling strategy includes:

[0129] Assume that the input length is 3. In a certain step of the decoder, the values of p1, p2, and p3 are 0.2, 0.7, and 0.1 respectively. Assume that in this step, the final sampling result of the pointer network is q`. Then

[0130] Pointer Network The probability of choosing q1 as The probability of choosing q2 as The probability of selecting q3 as q`, q` is the final decision result of this step;

[0131] S3.5_4: Sort the remaining inputs q according to their corresponding p values, and select the first five inputs q with the largest p values;

[0132] S3.5_5: Analyze the characteristics of the five inputs q and select the input q with the shortest execution time as the sampling result q*;

[0133] S3.5_6: Use the sampling result q* as the decision result of the decoder in this step.

[0134] Step S3.6: Define the reward function calculation formula used in pointer network training as:

[0135]

[0136] Use the data extracted in step S2 to train the model offline, and deploy the trained model in an edge environment for decision-making.

[0137] Step S4: When the task sequence is waiting to be executed inside the edge server, the specific execution order of the task sequence is determined using PtNet. S4 specifically includes:

[0138] Step S4.1: Input the waiting task sequence as input data into the pointer network model PtNet, which is divided into two parts: the Actor network and the Critic network.

[0139] Step S4.2: Based on step S4.1, the input data is determined by the Actor network to determine the execution order of the tasks. The specific decision-making process is as follows:

[0140] S4.2_1: Perform an embedding operation on the input sequence to convert it into an intermediate vector representation.

[0141] S4.2_2: Pass the intermediate vector through a recurrent neural network (RNN) composed of multiple long short-term memory (LSTM) networks to obtain the state of each hidden layer of the corresponding encoder. The number of hidden layers in the encoder and decoder depends on the length of the input sequence.

[0142] S4.2_3: The encoder state is used as the decoder input and the attention mechanism of the pointer network is used to obtain:

[0143]

[0144] where e j represents the state of the jth hidden layer of the encoder, d i Represents the hidden layer state of the decoder layer i, and then performs a softmax operation on it to obtain: p(C i |C1,…,C i-1 ,Q)=softmax(u i ); and p(C i |C1,…,C i-1 ,Q) represents the probability vector of the input sequence in the i-th hidden layer of the decoder, and its dimension is equal to the length of the input sequence, which means the probability that the decoder selects each task as the output of this layer in the i-th hidden layer.

[0145] S4.2_4: According to the pointer network sampling strategy proposed in S3.5, the output of each decoder hidden layer is selected. Since the number of decoder hidden layers is equal to the length of the input sequence, a new sequence with the same length as the input sequence can be formed as the output sequence.

[0146] S4.2_5: The edge server processes the tasks in the execution order defined by this output sequence.

[0147] Step S5: While PtNet is making decisions, reinforcement learning is used for online training to enable it to continuously adapt to the dynamic changes in data features and improve the robustness of PtNet. S5 specifically includes:

[0148] Step S5.1: Perform online training in the same manner as the offline training process in S3. The difference between online training and offline training is that online training involves deploying the model on the edge environment and using the task data as it arrives at the edge server. This allows the model to self-adjust based on the characteristics of the real-time task, optimizing the model's performance and adapting to dynamic changes in task characteristics.

[0149] In a specific embodiment, when using a multi-target task scheduling method in an edge computing environment, an edge computing simulation environment is first established, and edge server and task data are generated based on the real geographic location information provided by the EUA dataset. The Alibaba Cluster Trace dataset is used as the simulation data for the task, totaling 700,000 data items. 600,000 of these data items are used as a training set and 100,000 as a test set to train the pointer network model provided by the present invention. The specific method is as follows:

[0150] First, we applied the pointer network model to actual scheduling, designing two sets of experiments. The first set of experiments fixed five edge servers and varied the number of tasks between 300 and 50, 350 and 400, 400 and 450, and 450 and 500, respectively. The second set of experiments fixed the number of tasks to 500 and varied the number of edge servers to 5, 7, 9, 11, 13, and 15, respectively. Then, based on the edge server coverage, tasks within the coverage area were assigned to edge servers capable of handling them according to a load balancing strategy. Within the edge servers, the pointer network model was used to schedule waiting tasks, and three metrics—resource utilization, runtime, and average wait time—were calculated.

[0151] The three indicators were then compared with scheduling methods in related technologies to verify the effectiveness of multi-objective task scheduling in edge computing environments. The scheduling methods used for comparison included the First Come First Served (FCFS) algorithm, the High Response Ratio Priority (HRRN) algorithm, the Q-learning-based Online Reinforcement Learning (OnPQ) algorithm, and the Online Delay-Sensitive Task Scheduling (OnDisc) algorithm.

[0152] like Figure 5 、 6 As shown in the figure, the experimental results show that the multi-objective task scheduling method in the edge computing environment provided by the present invention is superior to the other four comparison methods in terms of resource utilization, running time and average waiting time.

[0153] On the other hand, an embodiment of the present invention further provides a multi-objective task scheduling device in an edge computing environment, characterized in that it includes:

[0154] A multi-service request decision module constructs a pointer network model for deciding the execution order of multiple service requests in the edge server, and uses the average waiting time of service requests as an optimization indicator in the result sampling strategy of the pointer network model;

[0155] The reinforcement learning module integrates the resource utilization and running time of the edge server into an optimization objective as the reward function of reinforcement learning to train the pointer network model.

[0156] In some embodiments, the pointer network model includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to assist the Actor network in the training. The Actor network includes an encoder and a decoder, and both the encoder and the decoder include a recurrent neural network composed of multiple long short-term memory networks.

[0157] The multi-service request decision module is also used to:

[0158] The queued service request sequence is used as an input sequence and converted into a first intermediate vector and input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder;

[0159] Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by the attention mechanism of the pointer network model for the state of each hidden layer of the decoder;

[0160] Obtaining, based on the second intermediate vector, a probability that the decoder selects each service request in a hidden layer as an output of the layer;

[0161] Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server.

[0162] In some embodiments, the multi-service request decision module is further configured to:

[0163] For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so,

[0164] Among the second intermediate vectors corresponding to the service requests that have not yet been selected, service requests corresponding to a portion of second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

[0165] In some embodiments, the multi-service request decision module is further configured to:

[0166] If the number of service requests that have not been selected exceeds a first preset threshold, probability sampling is performed in the second intermediate vectors corresponding to the service requests that have not been selected, and the output of the current layer is determined according to the sampling result.

[0167] In some embodiments, the reinforcement learning module is used to:

[0168] based on Determine the reward function reward, where m is the total number of edge servers, T_server j The total time required for edge server j to complete all service requests;

[0169] Training is performed using reinforcement learning with stochastic gradient descent.

[0170] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0171] The above are only specific implementations of the embodiments of the present invention, but the scope of protection of the embodiments of the present invention is not limited to them. Any person skilled in the art can easily conceive of various equivalent modifications or replacements within the technical scope disclosed in the embodiments of the present invention, and such modifications or replacements should be included in the scope of protection of the embodiments of the present invention. Therefore, the scope of protection of the embodiments of the present invention should be based on the scope of protection of the claims.

Claims

1. A multi-objective task scheduling method in an edge computing environment, characterized in that: It includes the steps of: Constructing a pointer network model for deciding the execution order of multiple service requests in the edge server, and making the result sampling strategy of the pointer network model use the average waiting time of the service requests as the optimization indicator; The resource utilization and running time of the edge server are integrated into an optimization objective as the reward function of reinforcement learning to train the pointer network model; The pointer network model includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to help the Actor network perform the training. The Actor network includes an encoder and a decoder, and the encoder and decoder both include a recurrent neural network composed of multiple long short-term memory networks. The decision on the execution order of multiple service requests in the edge server includes the following steps: The queued service request sequence is used as an input sequence and converted into a first intermediate vector and input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder; Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by the attention mechanism of the pointer network model for the state of each hidden layer of the decoder; Obtaining, based on the second intermediate vector, a probability that the decoder selects each service request in a hidden layer as an output of the layer; Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server; The method integrates the resource utilization and running time of the edge server into an optimization objective as a reward function for reinforcement learning to train the pointer network model, including the following steps: based on Determine the reward function ,in, is the total number of edge servers, The total time required for edge server j to complete all service requests; Training is performed using reinforcement learning with stochastic gradient descent.

2. The multi-objective task scheduling method in an edge computing environment according to claim 1, characterized in that: Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, including the steps of: For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so, Among the second intermediate vectors corresponding to the service requests that have not yet been selected, service requests corresponding to a portion of second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

3. The multi-objective task scheduling method in an edge computing environment according to claim 2, characterized in that: Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, including the steps of: If the number of service requests that have not been selected exceeds a first preset threshold, probability sampling is performed in the second intermediate vectors corresponding to the service requests that have not been selected, and the output of the current layer is determined according to the sampling result.

4. A multi-objective task scheduling device in an edge computing environment, characterized in that: It includes: A multi-service request decision module constructs a pointer network model for deciding the execution order of multiple service requests in the edge server, and uses the average waiting time of service requests as an optimization indicator in the result sampling strategy of the pointer network model; The reinforcement learning module integrates the resource utilization and running time of the edge server into an optimization objective as the reinforcement learning reward function to train the pointer network model; The pointer network model includes an Actor network and a Critic network, wherein the Actor network is used to decide the execution order of multiple service requests, and the Critic network is used to help the Actor network perform the training. The Actor network includes an encoder and a decoder, and the encoder and decoder both include a recurrent neural network composed of multiple long short-term memory networks; The multi-service request decision module is further configured to: The queued service request sequence is used as an input sequence and converted into a first intermediate vector and input into the encoder of the Actor network to obtain the state of each hidden layer of the corresponding encoder; Inputting the state of each hidden layer of the encoder into the decoder to obtain the state of each hidden layer of the decoder, and obtaining a second intermediate vector by the attention mechanism of the pointer network model for the state of each hidden layer of the decoder; Obtaining, based on the second intermediate vector, a probability that the decoder selects each service request in a hidden layer as an output of the layer; Each hidden layer selects a corresponding service request as the output of the layer based on the probability of the output of the layer and the result sampling strategy, and defines the order of executing the service requests according to the output of all hidden layers as the output sequence of the edge server; The reinforcement learning module is used to: based on Determine the reward function ,in, is the total number of edge servers, The total time required for edge server j to complete all service requests; Training is performed using reinforcement learning with stochastic gradient descent.

5. The multi-objective task scheduling device in an edge computing environment according to claim 4, characterized in that: The multi-service request decision module is further configured to: For the second intermediate vector and the corresponding service request, determine whether the number of service requests that have not been selected exceeds a first preset threshold. If so, Among the second intermediate vectors corresponding to the service requests that have not yet been selected, service requests corresponding to a portion of second intermediate vectors with larger values are selected, and the service request with the shortest running time is selected from these service requests as the output of this layer.

6. A multi-objective task scheduling device in an edge computing environment according to claim 5, characterized in that: The multi-service request decision module is further configured to: If the number of service requests that have not been selected exceeds a first preset threshold, probability sampling is performed in the second intermediate vectors corresponding to the service requests that have not been selected, and the output of the current layer is determined according to the sampling result.

Images