Task unloading prediction method and device for power grid processing task, computer equipment, readable storage medium and program product

Abstract

The invention relates to a task unloading prediction method and device for a power grid processing task, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an initial strategy network model and an initial value network model, and acquiring sample power grid processing task information and current training environment information; in cooperation with other edge servers and a central cloud server, according to the current training environment information and the sample power grid processing task information, performing centralized training on the initial strategy network model and the initial value network model in the edge servers to obtain a trained target strategy network model; under the condition that a new power grid processing task is received, obtaining current state information of an edge server; and inputting the current state information and the task information of the new power grid processing task into the target strategy network model to obtain task unloading prediction information for the new power grid processing task. By adopting the method, the resource utilization rate can be improved.

Description

Technical Field

[0001] The present application relates to the field of smart grid technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for predicting task offloading of grid processing tasks. Background Art

[0002] As the scale of power grids expands and electricity demand grows, the number of devices and the complexity of task processing in power grid systems have increased significantly. Traditionally, power grid task processing relies on centralized computing in cloud centers. Although this approach has powerful computing capabilities, it often results in higher communication delays and energy consumption. The introduction of edge computing enables some tasks to be processed on edge servers close to the data source, reducing latency and improving response speed.

[0003] However, edge servers have limited computing resources. Faced with a growing workload, uneven resource allocation is prone to occur, causing some servers to be overloaded while other server resources are idle, resulting in low overall computing resource utilization. Summary of the invention

[0004] Based on this, it is necessary to provide a task offloading prediction method, device, computer equipment, computer-readable storage medium and computer program product for power grid processing tasks that can improve resource utilization in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for predicting task offloading of a power grid processing task, comprising:

[0006] Acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0007] Cooperating with the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model;

[0008] Upon receiving a new power grid processing task, obtaining current status information of the edge server;

[0009] The current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0010] In one embodiment, the coordinating the other edge servers and the central cloud server to centrally train the initial policy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information to obtain a trained target policy network model includes:

[0011] Iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain a trained candidate strategy network model and a candidate value network model;

[0012] Send the uploaded model parameters to the central cloud server, and receive the aggregated model parameters returned by the central cloud server; the uploaded model parameters are the model parameters of the candidate policy network model and the candidate value network model; the aggregated model parameters are the model parameters determined by the central cloud server based on the uploaded model parameters of all edge servers;

[0013] According to the aggregation model parameters, a new initial strategy network model and a new initial value network model are determined, and the step of iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain trained candidate strategy network models and candidate value network models is jumped to, until a target strategy network model that meets the first preset training completion condition is obtained.

[0014] In one embodiment, the iterative training of the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and candidate value network model includes:

[0015] Using the initial policy network model as the current policy network model, and using the initial value network model as the current value network model;

[0016] Inputting the sample power grid processing task information and the state information of the edge server into the current policy network model to obtain first sample task unloading prediction information, and obtaining second sample task unloading prediction information of the other edge servers;

[0017] Obtaining a sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information, and the current value network model, and adjusting a model parameter of the current value network model according to the sample action value function value to obtain a new current value network model;

[0018] Inputting the sample task offloading prediction information into the new current value network model to obtain a new sample action value function value, and adjusting the model parameters of the current policy network model according to the new sample action value function value to obtain a new current policy network model;

[0019] Obtain new current training environment information, and jump to the step of inputting the power grid processing task information and the status information of the edge server into the current strategy network model to obtain sample task unloading prediction information, until a candidate strategy network model and a candidate value network model that meet the second preset training completion condition are obtained.

[0020] In one embodiment, the step of inputting the sample power grid processing task information and the edge server status information into the current policy network model to obtain the first sample task offloading prediction information includes:

[0021] Obtaining preset constraints for task offloading of power grid processing tasks;

[0022] Determining constraint judgment information corresponding to the preset constraint condition according to the sample power grid processing task information and the state information of the edge server;

[0023] According to the constraint judgment information, the preset constraint condition, the sample power grid processing task information and the state information of the edge server, first sample task offloading prediction information that meets the preset constraint condition is obtained.

[0024] In one embodiment, obtaining the sample action value function value according to the first sample task offloading prediction information, the second sample task offloading prediction information, the current training environment information and the current value network model includes:

[0025] obtaining a preset objective function for task offloading of a power grid processing task;

[0026] Determining objective function information corresponding to the preset objective function according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information;

[0027] A sample action value function value is obtained according to the objective function information, the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model.

[0028] In one embodiment, determining the objective function information corresponding to the preset objective function according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information includes:

[0029] Determine task completion delay information and task energy consumption information for the sample power grid processing task information respectively according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information;

[0030] The task completion delay information and the task energy consumption information are used as the objective function information.

[0031] In a second aspect, the present application further provides a task unloading prediction device for a power grid processing task, comprising:

[0032] A data acquisition module, used to acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information at least includes the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0033] A model training module, used to coordinate with the other edge servers and the central cloud server, centrally train the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, and obtain a trained target strategy network model;

[0034] A real-time application module, configured to obtain current status information of the edge server upon receiving a new power grid processing task;

[0035] The unloading prediction module is used to input the current state information and the task information of the new power grid processing task into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0037] Acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0038] Cooperating with the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model;

[0039] Upon receiving a new power grid processing task, obtaining current status information of the edge server;

[0040] The current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0042] Acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0043] Cooperating with the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model;

[0044] Upon receiving a new power grid processing task, obtaining current status information of the edge server;

[0045] The current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0046] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0047] Acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0048] Cooperating with the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model;

[0049] Upon receiving a new power grid processing task, obtaining current status information of the edge server;

[0050] The current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0051] The task offloading prediction method, device, computer equipment, computer-readable storage medium and computer program product of the above-mentioned power grid processing task first obtains the initial policy network model and the initial value network model, as well as obtains the sample power grid processing task information and the current training environment information, wherein the current training environment information at least includes the status information of the edge server, the status information of other edge servers and the status information of the central cloud server. By obtaining the initial policy network model and the initial value network model, a basic learning framework is provided so that the edge server can have an initial decision-making basis at the beginning of training. The sample power grid processing task information and the current training environment information can ensure that the training process can simulate the actual task processing environment, thereby improving the adaptability and generalization ability of the model; then, in collaboration with other edge servers and the central cloud server, the initial policy network model and the initial value network model in the edge server are centrally trained according to the current training environment information and the sample power grid processing task information. , and obtain the trained target strategy network model. Through collaborative centralized training, the edge server can use global environmental information for learning. The centralized training mechanism can make the model better adapt to complex power grid task unloading scenarios. Through collaboration with other edge servers and central cloud servers, the model can learn more effective resource allocation strategies, thereby avoiding the problem of uneven resource allocation and improving the overall utilization of computing resources. Finally, when receiving a new power grid processing task, the current state information of the edge server is obtained, and the current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain the task unloading prediction information for the new power grid processing task. In the actual operation stage, when the edge server receives a new power grid processing task, it can quickly obtain the current state information and combine the pre-trained target strategy model to make efficient task unloading predictions, which can significantly reduce the delay of task processing, improve the response speed, and optimize the resource utilization of the edge server. In the above method, by combining the initial model, centralized training and task unloading prediction, the edge server can effectively respond to dynamically changing power grid processing tasks. Through global collaborative learning in the centralized training phase, edge servers can optimize resource allocation and avoid waste and uneven utilization of computing resources. In the distributed execution phase, servers can make fast and accurate unloading decisions based on real-time status, reduce processing delays, and improve the overall operating efficiency and stability of the power grid system. This solves the problems of high communication delay and high energy consumption in the traditional centralized computing model, while avoiding the low resource utilization of edge servers in task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A diagram of an application environment of a task offloading prediction method for a power grid processing task in one embodiment;

[0054] Figure 2 A schematic flow chart of a method for predicting task offloading of a power grid processing task in one embodiment;

[0055] Figure 3 A flowchart of steps for obtaining a target policy network model in one embodiment;

[0056] Figure 4 A schematic diagram of a flow chart of steps for obtaining a candidate strategy network model and a candidate value network model in one embodiment;

[0057] Figure 5 A schematic diagram of the structure of a multi-agent proximal strategy optimization algorithm in one embodiment;

[0058] Figure 6 A schematic diagram of multi-model performance comparison in one embodiment;

[0059] Figure 7 A structural block diagram of a task offloading prediction device for a power grid processing task in one embodiment;

[0060] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] The task unloading prediction method for power grid processing tasks provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the edge server 102, other edge servers 104 and the central cloud server 106 communicate with each other through the network. The edge server 102 obtains the initial policy network model and the initial value network model, as well as the sample power grid processing task information and the current training environment information, wherein the current training environment information at least includes the status information of the edge server 102, the status information of other edge servers 104 and the status information of the central cloud server 106; then, the edge server 102 cooperates with other edge servers 104 and the central cloud server 106, and centrally trains the initial policy network model and the initial value network model in the edge server 102 according to the current training environment information and the sample power grid processing task information, and obtains the trained target policy network model; when receiving a new power grid processing task, the edge server 102 obtains the current state information of the edge server 102, and then inputs the current state information and the task information of the new power grid processing task into the target policy network model, and obtains the task unloading prediction information for the new power grid processing task. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0063] In an exemplary embodiment, Figure 2 As shown, a task unloading prediction method for power grid processing tasks is provided, and the method is applied to Figure 1 Taking the edge server 102 in the example as an example, the following steps are included:

[0064] Step S201, obtaining an initial strategy network model and an initial value network model, as well as obtaining sample power grid processing task information and current training environment information.

[0065] Among them, the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server.

[0066] The sample power grid processing task information refers to the power grid task data used to train the model, including the task computing requirements, data volume, latency requirements, etc. The current training environment information refers to the system status data that affects the task offloading decision during the training process, including at least the edge server status information (such as CPU usage, memory usage, network bandwidth, etc.), other edge servers status information (such as adjacent servers’ resource usage, task queue length, etc.) and the central cloud server status information (such as the cloud server’s available resources and current load, etc.).

[0067] Among them, the initial policy network model can be a neural network model set by the edge server before the training starts to generate task offloading decisions, and the initial value network model can be a neural network model used to evaluate the quality of task offloading decisions.

[0068] Exemplarily, the edge server first obtains the initial policy network model and the initial value network model, and the initial parameters of these models may come from user presets or pre-training results. Then, the edge server obtains sample power grid processing task information for training. Next, the edge server collects the current training environment information, including its own resource status information, and obtains their status information through communication with other edge servers and central cloud servers. The edge server stores and organizes this information into input data sets in preparation for subsequent centralized training steps. It should be noted that throughout the process, each edge server, as its own execution subject, actively obtains, processes and integrates this information to ensure that the initial model can be effectively trained in a real environment, ultimately improving the adaptability and prediction accuracy of the model.

[0069] Step S202, in collaboration with other edge servers and the central cloud server, centrally train the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information to obtain a trained target strategy network model.

[0070] Exemplarily, the edge server first establishes a communication link with other edge servers and the central cloud server to form a collaborative training network. In this network, the edge server uploads the current training environment information to the cloud center, and receives status information from other edge servers and the cloud center. Subsequently, the edge server uses this global environment information to optimize and update the initial policy network model and the initial value network model. Specifically, the edge server uses the collected environmental information and power grid processing task information to adjust the model parameters through optimization algorithms such as gradient descent, so that the model can better predict the optimal unloading strategy for the task. During the training process, the edge server continuously exchanges information with other servers to ensure that the model learns in a global environment, and gradually optimizes the performance of the model through multiple rounds of iterations. Finally, after the training is completed, the edge server obtains an optimized target policy network model.

[0071] Step S203, when a new power grid processing task is received, the current state information of the edge server is obtained.

[0072] For example, when the edge server completes model training and actually performs work, when the edge server receives a processing requirement for a new power grid processing task, it needs to collect current status information, which may include real-time CPU load, memory usage, length of the current task queue, and network connection status, etc. This status information reflects the resource availability and current processing capacity of the edge server.

[0073] Step S204, inputting the current state information and the task information of the new power grid processing task into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

[0074] Exemplarily, the edge server inputs the current state information and relevant information of the new task (such as the computational workload, data volume, latency requirements, etc. of the task) into the target strategy network model. The target strategy network model performs calculations based on the input information, evaluates the potential effects of different task offloading strategies, and outputs the optimal task offloading prediction information for the new task. The offloading prediction information may include strategies such as processing the task directly on the local server, offloading the task to other edge servers for processing, or transferring the task to the central cloud server for processing. Based on this prediction information, the edge server selects the optimal task processing path, thereby improving the efficiency of task processing and reducing the overall latency and energy consumption of the system.

[0075] It should be noted that during the entire model training phase, each edge server will simultaneously execute the above steps, independently obtain the initial model and current environment information, and participate in the centralized training process through collaboration with other edge servers and central cloud servers. This parallel training enables the policy network and value network of each edge server to be optimized from a global perspective. With the completion of training, the distributed execution phase begins, that is, each edge server uses the target policy network model to independently make task offloading decisions in actual operation. Therefore, each edge server will also apply the trained target policy network in the distributed execution phase, and make the optimal offloading decision based on real-time status information and task requirements to ensure the overall performance of the power grid system and maximize resource utilization efficiency.

[0076] In the task offloading prediction method for the above-mentioned power grid processing task, first, an initial policy network model and an initial value network model are obtained, as well as sample power grid processing task information and current training environment information, wherein the current training environment information at least includes the status information of the edge server, the status information of other edge servers and the status information of the central cloud server. By obtaining the initial policy network model and the initial value network model, a basic learning framework is provided, so that the edge server can have an initial decision basis at the beginning of training. The sample power grid processing task information and the current training environment information can ensure that the training process can simulate the actual task processing environment, thereby improving the adaptability and generalization ability of the model; then, in collaboration with other edge servers and the central cloud server, the initial policy network model and the initial value network model in the edge server are centrally trained according to the current training environment information and the sample power grid processing task information, and the trained target policy network model is obtained. The network model is constructed by the edge server. Through collaborative centralized training, the edge server can use the global environment information for learning. The centralized training mechanism can make the model better adapt to the complex power grid task offloading scenario. Through the collaboration with other edge servers and central cloud servers, the model can learn more effective resource allocation strategies, thereby avoiding the problem of uneven resource allocation and improving the overall computing resource utilization. Finally, when receiving a new power grid processing task, the current state information of the edge server is obtained, and the current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain the task offloading prediction information for the new power grid processing task. In the actual operation stage, when the edge server receives a new power grid processing task, it can quickly obtain the current state information and combine the pre-trained target strategy model to make an efficient task offloading prediction, which can significantly reduce the delay of task processing, improve the response speed, and optimize the resource utilization of the edge server. In the above method, by combining the initial model, centralized training and task offloading prediction, the edge server can effectively respond to dynamically changing power grid processing tasks. Through global collaborative learning in the centralized training phase, edge servers can optimize resource allocation and avoid waste and uneven utilization of computing resources. In the distributed execution phase, servers can make fast and accurate unloading decisions based on real-time status, reduce processing delays, and improve the overall operating efficiency and stability of the power grid system. This solves the problems of high communication delay and high energy consumption in the traditional centralized computing model, while avoiding the low resource utilization of edge servers in task processing.

[0077] In an exemplary embodiment, Figure 3As shown, the above step S202 cooperates with other edge servers and the central cloud server to centrally train the initial policy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information to obtain the trained target policy network model, which can also be achieved by the following steps:

[0078] Step S301, iteratively train the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and candidate value network model.

[0079] Step S302: sending the model parameters to the central cloud server, and receiving the aggregated model parameters returned by the central cloud server.

[0080] Among them, the uploaded model parameters are the model parameters of the candidate policy network model and the candidate value network model; the aggregated model parameters are the model parameters determined by the central cloud server based on the uploaded model parameters of all edge servers.

[0081] Step S303, determine a new initial strategy network model and a new initial value network model according to the aggregation model parameters, and jump to the step of iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and the candidate value network model, until the target strategy network model that meets the first preset training completion condition is obtained.

[0082] Exemplarily, first, the edge server randomly selects a task from the sample power grid processing task information, and inputs it into the initial policy network model and the initial value network model in combination with the current environmental information. The edge server uses optimization algorithms such as gradient descent to gradually adjust the model parameters through multiple rounds of iterative training, so that the model can better adapt to the unloading requirements of the current task. After each iteration, the edge server calculates the corresponding reward value according to the unloading strategy output by the model, and updates the evaluation results of the model through the value network until the model converges and generates candidate policy network models and candidate value network models. Then, the edge server sends the model parameters (such as weights and bias values) of the candidate policy network model and the candidate value network model as the model parameters sent to the central cloud server; the central cloud server receives the model parameters from all edge servers and uses an aggregation algorithm (such as average or weighted average) to calculate a global optimal model parameter set. Then, the central cloud server returns the aggregated model parameters to all edge servers to ensure that the next round of training is based on a unified global perspective. Next, after receiving the aggregated model parameters, the edge server will apply them to the local initial policy network model and initial value network model to update the initial state of the model. Subsequently, the edge server continues to perform the training process in step S301, but this time the initial model has been globally optimized for parameter adjustment, thereby improving the training efficiency and prediction accuracy of the model. This cycle will continue until the target policy network model of all edge servers reaches the preset training completion conditions, such as reaching a certain prediction accuracy or reaching the maximum number of iterations, then all edge servers can determine the same target policy network model.

[0083] In this embodiment, through iterative training, the strategy and value model of each edge server are gradually optimized to better adapt to specific task offloading scenarios. Through the aggregation process of the central cloud server, the training results of each edge server are integrated into a globally optimal model parameter set, and the collaborative optimization mechanism ensures the coordination and consistency of the model optimization on a global scale. Further through a new round of iterative training, the accuracy and adaptability of the model are continuously improved until the preset training goals are achieved. Through a combination of centralized training and distributed optimization, each edge server can learn the optimal task offloading strategy, thereby improving the efficiency of task processing and resource utilization in the power grid system.

[0084] In an exemplary embodiment, Figure 4 As shown, the above step S301 iteratively trains the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and candidate value network model, which can also be implemented by the following steps:

[0085] Step S401, using the initial policy network model as the current policy network model, and using the initial value network model as the current value network model;

[0086] Step S402, inputting the sample power grid processing task information and the state information of the edge server into the current policy network model, obtaining the first sample task unloading prediction information, and obtaining the second sample task unloading prediction information of other edge servers;

[0087] Step S403, obtaining a sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model, and adjusting the model parameters of the current value network model according to the sample action value function value to obtain a new current value network model;

[0088] Step S404, input the sample task offloading prediction information into the new current value network model to obtain a new sample action value function value, and adjust the model parameters of the current policy network model according to the new sample action value function value to obtain a new current policy network model;

[0089] Step S405, obtain new current training environment information, and jump to the step of inputting the power grid processing task information and the edge server status information into the current strategy network model to obtain sample task unloading prediction information, until a candidate strategy network model and a candidate value network model that meet the second preset training completion condition are obtained.

[0090] Exemplarily, the edge server first uses the loaded initial policy network model as the current policy network model, and uses the initial value network model as the current value network model. These two models will be continuously updated and optimized in the subsequent iterative training process to better adapt to the unloading requirements of the power grid processing tasks. Then, the edge server inputs the currently received power grid processing task information and its own state information into the current policy network model, calculates and generates the first sample task unloading prediction information. At the same time, the edge server also obtains the second sample task unloading prediction information generated by other edge servers through network communication. The first sample task unloading prediction information is the unloading decision prediction generated by the current policy network model of the edge server based on the input information, and the second sample task unloading prediction information is the unloading decision prediction generated by the current policy network model of each other edge server based on the input information. Then, the edge server inputs the first sample task unloading prediction information, the second sample task unloading prediction information and the current training environment information into the current value network model. The current value network model will calculate the sample action value function value of each possible action based on these input information. The sample action value function value is calculated by the current value network model and is used to evaluate the expected return of performing a specific action under the current policy. Next, the edge server uses these value function values ​​and an optimization algorithm to adjust the parameters of the current value network model, and finally obtains an updated value network model that can more accurately evaluate the effect of the offloading strategy. Next, the edge server applies the new current value network model to the current sample task offloading prediction information and recalculates the sample action value function value. Then, the edge server uses these updated value function values ​​to further adjust the parameters of the current policy network model to improve its task offloading decision strategy. This process will be repeated until the policy network model reaches the predetermined performance standard. After each round of iteration, the edge server will collect the current training environment information, including the latest resource status of itself and other servers. Then, the edge server will input this new environment information into the updated policy network model, generate task offloading prediction information again, and continue to optimize the policy and value network model through the above process. This cycle will continue until the second preset training completion condition is met, such as the number of model iterations reaches the requirement, or the model parameters no longer change significantly. Finally, the edge server will obtain the trained candidate policy network model and candidate value network model, ready to be used in actual task offloading.

[0091] In this embodiment, the value network model is gradually optimized through sample task information and sample task unloading prediction information of other servers, so that it can more accurately evaluate the effects of different task unloading strategies. Based on the adjusted value network model, the strategy network model is adjusted so that its decision can be closer to actual needs. The iterative process enables the edge server to make more accurate and efficient unloading decisions when facing complex and dynamically changing power grid processing tasks, significantly improving the system's resource utilization and task processing efficiency.

[0092] In an exemplary embodiment, the above step S402 inputs the sample power grid processing task information and the status information of the edge server into the current policy network model to obtain the first sample task unloading prediction information, and also includes: obtaining the preset constraint conditions for task unloading of the power grid processing task; determining the constraint judgment information corresponding to the preset constraint conditions based on the sample power grid processing task information and the status information of the edge server; and obtaining the first sample task unloading prediction information that meets the preset constraint conditions based on the constraint judgment information, the preset constraint conditions, the sample power grid processing task information and the status information of the edge server.

[0093] Among them, constraint judgment information refers to the parameter values ​​used to evaluate whether the preset constraint conditions can be met, including at least the current network bandwidth, estimated transmission delay, queuing delay, processing delay, available computing resources of the edge server, current task queue length, etc.

[0094] Exemplarily, the edge server first obtains the preset constraints corresponding to a specific type of power grid processing task from a predefined parameter library. For example, for a real-time power grid control task requiring low latency, the power grid system may preset a relatively strict upper limit of latency as a constraint to ensure that the task can be completed within the specified time. Then, the edge server extracts or calculates parameter information for comparison with the preset constraints based on the sample power grid processing task information and the state information of the edge server. After obtaining the constraint judgment information, the edge server needs to compare these parameters with the preset constraints to evaluate whether the task unloading prediction generated by the current policy network model can meet these constraints. If it cannot be met, the unloading strategy needs to be adjusted, which may include selecting other edge servers for unloading, or adjusting the processing order of tasks, etc., to ensure that the preset constraints are met. Finally, after the above evaluation and adjustment, the first sample task unloading prediction information generated by the edge server will contain an unloading strategy that meets the preset constraints, such as the estimated total latency of the task, the expected energy consumption, etc.

[0095] In this embodiment, by introducing comprehensive consideration of preset constraints and constraint judgment information, the feasibility and effectiveness of different unloading strategies can be more accurately evaluated when generating task unloading prediction information. This ensures that the task not only considers resource utilization efficiency during unloading, but also strictly abides by business requirements and system performance limitations. This greatly improves the responsiveness and reliability of the edge computing system in power grid processing tasks and meets the needs of various complex business scenarios.

[0096] In an exemplary embodiment, the above-mentioned step S403 obtains the sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model, and also includes: obtaining a preset objective function for task unloading of the power grid processing task; determining the objective function information corresponding to the preset objective function according to the first sample task unloading prediction information, the second sample task unloading prediction information and the current training environment information; obtaining the sample action value function value according to the objective function information, the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model.

[0097] The preset objective function refers to the objective function defined to optimize the task offloading strategy, which is used to guide the edge server to weigh different factors when processing tasks. The objective function can be a single objective or a combination of multiple objectives, such as minimizing task processing latency, minimizing energy consumption, maximizing resource utilization, etc.

[0098] The objective function information refers to the relevant parameter values ​​used to calculate the preset objective function value.

[0099] Exemplarily, the edge server obtains a preset objective function, and then, to calculate the function value of the preset objective function, the edge server needs to obtain objective function information such as the transmission delay, processing delay, and power consumption model of each edge server of the current task. The sample action value function value is calculated based on the objective function information and the current value network model to evaluate the value of each possible task offloading strategy. The value network model generates a value score corresponding to each strategy by combining the task offloading prediction information and the training environment information, indicating the pros and cons of the strategy.

[0100] In this embodiment, by introducing a comprehensive analysis of the preset objective function and its related parameters, the value of different offloading strategies can be evaluated more accurately. This ensures the scientific nature and optimization effect of task offloading decisions, so that the edge server not only considers the real-time status of resources, but also allocates computing resources and network bandwidth in an optimal way, thereby achieving the best balance between performance and energy efficiency when processing tasks, thereby significantly improving the efficiency and reliability of power grid task processing.

[0101] In an exemplary embodiment, the above-mentioned determination of the objective function information corresponding to the preset objective function based on the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information also includes: determining the task completion delay information and the task energy consumption information for the sample power grid processing task information respectively based on the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information; and using the task completion delay information and the task energy consumption information as the objective function information.

[0102] Among them, the task completion delay information refers to the total time required for the task to be executed from the beginning to the completion under the current task offloading strategy. It usually includes a combination of multiple time factors such as transmission delay, queuing delay, and processing delay. Task energy consumption information refers to the total amount of energy consumed when completing the task, which is usually determined by the computing resources required to process the task, the execution time of the task, and the power consumption characteristics of the device.

[0103] Exemplarily, the transmission delay of a task can be determined by the network bandwidth and the size of the data packet; the queuing delay depends on the current task queue length of the edge server; and the processing delay is related to the computing power of the edge server and the complexity of the task. The energy consumption information is closely related to the computing resources and server power consumption required to perform the task. If the transmission delay of the task is 0.3 seconds, the queuing delay is 0.2 seconds, and the processing delay is 0.8 seconds, the task completion delay information is 1.3 seconds. If the power consumption of the server is 50 watts / second, the task energy consumption information can calculate the corresponding energy consumption based on the processing time and power consumption. The edge server uses the task completion delay information and the task energy consumption information as the objective function information, so that the objective function value can be obtained by weighted calculation.

[0104] In this embodiment, by separately calculating the task completion delay information and task energy consumption information and using them as the objective function information, it is ensured that the evaluation of the task offloading strategy is more scientific and practical, and can comprehensively consider the time efficiency and energy consumption during task processing, thereby selecting the optimal offloading strategy for the power grid system, effectively improving the resource utilization efficiency and operation performance of edge computing in the power grid system.

[0105] In another exemplary embodiment, the present application provides a task unloading prediction method for power grid processing tasks, and the specific process of the method is as follows:

[0106] Step 1: Users establish a cloud-edge collaborative task scenario and build a resource deployment model. A represents the user set, which consists of n(a) mobile terminal users a, A = {a1, a2, ···, a n(a)}, the mobile terminal users are independent of each other. J represents the task set, which consists of n(j) tasks j, J = {j1, j2, ···, j n(j)Mobile terminal users generate tasks with different computing requirements according to their needs. Task j has three dimensions of information: Among them, α j represents the amount of data for task j; β j represents the computational effort of task j, measured by the number of CPU clock cycles required to execute each unit of task data, and different types of tasks have different computational efforts; d j represents the maximum completion deadline of task j. B represents the set of edge devices, the number of which is n(b), B={b1,b2,···,b n(b)}. Edge devices have five dimensions of information: Among them, χ b Indicates the maximum computational effort of edge device b, measured by the maximum clock frequency of the device CPU; δ b represents the wired transmission power of edge device b; represents the task execution energy efficiency coefficient of edge device b, which is the ratio of the energy required for task execution to the task execution time; γ b represents the maximum wireless link bandwidth of edge device b; ν b represents the maximum wired link bandwidth of edge device b. In the cloud-edge collaborative computing model, the user's tasks are sent directly to the local edge device, and other edge devices play a collaborative role. It should be noted that tasks of different users can be exchanged and superimposed between edge devices. Y is used to represent the task offloading location set, that is, Y = {y1,y2,···,y n(j) Among them, the offloading location of task j is divided into the following three cases: First, the task can be executed on the local edge device, which is expressed as Secondly, the task can be offloaded from the local edge device to other collaborative edge devices for execution, which is expressed as Finally, the task can be offloaded from the local edge device to the cloud center for execution, which is represented as

[0107] Step 2: Considering the transmission and execution process of the task, the delay in completing the task includes the task transmission delay and the execution delay.

[0108] First consider the task transmission delay:

[0109] In the cloud-edge collaboration scenario, mobile terminal users use multiple access (OFDMA (Orthogonal Frequency Division Multiple Access), MIMO (multiple input multiple output)) methods to communicate with the nearest edge device, and the uplink data transmission rate is ν ab , expressed as:

[0110]

[0111] Among them, γ a,b The wireless link bandwidth allocated to the edge device for the mobile terminal user, δ a,b is the wireless transmission power between the mobile terminal user and the edge device, g a,b is the channel gain coefficient, σ represents the background noise power, ∑ a′∈A δ a′ ·g a′,b represents the wireless interference signal power of other mobile terminal users communicating with the edge device. The wired link bandwidth of the edge device is ν b ,The uplink data transmission rate is limited by many factors, including the wired link bandwidth, wireless network limitations, signal transmission distance and interference, and the resource limitations of mobile devices. ab <<v b .

[0112] The transmission delay for a mobile terminal user to complete task processing includes the delay in sending the task request and the delay in returning the task result. The request sending delay for task j is expressed as:

[0113]

[0114] Among them, α j / v a,b represents the transmission delay of the mobile terminal user sending task j, represents the transmission delay of task j from the local edge device to other collaborative edge devices, represents the transmission delay of the local edge device sending task j to the cloud center. When task j is executed on the local edge device That When the local edge device offloads task j to other edge devices for execution That When the local edge device offloads task j to the cloud center for execution That Since the downlink data transmission rate from the edge device or cloud center back to the mobile terminal user is much greater than the uplink data transmission rate of the mobile terminal user, and the task return result is much smaller than the amount of data sent by the task, this embodiment does not consider the task result return delay.

[0115] Secondly, there is the task execution delay:

[0116] The tasks generated by mobile terminal users are usually received and processed by edge devices. Due to the computing and storage capacity limitations of edge devices, tasks may not be completed locally. In this case, the local edge device will dynamically offload the task to the cloud or adjacent edge devices to provide more efficient and reliable task processing capabilities. Task execution delay of task j for:

[0117]

[0118] in, represents the execution latency of task j offloaded to the edge device. represents the amount of computation assigned to task j by the edge device, and there exists That is, it cannot exceed the maximum computing capacity of the edge device χ b . represents the execution delay of task j offloaded to the cloud center. Since the cloud center has high performance and powerful computing power, the execution delay of tasks is smaller than that of edge devices, that is, Less than

[0119] Step three, considering the transmission and execution of tasks, the energy consumption of completing tasks includes two aspects: task transmission and task execution.

[0120] The first is the task transmission energy consumption:

[0121] The task transmission delay is affected by multiple factors such as network bandwidth, network delay, data volume and data transmission method, and can be expressed as

[0122]

[0123] When task j is executed on the local edge device Its task transmission energy consumption It is related to the wireless transmission power of the device and the transmission delay of task j, expressed as δ a,b α j / v u,b When task j is offloaded from the local edge device to other edge devices or cloud centers for collaborative execution Considering δ a,b α j / v u,b On this basis, it is also necessary to consider the task transmission energy consumption of the local edge device sending task j to the collaborative edge device or cloud center, which are and

[0124] The second is the energy consumption of task execution:

[0125] The energy consumption of task execution is related to whether it is executed on an edge device or a cloud center. For task execution on edge devices, the energy consumption can be calculated by considering the dynamic power consumption of the CPU. When task j is executed on a local edge device or a collaborative edge device Its energy consumption is related to the computational amount of task j α j β j , the amount of computation allocated by the edge device to task j And the energy efficiency coefficient of task execution of edge devices When task j is executed in the cloud center Its energy consumption and the execution delay D of task j j c and the task execution power of the cloud center Therefore, the task execution energy consumption of task j is:

[0126]

[0127] Step 4: Establish an optimization target for cloud-edge collaborative resource deployment based on task prediction. In cloud-edge collaborative computing, task requests are sent to edge devices with optimal channel conditions. When processing tasks, edge devices need to consider task delays and power consumption, and formulate task offloading strategies based on factors such as device resource consumption and task load, so as to decide whether to execute the task on the local edge device or send it to other edge devices or cloud centers for collaborative execution. For tasks executed on edge devices, based on step 3, if the edge device increases the computing resources allocated to the task, while reducing the execution delay, it will also increase the energy consumption of task execution. Therefore, when optimizing the task offloading strategy, it is necessary not only to fully consider the execution delay and energy consumption overhead of the task, but also to consider the load balancing of edge devices, so as to provide high-quality services and minimize energy overhead.

[0128] Considering that computing tasks are latency-sensitive, in order to ensure the quality of consumer service, it is necessary to meet the maximum time limit constraints of the tasks while completing the tasks. The task completion delay is a key indicator used to measure the response speed and efficiency of the system. For latency-sensitive tasks, if the task execution time exceeds the user's maximum tolerance time limit, it may lead to a decline in user experience or fail to meet the requirements of real-time applications. Therefore, it is necessary to fully consider the time overhead of task queuing and task transmission during task execution. The task completion delay is expressed as:

[0129]

[0130] in, Indicates the time the task waits in the queue for execution. Indicates the time when the task is executed on the edge / cloud. It represents the time to transfer the task from the terminal device to the edge / cloud. The total task completion delay (TTCD) is the sum of all task completion times. The smaller the total task completion delay, i.e., D o The smaller the value, the better the user's application service experience.

[0131] In order to reduce the overall energy consumption of the system, it is also necessary to consider the total task energy consumption (Total Task Energy Consumption, TTEC), which is expressed as:

[0132]

[0133] Step 5: Establish the task offloading optimization problem for the cloud-edge collaborative computing scenario. Based on the expressions of latency and energy consumption in step 4, the optimization goal is to minimize the latency and energy consumption system utility value. The system utility value mainly focuses on the user service quality and the total energy consumption of the cloud-edge collaborative computing. The former depends on the total latency to complete the task, and the latter depends on the total energy consumption to complete the task. The system utility value is represented by ψ. Generally speaking, the smaller the total latency to complete the task and the lower the total energy consumption, the lower the system utility value ψ. The problem is modeled as:

[0134]

[0135] By normalizing the total task delay D o and total task energy consumption E o Weighted linear superposition is performed to obtain the system utility value. C1 represents the weight coefficient ω of the total delay in task completion D and the weight coefficient of total energy consumption ω E and the sum of the two is 1. C2 states that the task completion delay cannot exceed the maximum completion deadline of the task. Through this constraint, it can be ensured that the task can be completed within the expected time to meet the user's timeliness requirements for the task. C3 describes that the wireless link bandwidth between the mobile terminal user and the edge device cannot be negative, and must be less than or equal to the maximum wireless link bandwidth of the edge device. C4 defines that the amount of computing power allocated by the edge device to the task cannot be negative, and must be less than or equal to the maximum computing power of the edge device, which is used to limit the computing power of the edge device when processing tasks and ensure that the execution of the task does not exceed the processing power of the edge device. C5 and C6 respectively state that the data transmission rate when the task is dispatched from the local edge device to other collaborative edge devices or cloud centers cannot be negative, and must be less than or equal to the wireless link bandwidth of the edge device. C7 states that the tasks generated by the mobile terminal user can only be unloaded at one location among the local edge device, collaborative edge device or cloud center, which is used to determine the location where the task is generated so as to reasonably allocate tasks and utilize system resources.

[0136] Step 6: Based on the optimization problem in step 5, the multi-agent proximal policy optimization (MAPPO) algorithm is used to solve the task offloading problem. Each edge device and cloud center deploys a proximal policy optimization (PPO) network and shares network parameters. By interacting and learning with the environment, the edge device can select the optimal action based on the output of the policy network to maximize the cumulative reward. Since the problem under study is the task offloading decision, the state set needs to include the total number of CPU cycles c required for the current task. i,t , task size d i,t and maximum tolerable delay There are three ways to offload tasks. The state set also needs to include the CPU clock frequency F of the cloud server C. C and the CPU clock frequency F of edge server E E and the CPU resources consumed E cpu 、Memory resources E mem and Communication Resources com ; Since both edge server E and cloud server C maintain task queues, the state set also needs to know the length of the task queue, which is denoted by L E,t and L C,t , where L E,t represents the length of the task queue of edge server E at time t. Therefore, the state set represents:

[0137]

[0138] Since there are only three ways to make offloading decisions: completing task calculations on local edge devices, other edge devices, or cloud centers, if offloading to other edge devices, it is also necessary to consider which edge device to offload to. Therefore, the action space is expressed as:

[0139] Since the optimization goal is to minimize the weighted sum of the total time and total energy consumption to complete the task, the reward function needs to take these two aspects into consideration, and both values ​​must be positive. In order to ensure that the reward function is positive and the reward value is positive, this chapter will use an exponential function to define the reward function. In this exponential function, the system utility refers to the base of the exponent, that is:

[0140]

[0141] Specifically, MAPPO adopts a centralized training and decentralized execution (CTDE) architecture to improve learning efficiency, promote collaborative learning, reduce communication overhead, and improve the flexibility and robustness of the system. Such an architecture enables each agent to more effectively select task offloading nodes, thereby significantly improving the performance of the system. Specifically, each edge server is configured with a separate actor-critic (policy-value) network, and the Critic network (value network) is used to estimate the value function of the environment state, that is, to evaluate the behavior of each individual actor network (policy network). The entire algorithm consists of two stages: centralized training and distributed execution. In the centralized training stage, each edge server's own actor network accepts scores from each critic network. After a certain training step, the edge server sends its model parameters to the cloud center. The cloud center is responsible for aggregating the model parameters of all edge servers and storing the averaged results in its own model. In the distributed execution stage, the actor network of each edge server no longer communicates with the critic network. Instead, it can be based on its own local observations. i and the shared strategy π θ (a i |o i ) Generate action a i , to maximize the discounted cumulative reward:

[0142]

[0143] Here, γ is the discount factor.

[0144] The PPO algorithm solves the problem that the policy gradient algorithm is sensitive to the number of iterative steps and it is difficult to determine a reasonable step size. It limits the update of the new strategy by truncation. θ The policy network of the actor network learns a mapping from observations A distribution to a range or a mean and variance of actions mapped to a Gaussian function for subsequent sampling of actions. φ The value network of the critic network evaluates the actions output by the actor network and needs to learn a mapping: Where θ and φ represent the parameters of the policy network and the value network respectively. The truncated surrogate loss function is defined as:

[0145]

[0146] Among them, ρ t is the importance sampling, which represents the ratio of the new strategy to the old strategy:

[0147]

[0148] is the estimate of the advantage function at time t:

[0149]

[0150] Use the gradient descent method to update the network parameters φ of the critic network:

[0151] L VF (φ)=E t [(V φ (s t )-y(t)) 2 ]

[0152] Where y(t) = r t +γV φ (s t+1 ).

[0153] In addition, entropy addition X[π θ ] to enhance the overall objective function to promote better algorithm performance. The overall objective function is:

[0154] L total =E t [L CLIP (θ)-c1L VF (φ)+c2X[π θ ](s t )]

[0155] Where c1 and c2 are coefficients, c2X[π θ ](s t ) represents the input state s t The entropy of the provided strategy. The overall objective function can be used to determine the conditions for completing iterative training, such as when the function value of the overall objective function reaches a threshold or when the function value of the overall objective function after multiple consecutive iterations changes below a threshold.

[0156] Then, in the centralized training phase, the critic network of each edge server calculates the action-value function Q(s,a1,a2,···,a u |φ). The Q function evaluates the offloading actions of the edge server from a global perspective and guides it to choose a better action. The critic network then updates the parameter φ by minimizing the loss c :

[0157] loss(φ c )=E[Q(s,a1,a2,···,a u |φ)-yMAPPO ] 2

[0158] Among them, the objective function y MAPPO for:

[0159] y MAPPO =r t +γQ′(s′,a1′,a′2,···,a′ u |φ′)

[0160] Where, s=(s1,s2,······,s u ), φ is the parameter of the critic network, s' represents the next state of s, and φ′ is the updated network parameter of the critic network. On the other hand, the actor network updates the network parameter θ and outputs the action according to the Q function calculated by the critic network and its own observation results. The actor network calculates the policy gradient ▽ θ J(θ) and update the network parameters θ:

[0161] ▽θJ(θ)≈E[▽ a Q(s,a1,a2,···,a u |φ)▽θπ(s|θ)]

[0162] In the distributed execution phase, the agent makes decisions based only on local observations from edge devices, thereby obtaining close to the global optimal action without knowing the information of other agents, which greatly reduces time consumption and computing resource consumption.

[0163] In a specific example, the effect of the method of the embodiment of the present invention is verified through simulation experiments:

[0164] Simulation conditions: Design two network services. One network service provides edge computing services, which are computationally intensive services and have higher requirements for CPU. The other network service provides edge caching services, which are high-throughput services and have higher requirements for memory and network resources. Specifically, edge computing services are large-scale matrix operations, and edge caching services are video resource requests. Through a series of performance indicator tests, the advantages of the proposed MAPPO algorithm in solving task offloading problems are verified. Figure 5 It is a structural diagram of the multi-agent proximal strategy optimization algorithm.

[0165] The model parameters obtained after model training are shown in Table 1:

[0166] Table 1 Model parameters

[0167]

[0168] In order to better demonstrate the performance of the MAPPO algorithm proposed in this embodiment, the total task completion time and total energy consumption corresponding to the distributed proximal policy optimization (DPPO) algorithm and the PPO algorithm are compared. Figure 6 It can be found that after the number of training episodes reaches 100, the model begins to converge, and the MAPPO algorithm performs better than the DPPO and PPO algorithms in reducing the total task completion time and total energy consumption.

[0169] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0170] Based on the same inventive concept, the embodiment of the present application also provides a task unloading prediction device for power grid processing tasks for implementing the task unloading prediction method for power grid processing tasks involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiment of the task unloading prediction device for one or more power grid processing tasks provided below can be referred to the limitations of the task unloading prediction method for power grid processing tasks above, and will not be repeated here.

[0171] In an exemplary embodiment, Figure 7 As shown, a task unloading prediction device for power grid processing tasks is provided, comprising: a data acquisition module 701, a model training module 702, a real-time application module 703 and an unloading prediction module 704, wherein:

[0172] The data acquisition module 701 is used to acquire the initial strategy network model and the initial value network model, as well as the sample power grid processing task information and the current training environment information; the current training environment information at least includes the status information of the edge server, the status information of other edge servers and the status information of the central cloud server;

[0173] The model training module 702 is used to coordinate with other edge servers and the central cloud server to centrally train the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information to obtain the trained target strategy network model;

[0174] The real-time application module 703 is used to obtain the current status information of the edge server when receiving a new power grid processing task;

[0175] The unloading prediction module 704 is used to input the current state information and the task information of the new power grid processing task into the target strategy network model to obtain the task unloading prediction information for the new power grid processing task.

[0176] In one embodiment, the above-mentioned model training module 702 is also used to iteratively train the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and the candidate value network model; send the uploaded model parameters to the central cloud server, and receive the aggregated model parameters returned by the central cloud server; the uploaded model parameters are the model parameters of the candidate strategy network model and the candidate value network model; the aggregated model parameters are the model parameters determined by the central cloud server based on the uploaded model parameters of all edge servers; according to the aggregated model parameters, determine a new initial strategy network model and a new initial value network model, and jump to the step of iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain the trained candidate strategy network model and the candidate value network model, until the target strategy network model that meets the first preset training completion condition is obtained.

[0177] In one embodiment, the model training module 702 is also used to use the initial policy network model as the current policy network model, and the initial value network model as the current value network model; input the sample power grid processing task information and the status information of the edge server into the current policy network model to obtain the first sample task unloading prediction information, and obtain the second sample task unloading prediction information of other edge servers; obtain the sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model, and adjust the model parameters of the current value network model according to the sample action value function value to obtain a new current value network model; input the sample task unloading prediction information into the new current value network model to obtain a new sample action value function value, and adjust the model parameters of the current policy network model according to the new sample action value function value to obtain a new current policy network model; obtain the new current training environment information, and jump to the step of inputting the power grid processing task information and the status information of the edge server into the current policy network model to obtain the sample task unloading prediction information, until the candidate policy network model and the candidate value network model that meet the second preset training completion condition are obtained.

[0178] In one embodiment, the above-mentioned model training module 702 is also used to obtain preset constraints for task unloading of power grid processing tasks; determine constraint judgment information corresponding to the preset constraints based on sample power grid processing task information and edge server status information; and obtain first sample task unloading prediction information that meets the preset constraints based on the constraint judgment information, preset constraints, sample power grid processing task information and edge server status information.

[0179] In one of the embodiments, the above-mentioned model training module 702 is also used to obtain a preset objective function for task unloading of power grid processing tasks; determine the objective function information corresponding to the preset objective function based on the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information; obtain the sample action value function value based on the objective function information, the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model.

[0180] In one of the embodiments, the above-mentioned model training module 702 is also used to determine the task completion delay information and task energy consumption information for the sample power grid processing task information based on the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information; and use the task completion delay information and task energy consumption information as objective function information.

[0181] Each module in the task offloading prediction device for the above-mentioned power grid processing task can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0182] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data required for training and parameter data of the target model. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a task unloading prediction method for a power grid processing task is implemented.

[0183] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0184] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0185] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0186] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0187] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0188] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0189] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0190] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task offloading prediction method for power grid processing tasks, characterized in that: Applied to an edge server, the method comprises: Acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information includes at least the status information of the edge server, the status information of other edge servers and the status information of the central cloud server; Cooperating with the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model; Upon receiving a new power grid processing task, obtaining current status information of the edge server; The current state information and the task information of the new power grid processing task are input into the target strategy network model to obtain task unloading prediction information for the new power grid processing task.

2. The method according to claim 1, characterized in that The coordinating the other edge servers and the central cloud server, centrally training the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, to obtain a trained target strategy network model, includes: Iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain a trained candidate strategy network model and a candidate value network model; Send the uploaded model parameters to the central cloud server, and receive the aggregated model parameters returned by the central cloud server; the uploaded model parameters are the model parameters of the candidate policy network model and the candidate value network model; the aggregated model parameters are the model parameters determined by the central cloud server based on the uploaded model parameters of all edge servers; According to the aggregation model parameters, a new initial strategy network model and a new initial value network model are determined, and the step of iteratively training the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain trained candidate strategy network models and candidate value network models is jumped to, until a target strategy network model that meets the first preset training completion condition is obtained.

3. The method according to claim 2, characterized in that The iterative training of the initial strategy network model and the initial value network model according to the sample power grid processing task information and the current training environment information to obtain a trained candidate strategy network model and a candidate value network model includes: Using the initial policy network model as the current policy network model, and using the initial value network model as the current value network model; Inputting the sample power grid processing task information and the state information of the edge server into the current policy network model to obtain first sample task unloading prediction information, and obtaining second sample task unloading prediction information of the other edge servers; Obtaining a sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information, and the current value network model, and adjusting a model parameter of the current value network model according to the sample action value function value to obtain a new current value network model; Inputting the sample task offloading prediction information into the new current value network model to obtain a new sample action value function value, and adjusting the model parameters of the current policy network model according to the new sample action value function value to obtain a new current policy network model; Obtain new current training environment information, and jump to the step of inputting the power grid processing task information and the status information of the edge server into the current strategy network model to obtain sample task unloading prediction information, until a candidate strategy network model and a candidate value network model that meet the second preset training completion condition are obtained.

4. The method according to claim 3, characterized in that The step of inputting the sample power grid processing task information and the edge server status information into the current policy network model to obtain first sample task offloading prediction information includes: Obtaining preset constraints for task offloading of power grid processing tasks; Determining constraint judgment information corresponding to the preset constraint condition according to the sample power grid processing task information and the state information of the edge server; According to the constraint judgment information, the preset constraint conditions, the sample power grid processing task information and the state information of the edge server, first sample task offloading prediction information that meets the preset constraint conditions is obtained.

5. The method according to claim 3, characterized in that: The obtaining of the sample action value function value according to the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model includes: obtaining a preset objective function for task offloading of a power grid processing task; Determining objective function information corresponding to the preset objective function according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information; A sample action value function value is obtained according to the objective function information, the first sample task unloading prediction information, the second sample task unloading prediction information, the current training environment information and the current value network model.

6. The method according to claim 5, characterized in that The determining, according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information, objective function information corresponding to the preset objective function includes: Determine task completion delay information and task energy consumption information for the sample power grid processing task information respectively according to the first sample task unloading prediction information, the second sample task unloading prediction information, and the current training environment information; The task completion delay information and the task energy consumption information are used as the objective function information.

7. A task unloading prediction device for power grid processing tasks, characterized in that: The device comprises: A data acquisition module, used to acquire an initial strategy network model and an initial value network model, as well as sample power grid processing task information and current training environment information; the current training environment information at least includes the status information of the edge server, the status information of other edge servers and the status information of the central cloud server; A model training module, used to coordinate with the other edge servers and the central cloud server, centrally train the initial strategy network model and the initial value network model in the edge server according to the current training environment information and the sample power grid processing task information, and obtain a trained target strategy network model; A real-time application module, configured to obtain current status information of the edge server upon receiving a new power grid processing task; The unloading prediction module is used to input the current state information and the task information of the new power grid processing task into the target strategy network model to obtain the task unloading prediction information for the new power grid processing task.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images