Task processing method, device and system for photovoltaic edge computing node

Abstract

The application discloses a task processing method, device and system for photovoltaic edge computing nodes, and the task processing method comprises the following steps: generating a task processing queue of each photovoltaic edge computing node based on a user task unloading queue and a migration task receiving queue; taking the matching between the task processing capacity of each photovoltaic edge computing node and the current power supply capacity as a target, constructing a processor resource allocation optimization sub-problem and a task migration optimization sub-problem based on a pre-constructed service delay model, a service energy consumption model and an electric energy storage amount model; and jointly solving the processor resource allocation optimization sub-problem and the task migration optimization sub-problem by using a multi-agent reinforcement learning algorithm based on the task processing queue of each photovoltaic edge computing node, so as to obtain an optimized processor resource allocation strategy and a task migration strategy. The application improves the service performance of a photovoltaic edge computing node network system by controlling the task migration and processor resource allocation of the photovoltaic edge computing nodes.

Description

Technical Field

[0001] This invention belongs to the technical field of multi-agent reinforcement learning and photovoltaic edge computing node technology integration, specifically involving a task processing method, device and system for photovoltaic edge computing nodes. Background Technology

[0002] Photovoltaic edge computing nodes refer to edge computing nodes equipped with photovoltaic power supply systems. These nodes convert solar energy into electricity and store it in energy storage devices to power their communication, computing, and storage needs. Due to their inherent advantages, they have been increasingly deployed in remote mountainous areas, deserts, and polar regions in recent years, significantly alleviating the difficulties faced by traditional node deployments, such as the lack of mains power and the high cost of diesel generators. By deploying photovoltaic edge computing node network systems in complex and special environments, the dependence of mobile communication needs on mains power can be effectively reduced. However, since photovoltaic power supply systems primarily rely on solar energy conversion, photovoltaic edge computing nodes may experience a mismatch between their power supply capacity and communication / computing loads. In edge computing scenarios provided by photovoltaic edge computing nodes, excessive load on any single node may cause localized network outages, thus affecting the stability of the photovoltaic edge computing node network system. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a task processing method, apparatus, and system for photovoltaic edge computing nodes. By controlling task migration and processor resource allocation of photovoltaic edge computing nodes, the service performance of the photovoltaic edge computing node network system can be improved.

[0004] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0005] In a first aspect, the present invention provides a task processing method for photovoltaic edge computing nodes, applied to a photovoltaic edge computing node network system, comprising:

[0006] Based on the user task unloading queue and the migration task receiving queue, a task processing queue is generated for each photovoltaic edge computing node.

[0007] With the goal of matching the task processing volume of each photovoltaic edge computing node with the current power supply capacity, based on the pre-built service latency model, service energy consumption model and energy storage model, we construct processor resource allocation optimization sub-problems and task migration optimization sub-problems;

[0008] Based on the task processing queues of each photovoltaic edge computing node, a preset multi-agent reinforcement learning algorithm is used to jointly solve the processor resource allocation optimization sub-problem and the task migration optimization sub-problem, resulting in optimized processor resource allocation strategy and task migration strategy. The processor resource allocation strategy is used to determine whether each processor in each photovoltaic edge computing node is used to process tasks, and the task migration strategy is used to determine whether to migrate tasks in this photovoltaic edge computing node to other photovoltaic edge computing nodes.

[0009] Secondly, the present invention provides a task processing device for photovoltaic edge computing nodes, applied to a photovoltaic edge computing node network system, comprising:

[0010] The task processing queue generation module is used to generate task processing queues for each photovoltaic edge computing node based on the user task unloading queue and the migration task receiving queue.

[0011] The optimization problem construction module is used to construct processor resource allocation optimization sub-problems and task migration optimization sub-problems based on pre-built service latency model, service energy consumption model and energy storage model, with the goal of matching the task processing volume of each photovoltaic edge computing node with the current power supply capacity.

[0012] The task processing module is used to solve the processor resource allocation optimization sub-problem and the task migration optimization sub-problem jointly using a preset multi-agent reinforcement learning algorithm based on the task processing queues of each photovoltaic edge computing node, so as to obtain the optimized processor resource allocation strategy and task migration strategy. The task migration strategy is used to determine whether to migrate the tasks in this photovoltaic edge computing node to other photovoltaic edge computing nodes, and the processor resource allocation strategy is used to determine whether each processor in each photovoltaic edge computing node is used to process tasks.

[0013] Thirdly, the present invention provides a task processing system for photovoltaic edge computing nodes, including a storage medium and a processor;

[0014] The storage medium is used to store instructions;

[0015] The processor is configured to operate according to the instructions to perform the method according to any one of the first aspects.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0017] In this invention, the photovoltaic edge computing node network system achieves task load balancing among photovoltaic edge computing nodes by optimizing the allocation of processor resources and task migration among each photovoltaic edge computing node.

[0018] This invention addresses the photovoltaic edge computing node network system by comprehensively considering the task processing requirements and power supply status of each photovoltaic edge computing node to establish a joint optimization problem. By controlling the processors of each photovoltaic edge computing node, it achieves the balance of power storage of each node while meeting task latency requirements, thereby improving the stability of the network system.

[0019] This invention divides the joint optimization problem into two sub-problems (processor resource allocation optimization and task migration optimization). A multi-agent reinforcement learning optimization algorithm is designed to achieve online decision-making for processor resource allocation and task migration. Compared with traditional algorithms, it exhibits good convergence. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0021] Figure 1 The light intensity is 350W / m 2 A schematic diagram of the algorithm's convergence process at that time;

[0022] Figure 2 A comparison chart of the final battery levels of each node after running for 50 seconds under different light intensities using different optimization schemes;

[0023] Figure 3 A comparison chart of the final battery consumption of each node after running for 50 seconds under different optimization schemes for different arrival task volumes;

[0024] Figure 4 A comparison chart of the final battery consumption after each node runs for 50 seconds using different optimization schemes at different CPU single-core frequencies;

[0025] Figure 5 A comparison chart showing the normal operating time of network systems using different optimization schemes under different light intensities;

[0026] Figure 6 A comparison chart of the uptime of network systems using different optimization schemes under different arrival volumes;

[0027] Figure 7 A comparison chart of the uptime of network systems using different optimization schemes at different CPU single-core frequencies;

[0028] Figure 8 This is a flowchart of a task processing method for photovoltaic edge computing nodes in one embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0030] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0031] Example 1

[0032] This invention provides a task processing method for photovoltaic edge computing nodes (PCNs), applied to a photovoltaic edge computing node network system, such as... Figure 8 As shown, it includes the following steps:

[0033] (1) Based on the user task unloading queue and the migration task receiving queue, generate the task processing queue for each photovoltaic edge computing node;

[0034] (2) With the goal of matching the task processing volume of each photovoltaic edge computing node with the current power supply capacity, based on the pre-built service delay model, service energy consumption model and power storage model, construct processor resource allocation optimization sub-problem and task migration optimization sub-problem;

[0035] (3) Based on the task processing queue of each photovoltaic edge computing node, the processor resource allocation optimization sub-problem and the task migration optimization sub-problem are jointly solved by the preset multi-agent reinforcement learning algorithm to obtain the optimized processor resource allocation strategy and the task migration strategy. The processor resource allocation strategy is used to determine whether each processor in each photovoltaic edge computing node is used to process tasks, and the task migration strategy is used to determine whether to migrate the tasks in this photovoltaic edge computing node to other photovoltaic edge computing nodes.

[0036] In the above scheme, task load balancing of each photovoltaic edge computing node is achieved by optimizing the allocation of processor resources and task migration of each photovoltaic edge computing node.

[0037] In specific implementation, the task processing method for photovoltaic edge computing nodes (PCNs) includes the following steps:

[0038] Step 1: Build the PCN task processing queue

[0039] In one specific embodiment of the present invention, PCNs distributed in different locations continuously receive tasks uninstalled by the user. Definition For task number i, the PCN receives tasks and determines the order in which they are served by using a queue. Taking PCN n as an example, the queues it needs to maintain include the user task offloading queue. Transmission queue and migration task receive queue The PCN n user task unloading queue and migration task receiving queue together constitute the task processing queue. The queue follows a first-in, first-out (FIFO) rule, prioritizing the processing of tasks from the migration task receiving queue. The task processing queue... The expression is:

[0040] ,

[0041] In the formula, Photovoltaic edge computing node The migration task receiving queue stores historically received tasks migrated from other photovoltaic edge computing nodes. Photovoltaic edge computing node The user task unloading queue stores historically received user unloading tasks.

[0042] Step 2: Build a service latency model

[0043] Users will continuously generate tasks that need to be uninstalled. During the process of uninstalling these tasks to PCN for processing, corresponding service delays will occur. The specific methods for constructing the service delay model include:

[0044] Step 2-1: Construct a task processing delay model

[0045] PCN n allocates task processing queues based on the load status of its processors. The tasks in PCN n are processed. If there is an idle processor (in specific implementation, the processor can be a CPU), then the task processing queue... Tasks at the front of the queue are processed immediately without waiting. Define Task The delay for processing by the k-th processor of PCN n is The processing delay model for the task is obtained as follows:

[0046] ,

[0047] in, For the number Task via photovoltaic edge computing nodes The delay for the k-th processor to process, The computing frequency of the k-th processor is expressed in Hz. To complete the calculation task The number of processor cycles required (i.e., the number of CPU cycles).

[0048] Step 2-2: Construct the task's waiting delay model

[0049] If PCN n has no idle processors at this time, then the task processing queue... The first mission in the mission requires waiting for a while. The waiting time includes the task processing queue. Medium task The expected processing time of the preceding tasks and the tasks The preparation time from becoming the head of the queue to the start of processing. Define the task processing queue. Tasks in The waiting time in PCN n is The expression for the waiting delay model of the task is:

[0050] ,

[0051] in, For task processing queue Tasks in At photovoltaic edge computing nodes The waiting time in the middle, Represents photovoltaic edge computing nodes The remaining time for the k-th processor to complete the current task. , Represents the photovoltaic edge computing node in time slot t. The working state of the k-th processor. Represents the photovoltaic edge computing node in time slot t. The working state of the k-th processor. This indicates that a task is in progress. Indicates free time. Represents photovoltaic edge computing nodes In time slot t, the k-th processor is used to process the task. The decision, This indicates that PCN n will use the k-th processor to process the task in time slot t. ,otherwise, ; Represents the task processing queue Medium task The set of preceding tasks; Indicates completion of task Number of processor cycles required;

[0052] Steps 2-3: Construct a transmission delay model for the task

[0053] When PCN n is overloaded or energy storage is insufficient, the task It will enter the sending queue. The tasks are then migrated to other PCNs for processing. If the transfer queue is migrated If there are no other tasks, only the transmission time between PCNs needs to be considered. The transmission rate between PCN n and PCN m is defined as... Then the task The transmission time between PCN n and m is The expression for the transmission delay model of the task is:

[0054] ,

[0055] in, For the task At photovoltaic edge computing nodes Transmission time between the photovoltaic edge computing node m and the photovoltaic edge computing node m For the task The size of the data, Represents photovoltaic edge computing nodes The transmission rate between the photovoltaic edge computing node m and the photovoltaic edge computing node m , , These represent the minimum and maximum transmission rates between photovoltaic edge computing node n and P photovoltaic edge computing node m, respectively.

[0056] Steps 2-4: Constructing the task's waiting transmission delay model

[0057] Task If other tasks are in the transfer queue during migration, a waiting transfer delay will occur. The sending wait time is recorded as the task. The cumulative transmission time of the preceding tasks. From this, the task can be obtained. In the sending queue Waiting time The expression for the waiting transmission delay model of the task is:

[0058] ,

[0059] in, For the task In the sending queue The waiting time in the middle, Indicates the send queue Medium task Precedence task set; Send queue The construction method is as follows: when PCN n is overloaded or has insufficient energy storage, the task... It will enter the sending queue. ; For the task At photovoltaic edge computing nodes Transmission time between the photovoltaic edge computing node m and the photovoltaic edge computing node m;

[0060] Steps 2-5: Construct a service latency model for photovoltaic edge computing nodes (i.e., the total service latency of tasks).

[0061] The completed task calculation results are sent to the corresponding user via the appropriate PCN. Considering the small data volume of the calculation results, the return latency is negligible. Given the transmission latency, calculation latency, and waiting latency, the total service latency of the task needs to be comprehensively considered in relation to the PCN's decisions. Service delay is The service latency model expression is:

[0062] ,

[0063] in, For the task estimated in time slot t Total service delay; This indicates that the photovoltaic edge computing node n will use the k-th processor to process tasks in time slot t. The decision, This represents the total number of processors in photovoltaic edge computing node n; This indicates that the photovoltaic edge computing node n will use the k-th processor to process tasks in time slot t. ,otherwise, , This indicates that the photovoltaic edge computing node n will perform the task in time slot t. The decision to migrate to the photovoltaic edge computing node m This indicates that the photovoltaic edge computing node n will perform the task in time slot t. Migrate to the photovoltaic edge computing node m; otherwise... , This represents the total number of photovoltaic edge computing nodes. This indicates the task processed by the k-th processor of node m. Required time Indicates task The time required for processing in the processing queue of node m.

[0064] Step 3: Build a service energy consumption model

[0065] Tasks generate energy consumption at different stages of processing and transmission. Methods for constructing service energy consumption models include:

[0066] Step 3-1: Construct a task computation energy consumption model

[0067] If PCN chooses to process tasks locally, then energy consumption only needs to consider the computational energy consumption of the task. The computational energy consumption of a task is related to the processor's operating power and the processor cycles required to complete the task. The operating power of the k-th processor in PCN n is defined as... Its value is related to the processor architecture, and the expression for the task computing energy consumption model is:

[0068] ,

[0069] ,

[0070] in, For the task The computational energy consumption of the k-th processor in photovoltaic edge computing node n during time slot t. and To calculate the power parameters of the CPU architecture, The frequency of the k-th processor in the n-th photovoltaic edge computing node is given in Hz. express Time slot to complete the task Number of remaining processor cycles required The time slot length, Let be the operating power of the k-th processor in the photovoltaic edge computing node n;

[0071] Step 3-2: Construct a task migration energy consumption model

[0072] When PCN n is overloaded or has insufficient energy storage, the task can be... Migrate to another PCN for processing. Task migration will incur corresponding transmission energy consumption. Define the task. The transmission energy consumption from PCN n to PCN m in time slot t is: The expression for the task migration energy consumption model is:

[0073] ,

[0074] in, For the task The transmission energy consumption from photovoltaic edge computing node n to photovoltaic edge computing node m in time slot t. This represents the transmission power of photovoltaic edge computing node n. Indicates task The amount of untransmitted data remaining in time slot t;

[0075] Step 3-3: Construct a task service energy consumption model

[0076] Given the transmission and computational energy consumption, the service energy consumption of a task needs to be calculated based on the PCN decision, and task offloading based on PCN n. With migration decision For the task Service energy consumption required in time slot t The expression for the task service energy consumption model is:

[0077] ,

[0078] in, For the task The service energy consumption required in time slot t This represents the total number of processors in photovoltaic edge computing node n; This indicates that the photovoltaic edge computing node n will perform the task in time slot t. The decision to migrate to the photovoltaic edge computing node m; This indicates that the photovoltaic edge computing node n will use the k-th processor to process tasks in time slot t. Decision-making;

[0079] Steps 3-4: Constructing a service energy consumption model for photovoltaic edge computing nodes

[0080] The service energy consumption of each PCN requires calculating the total energy consumption of all tasks served by that PCN in time slot t. The expression for the service energy consumption model of the photovoltaic edge computing node is as follows:

[0081] ,

[0082] in, The total energy consumption for all tasks served by photovoltaic edge computing node n in time slot t. Photovoltaic edge computing node The task processing queue.

[0083] Step 4: Construct an energy storage model. The method for constructing the energy storage model includes:

[0084] Step 4-1: Construct a photovoltaic conversion model

[0085] PCN n photovoltaic power supply system power supply power Determined by factors such as light intensity, illuminated area, and illumination time, the expression for the photovoltaic conversion model is as follows:

[0086] ,

[0087] in, The power supply of the photovoltaic power supply system to the photovoltaic edge computing node n is given by η, where η represents the power conversion efficiency. This represents the area of ​​the photovoltaic panel receiving sunlight at photovoltaic edge computing node n. Let n be the intensity of solar radiation received by the photovoltaic edge computing node n in time slot t;

[0088] Step 4-2: Construct a model of the change in electrical energy for each photovoltaic edge computing node in each time slot.

[0089] Each photovoltaic power supply system of a PCN has a certain energy storage capacity, the amount of which depends on the energy consumption and charging time in each time slot. The change in energy of PCN n in time slot t is defined as... The expression for the power change model for each photovoltaic edge computing node in each time slot is:

[0090]

[0091] in, Let n be the change in electrical energy at photovoltaic edge computing node n in time slot t. The static power consumption of PCN n depends on the processor's operating state. The time slot length;

[0092] Step 4-3: Construct an energy storage model

[0093] PCN has a certain energy storage capacity. The amount of energy stored depends on the energy consumption and power generation in each time slot. The current energy storage capacity can be obtained by summing up the energy changes in each time slot. The expression for the energy storage capacity model is as follows:

[0094] ,

[0095] in, Let n be the amount of electrical energy stored by the photovoltaic edge computing node n in time slot t. Let n be the initial electrical energy storage capacity of the photovoltaic edge computing node n.

[0096] Step 5: Constructing the energy consumption optimization problem of PCN

[0097] In the network system described above, each PCN needs to comprehensively consider the amount of data to be processed and the amount of electrical energy to be stored. The key is to control the amount of data processed to match the current power supply capacity. The energy consumption optimization problem is modeled as follows:

[0098] Step 5-1: Set the minimum amount of task data to be processed per time slot for each PCN:

[0099] When the remaining power When fully charged, PCN n can be configured with more processors for task processing; when the remaining power is sufficient... At lower speeds, PCN n still needs to maintain a certain task processing capacity to ensure it doesn't crash and can complete current tasks on time. The minimum task processing capacity of PCN n in a non-crash state is defined as... The specific calculations are as follows:

[0100]

[0101] in, The low battery threshold is used to determine the battery level. The threshold for determining high battery level. This represents the maximum energy storage capacity of PCN n. and These represent the estimated task processing volume for PCN n based on the newly added power supply in time slot t and the load of the photovoltaic edge computing node network system, respectively. The calculation is as follows:

[0102]

[0103] in, For the task The complexity, that is, the number of processor cycles required to process each bit of task data. This indicates that the user will send the task in time slot t. The decision to unload to PCN n, if Then the user will send the task in time slot t. Uninstall to PCN n, otherwise, . The maximum duration for which the additional electrical energy of PCN n in time slot t can power the processor is sufficient to run is calculated using the following formula:

[0104]

[0105] in, The time taken for PCN n to transmit task data in time slot t is represented by the following formula:

[0106]

[0107] The calculation formula is:

[0108]

[0109] Step 5-2: Construct the constraints for the optimization problem:

[0110] The total service latency for the task must be lower than the maximum latency tolerance, that is:

[0111]

[0112] The amount of data processed by the task must meet a minimum limit over a long period of time, namely:

[0113]

[0114] in, The amount of data processed by PCN n in time slot t is specifically calculated as follows:

[0115]

[0116] At any given time slot, a CPU core can only be used to process one task, that is:

[0117]

[0118] Any time slot task Only one PCN can be migrated for processing, that is:

[0119]

[0120] Step 5-3: Establish a joint optimization problem:

[0121] The objective of this invention is to maximize the remaining energy and achieve energy consumption and load balancing of a PCN by optimizing task offloading, task migration, and processor allocation decisions. Essentially, it addresses the problem of maximizing the long-term energy storage of the system while ensuring task latency requirements and stable operation of each PCN. Therefore, this invention aims to maximize the long-term energy storage of all photovoltaic edge computing nodes in the photovoltaic edge computing node network system, establishing a joint optimization problem of task migration and resource allocation. The expression of the joint optimization problem is as follows:

[0122] ,

[0123] The constraints include:

[0124] ,

[0125] ,

[0126] ,

[0127] ,

[0128] in, This represents the number of operational slots in the photovoltaic edge computing node network system. This represents the total number of photovoltaic edge computing nodes. Let n be the amount of electrical energy stored by the photovoltaic edge computing node n in time slot t. For the task Service delays, For the task Maximum service latency, This represents the minimum task processing capacity of photovoltaic edge computing node n when it is in a non-downtime state. The amount of data processed by the photovoltaic edge computing node n in time slot t; For photovoltaic edge computing node n, the k-th processor will be used to process tasks in time slot t. The decision, For photovoltaic edge computing node n, the task will be performed in time slot t. The decision to migrate to the photovoltaic edge computing node m;

[0129] Step 6: Transformation and decomposition of the joint optimization problem

[0130] Step 6-1: Construct the amount of tasks to be processed by photovoltaic edge computing node n in time slot t.

[0131] Regarding constraints Construct the amount of unprocessed tasks of photovoltaic edge computing node n in time slot t. , ; ;

[0132] The amount of pending tasks to be constructed Conditions for achieving stability:

[0133] ,

[0134] in, This represents the Lyapunov drift function value for node n in time slot t. , , , When the time slot is t Expected value Represents the virtual queue of node n in time slot t. The congestion value;

[0135] Step 6-2: Transformation of the Joint Optimization Problem

[0136] Based on the number of tasks to be processed and the number of tasks to be processed Under stable conditions, the joint optimization problem of task migration and resource allocation is addressed. Transform into a problem ,question The expression is:

[0137]

[0138] The constraints include:

[0139]

[0140] in, It is a non-negative parameter, which serves to balance the amount of task processing and the amount of electrical energy stored.

[0141] Step 6-3: Decomposition of the joint optimization problem

[0142] On the issue Decompose the problem to obtain subproblems. The problem of succubi ,

[0143] Subproblems Known task migration strategy Resource allocation optimization subproblem under certain conditions, subproblem The expression is:

[0144] ,

[0145] The constraints are:

[0146] ,

[0147] Subproblems To implement a given processor resource allocation strategy In the case of migration strategy optimization subproblem, subproblem The expression is:

[0148] ,

[0149] The constraints are:

[0150] ,

[0151] in, Let represent the running time of the k-th processor of photovoltaic edge computing node n in time slot t, specifically expressed as:

[0152] .

[0153] The decision made for subproblem 2 should be used as the observation for the next decision on subproblem 1.

[0154] Based on the above analysis, this invention proposes to use a pre-defined multi-agent reinforcement learning algorithm to jointly solve the allocation optimization sub-problem and the task migration optimization sub-problem, thereby obtaining the optimized task migration strategy and resource allocation strategy.

[0155] Step 7: Construct the task decision Markov process:

[0156] The joint optimization problem proposed in step 6 is transformed into an MDP problem, containing basic elements such as state, action, and reward. The specific process is as follows:

[0157] Step 7-1: Construct the state of each photovoltaic edge computing node, including:

[0158] The state of photovoltaic edge computing node n in time slot t Represented as:

[0159] ,

[0160] in, Indicates task The size of the data, express Time slot to complete the task Number of remaining processor cycles required Let n be the amount of electrical energy stored by the photovoltaic edge computing node n in time slot t. This represents the offloading decision of the photovoltaic edge computing node n for task i in time slot t-1. This indicates that the photovoltaic edge computing node n will perform the task in time slot t-1. The decision to migrate to the photovoltaic edge computing node m;

[0161] Specifically, regarding sub-problem 1, the state of the photovoltaic edge computing node n in time slot t. Represented as:

[0162] ,

[0163] Regarding sub-problem 2, the state of photovoltaic edge computing node n in time slot t. Represented as:

[0164] ,

[0165] Step 7-2: Construct the actions for each photovoltaic edge computing node, including:

[0166] Actions performed by photovoltaic edge computing node n in time slot t for:

[0167] ,

[0168] Among them, the actions performed by the photovoltaic edge computing node n in time slot t for subproblem 1 for:

[0169] ,

[0170] Actions performed by photovoltaic edge computing node n in time slot t for subproblem 2 for:

[0171] ,

[0172] Step 7-3: Construct the reward for each photovoltaic edge computing node:

[0173] Rewards earned by photovoltaic edge computing node n in time slot t for:

[0174] ,

[0175] in, This represents the reward for subproblem 1 of the photovoltaic edge computing node n in time slot t, specifically calculated as follows:

[0176] ,

[0177] in, This represents the state of subproblem 1 at node n in time slot t. This represents the action taken by node n in time slot t in response to subproblem 1. This represents the state of subproblem 2 at node n in time slot t. This represents the action taken by node n in time slot t in response to subproblem 2;

[0178] The reward for subproblem 2 of the photovoltaic edge computing node n in time slot t is calculated as follows:

[0179] ,

[0180] Where F is a single binary digit; if the value within the parentheses is true, the value is 1; otherwise, the value is 0. Let t be the set of tasks arriving at the photovoltaic edge computing node n in time slot t. Indicates the number of the photovoltaic edge computing node; The task is processed by the kth processor core of node m. Required delay The time slot length; For the task The size of the data, Represents photovoltaic edge computing nodes In time slot t, the k-th processor is used to process the task. The decision.

[0181] Step 8: Processor resource allocation strategy and task migration strategy based on the preset multi-agent reinforcement learning algorithm:

[0182] For the MDP problem in step 7, an online decision-making algorithm based on multi-agent interaction reinforcement learning is designed as follows:

[0183] Step 8-1: Initialize the strategies for each subproblem:

[0184] First, initialize the evaluation network of Deep Reinforcement Learning (DQN) for subproblem 1. and target network and the corresponding evaluation network parameters and target network parameters The critics of the Multi-Agent Deep Deterministic Policy Gradient Algorithm (MADDPG) for subproblem 2 then initialize the target network. and critic evaluation network The network parameters for critics' target and evaluation are respectively... and Then initialize the actor target network for subproblem 2. and actor evaluation network The network parameters for actor target and actor evaluation are respectively... and ;

[0185] Repeat the following steps until subproblem 1 and subproblem 2 converge:

[0186] Based on the photovoltaic edge computing node n in time slot t Make decisions and take actions for sub-problem 1 in turn. Obtain the reward for subproblem 1 And observe the photovoltaic edge computing node n in The state of the time slot Generate data And put it into the experience replay pool of subproblem 1;

[0187] For subproblem 1, H data points are sampled from the experience replay pool, and the loss function is calculated:

[0188] Loss function of subproblem 1 The calculation formula is:

[0189] ,

[0190] in, This represents the expected value of the function corresponding to the H sampled data points. For photovoltaic edge computing node n, the h-th sampled for subproblem 1 The states corresponding to [1,H] samples For the photovoltaic edge computing node n, the action to be taken for the h-th sample sampled from subproblem 1 is... For the function value of the target network in subproblem 1 of photovoltaic edge computing node n under the h-th sample, the calculation formula is:

[0191] ,

[0192] in, For the photovoltaic edge computing node n, the state of the next time slot is sampled in the h-th sample for subproblem 1. The action taken by photovoltaic edge computing node n in response to the h-th sample of subproblem 1 for the next time slot state. As a discount factor, For node n in subproblem 1, in state... Make an action The reward received at that time The target network for subproblem 1 of node n has parameters. Status is Make an action The output value is then used to update the evaluation network for subproblem 1 using gradient descent.

[0193] Based on the observation status of photovoltaic edge computing node n in time slot t The task transfer decision output by sub-problem 1 determines the action to be taken. Obtain the reward for subproblem 2. And observe the photovoltaic edge computing node n in The state of the time slot Generate data And put it into the experience replay pool of subproblem 2;

[0194] For subproblem 2, H samples are sampled from the empirical replay pool, and the critic-evaluated network loss function is calculated. The calculation formula is:

[0195] ,

[0196] in, ,for The corresponding global state data, For the state corresponding to the h-th sample of the photovoltaic edge computing node n for sub-problem 2, For node n in subproblem 2, with model parameters... Status is Make an action Actor Evaluation Network Output value, The calculation is as follows:

[0197] ,

[0198] in, ,for The corresponding global state data, For the photovoltaic edge computing node n, the state of the next time slot is sampled in the h-th sample for subproblem 2. For node n in subproblem 2, with model parameters... Status is Make an action Actor Target Network Output value;

[0199] Calculate the cumulative policy function value of the actor evaluation network. The calculation formula is:

[0200] ,

[0201] in, For node n in subproblem 2, with model parameters... Status is Make an action Critics evaluate the network output values;

[0202] right Sampling strategy gradient method gradient The calculation is as follows:

[0203] ,

[0204] in, Indicates the parameter Regarding parameters Perform gradient calculations.

[0205] Update actor evaluation networks and critic evaluation networks:

[0206] Critics assess network parameters And actor evaluation network parameters Update via gradient of the loss function:

[0207] ,

[0208] in, Indicates the parameter Regarding parameters Perform gradient calculations. Discount factor;

[0209] ,

[0210] in, Indicates the parameter Regarding parameters Perform gradient calculations. Discount factor;

[0211] The target network parameters for critics and actors are updated using a soft update method.

[0212] ,

[0213] ,

[0214] in, , These are the updated weights for the network parameters evaluated by critics and those evaluated by actors, respectively.

[0215] Combination Figures 2 to 7 As shown, the optimization effect of the embodiments of this application is explained in detail through comparison under different parameter states.

[0216] Figure 2 This chart compares the final battery levels of edge nodes after running for 50 seconds using different optimization schemes under varying average light intensities. Figure 2 COTA is a task offloading and resource allocation scheme based on migration cost; LOTA is a task offloading, migration, and resource allocation scheme based on load status; and NCTA is a latency-oriented optimization scheme for computation offloading and resource allocation. The average remaining battery power refers to the remaining battery power when all tasks arriving within 50 seconds are completed. Figure 2 It can be seen that when there is insufficient light, the advantages of the solution in this embodiment of the invention are small, while the advantages of this solution are obvious when there is sufficient light.

[0217] Figure 3 This chart compares the final battery levels of edge nodes after 50 seconds of operation using different schemes listed in the diagram, under varying task loads. Figure 3 It can be seen that when there are fewer tasks, the task load is smaller, and the solution in this embodiment of the invention has a greater advantage. When there are more tasks, the advantage of this solution gradually diminishes.

[0218] Figure 4This chart compares the final power consumption of a photovoltaic edge computing node network system after 50 seconds of operation using different optimization schemes at different processor frequencies. Figure 4 As shown in the processor power model, task processing time is directly proportional to processor frequency, and processor power has a high-order function relationship with processor frequency. Frequency selection requires balancing operating power and processing speed. Therefore, the optimal frequency for this simulation environment is approximately 4 GHz.

[0219] Figure 5 This chart compares the normal operating time of a photovoltaic edge computing node network system using different optimization schemes under varying light intensities. The stable duration refers to the length of time the task latency compliance rate remains within acceptable limits. Since the maximum operating time of the photovoltaic edge computing node network system is 50 seconds, the maximum normal operating time is 50 seconds. Figure 5 It can be seen that, when comparing normal operating time, the solution in this embodiment of the invention still maintains a significant advantage when light resources are relatively scarce.

[0220] Figure 6 A comparison chart showing the uptime of node systems using different optimization schemes under different task requirements, provided by... Figure 6 It can be seen that the normal operating time can reach its maximum value when the number of tasks is less than 8, and the normal operating time only begins to decrease when the number of tasks is greater than 8.

[0221] Figure 7 This chart compares the uptime of node network systems using different optimization schemes at different processor frequencies. Figure 7 It can be seen that the solution in this embodiment of the invention has the most obvious advantages when the processor frequency is around the optimal frequency, and still has significant advantages when the processor frequency is other.

[0222] Example 2

[0223] Based on the same inventive concept as Embodiment 1, this embodiment of the invention provides a task processing device for photovoltaic edge computing nodes, applied to a photovoltaic edge computing node network system, comprising:

[0224] The task processing queue generation module is used to generate task processing queues for each photovoltaic edge computing node based on the user task unloading queue and the migration task receiving queue.

[0225] The optimization problem construction module is used to construct processor resource allocation optimization sub-problems and task migration optimization sub-problems based on pre-built service latency model, service energy consumption model and energy storage model, with the goal of matching the task processing volume of each photovoltaic edge computing node with the current power supply capacity.

[0226] The task processing module is used to solve the processor resource allocation optimization sub-problem and the task migration optimization sub-problem jointly using a preset multi-agent reinforcement learning algorithm based on the task processing queues of each photovoltaic edge computing node, so as to obtain the optimized processor resource allocation strategy and task migration strategy. The task migration strategy is used to determine whether to migrate the tasks in this photovoltaic edge computing node to other photovoltaic edge computing nodes, and the processor resource allocation strategy is used to determine whether each processor in each photovoltaic edge computing node is used to process tasks.

[0227] It should be noted that the explanation of the aforementioned task processing method embodiment for photovoltaic edge computing nodes also applies to the task processing system for photovoltaic edge computing nodes in this embodiment, and will not be repeated here.

[0228] Example 3

[0229] Based on the same inventive concept as in Embodiment 1, this embodiment of the invention provides a task processing system for photovoltaic edge computing nodes, applied to a photovoltaic edge computing node network system, including a storage medium and a processor;

[0230] The storage medium is used to store instructions;

[0231] The processor is configured to operate according to the instructions to execute the method according to any one of Embodiment 1.

[0232] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0233] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0234] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0235] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0236] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

[0237] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for task processing oriented to a photovoltaic edge computing node, characterized in that, The application is applied to a photovoltaic edge computing node network system, comprising: Based on the user task offloading queue and the migrated task receiving queue, a task processing queue of each photovoltaic edge computing node is generated; In order to match the task processing capacity of each photovoltaic edge computing node with the current power supply capacity, a processor resource allocation optimization sub-problem and a task migration optimization sub-problem are constructed based on a pre-constructed service delay model, a service energy consumption model and an electric energy storage quantity model; the matching of the task processing capacity with the current power supply capacity means that the power supply capacity of the current photovoltaic edge computing node matches the maximum task processing capacity supported by the processor; Based on the task processing queue of each photovoltaic edge computing node, a preset multi-agent reinforcement learning algorithm is used to jointly solve the processor resource allocation optimization sub-problem and the task migration optimization sub-problem, so as to obtain an optimized processor resource allocation strategy and a task migration strategy; the processor resource allocation strategy is used to determine whether each processor in each photovoltaic edge computing node is used to process a task, and the task migration strategy is used to determine whether a task in the current photovoltaic edge computing node is migrated to another photovoltaic edge computing node. 2.The method of claim 1, wherein: The expression of the task processing queue of the photovoltaic edge computing node is: ， In the formula, a photovoltaic edge computing node a task processing queue of the photovoltaic edge computing node, a photovoltaic edge computing node a migration task receiving queue of the photovoltaic edge computing node, the migration task receiving queue storing tasks migrated from other photovoltaic edge computing nodes in history, a photovoltaic edge computing node a user task unloading queue of the photovoltaic edge computing node, the user task unloading queue storing tasks unloaded by users in history. 3.The method of claim 1, wherein, The construction method of the service delay model comprises: A task processing delay model is constructed, and the expression of the task processing delay model is: ， in, For the number Task via photovoltaic edge computing nodes The delay for the k-th processor to process, Let k be the computing frequency of the k-th processor. To complete the calculation task Number of processor cycles required; A task waiting delay model is constructed, and the expression of the task waiting delay model is: ， wherein, a task processing queue of tasks a waiting time in a photovoltaic edge computing node , a remaining time for the kth processor in the photovoltaic edge computing node to complete a current task, , a working state of the kth processor in the photovoltaic edge computing node at time slot t, indicates that a task processing is in progress, indicates idle, a decision of the photovoltaic edge computing node to use the kth processor to process a task at time slot t, , indicates that the photovoltaic edge computing node n uses the kth processor to process a task at time slot t, otherwise, , ; a predecessor task set of a task in a task processing queue ; a number of processor cycles required to complete a task ; a total number of processors in the photovoltaic edge computing node n; A task transmission delay model is constructed, and the expression of the task transmission delay model is: ， wherein, is a task of a photovoltaic edge computing node and a transmission time between the photovoltaic edge computing node is a task of a data amount size, denotes a transmission rate between the photovoltaic edge computing node and the photovoltaic edge computing node , , denote a minimum transmission rate and a maximum transmission rate between the photovoltaic edge computing node n and the photovoltaic edge computing node m, respectively; A task waiting transmission delay model is constructed, and the expression of the task waiting transmission delay model is: ， wherein, is a task waiting time in a sending queue , represents a predecessor task set of a task in a sending queue ; the construction method of the sending queue is that when the photovoltaic edge computing node is overloaded or energy storage is insufficient, the task will enter the sending queue ; is the transmission time of a task between the photovoltaic edge computing node and the photovoltaic edge computing node m; A service delay model of the photovoltaic edge computing node is constructed, and the expression of the service delay model of the photovoltaic edge computing node is: ， wherein, estimated task for time slot t total service delay of the task; denotes the decision of photovoltaic edge computing node n to migrate the task to photovoltaic edge computing node m at time slot t, denotes the decision of photovoltaic edge computing node n to migrate the task to photovoltaic edge computing node m at time slot t, otherwise, , denotes the total number of photovoltaic edge computing nodes, denotes the duration of time required by the kth processor of photovoltaic edge computing node m to process the task , denotes the duration of time required for the task to wait for processing in the processing queue of photovoltaic edge computing node m, . 4.The method of claim 1, wherein, The construction method of the service energy consumption model comprises: A task computing energy consumption model is constructed, and the expression of the task computing energy consumption model is: ， ， wherein, is the task the computational energy consumed by the kth processor in the photovoltaic edge computing node n at time slot t, and the power computation parameters related to the processor architecture, is the computation frequency of the kth processor in the photovoltaic edge computing node n, denotes the time slot to complete the task the number of remaining processor cycles required, is the time slot length, is the operating power of the kth processor in the photovoltaic edge computing node n; A task migration energy consumption model is constructed, and the expression of the task migration energy consumption model is: ， wherein, is a task transmission energy consumption of the task from the photovoltaic edge computing node n to the photovoltaic edge computing node m at the t time slot, denotes a transmission power of the photovoltaic edge computing node n, denotes a task remaining amount of data not transmitted at the t time slot, denotes a transmission rate between the photovoltaic edge computing node and the photovoltaic edge computing node m; A task service energy consumption model is constructed, and the expression of the task service energy consumption model is: ， wherein, is the task the energy consumption of the service required at time slot t, denotes the total number of processors in the photovoltaic edge computing node n; denotes the decision of the photovoltaic edge computing node n to migrate the task to the photovoltaic edge computing node m at time slot t; denotes the decision of the photovoltaic edge computing node n to use the kth processor for processing the task at time slot t, denotes the decision of the photovoltaic edge computing node n to use the kth processor for processing the task at time slot t, otherwise, ; denotes the total number of photovoltaic edge computing nodes; A service energy consumption model of the photovoltaic edge computing node is constructed, and the expression of the service energy consumption model of the photovoltaic edge computing node is: ， wherein, total energy consumption of all tasks served by the photovoltaic edge computing node n in time slot t, total energy consumption of all tasks served by the photovoltaic edge computing node task processing queue of the photovoltaic edge computing node 5.The method of claim 1, wherein: The construction method of the electric energy storage quantity model comprises: A photovoltaic conversion model is constructed, and the expression of the photovoltaic conversion model is: ， wherein, Pn(t) is the photovoltaic supply system power for the photovoltaic edge computing node n, and η represents the electrical energy conversion efficiency, An(t) represents the photovoltaic panel illuminated area of the photovoltaic edge computing node n, Irrn(t) is the light radiation intensity received by the photovoltaic edge computing node n at the t time slot; An electric energy change quantity model of each photovoltaic edge computing node in each time slot is constructed, and the expression of the electric energy change quantity model of each photovoltaic edge computing node in each time slot is: ， wherein, is the amount of change in the electrical energy of the photovoltaic edge computing node n at time slot t, is the static energy consumption of the photovoltaic edge computing node n, depending on the processor running state, is the time slot length, is the total energy consumption of all tasks served by the photovoltaic edge computing node n at time slot t. An electric energy storage quantity model is constructed, and the expression of the electric energy storage quantity model is: ， wherein, is the amount of energy stored by the photovoltaic edge computing node n at time slot t, is the initial amount of energy stored by the photovoltaic edge computing node n. 6.The method of claim 1, wherein: The matching of the task processing capacity with the current power supply capacity means that the power supply capacity of the current photovoltaic edge computing node matches the maximum task processing capacity supported by the processor, and the construction method of the processor resource allocation optimization sub-problem and the task migration optimization sub-problem comprises: A joint optimization problem of processor resource allocation and task migration is established, and the expression of the joint optimization problem is : ， The constraint condition comprises: ， ， ， ， wherein, is the number of time slots that the photovoltaic edge computing node network system has been running, is the total number of photovoltaic edge computing nodes, is the amount of electrical energy storage of photovoltaic edge computing node n at time slot t, is the service delay of task , is the maximum service delay of task , is the minimum amount of tasks that photovoltaic edge computing node n can handle when it is not in a down state, is the amount of data that photovoltaic edge computing node n handles for task at time slot t; is the decision of photovoltaic edge computing node n to use the kth processor for handling task at time slot t, denotes that photovoltaic edge computing node n uses the kth processor for handling task at time slot t, otherwise, , is the decision of photovoltaic edge computing node n to migrate task to photovoltaic edge computing node m at time slot t, denotes that photovoltaic edge computing node n migrates task to photovoltaic edge computing node m at time slot t, otherwise, ; The calculation formula is: ， wherein, is a low power determination threshold, is a high power determination threshold, is a maximum energy storage of the photovoltaic edge computing node n; and are respectively the newly added energy of the photovoltaic edge computing node n based on the time slot t and the estimated task processing capacity of the photovoltaic edge computing node network system based on the time slot t. The calculation formula is: ， in, For the task The complexity is the number of processor cycles required to process each bit of task data; , Represents the photovoltaic edge computing node in time slot t. The working state of the k-th processor. This indicates that a task is in progress. Indicates that the space is available; This indicates that the user will send the task in time slot t. The decision to offload to the photovoltaic edge computing node n, if Then the user will send the task in time slot t. Unload to photovoltaic edge computing node n; otherwise... ; Photovoltaic edge computing nodes Task processing queue, This represents the computing frequency of the k-th processor. This represents the total number of processors in photovoltaic edge computing node n; The maximum duration for which the additional electrical energy generated by the photovoltaic edge computing node n in time slot t can power the processor is calculated using the following formula: ， wherein, is the power supplied by the photovoltaic power supply system of the photovoltaic edge computing node n, is the time slot length, is the transmission power of the photovoltaic edge computing node n; is the static energy consumption of the photovoltaic edge computing node n, which depends on the running state of the processor; is the running power of the kth processor in the photovoltaic edge computing node n; is the time consumed by the photovoltaic edge computing node n for transmitting task data in the t time slot, and the calculation formula is: ， wherein, is a sending queue, denotes a task a remaining amount of data to be transmitted at time slot t, denotes a photovoltaic edge computing node and a transmission rate between the photovoltaic edge computing node m; The calculation formula is: ， wherein, a photovoltaic edge computing node task processing queue; The calculation formula is: ； The joint optimization problem of the processor resource allocation and the task migration is transformed and decomposed to obtain the processor resource allocation optimization sub-problem and the task migration optimization sub-problem respectively; In the solving process by using the preset multi-agent reinforcement learning algorithm, the decision made for the task migration optimization sub-problem should be taken as an observation value for making a decision for the processor resource allocation optimization sub-problem next time.

7. The method of claim 6, wherein the method further comprises: The joint optimization problem of processor resource allocation and task migration is transformed and decomposed to obtain a processor resource allocation optimization sub-problem and a task migration optimization sub-problem, including: For the constraint condition , construct the amount of tasks to be processed by the photovoltaic edge computing node n at time slot t , ; amount of tasks to be processed the condition of convergence is that: ， in, , , , When the time slot is t Expected value For constant terms, This represents the maximum amount of tasks processed by the photovoltaic edge computing node n across all time slots. This is the difference between the minimum task processing volume and the actual task processing volume of the photovoltaic edge computing node n in time slot t; Based on the amount of tasks to be processed Stabilizing conditions, joint optimization of processor resource allocation and task migration Transforming into a problem , the problem The expression is: ， The constraints of the condition include: ， wherein, is a non-negative parameter that plays a role of trade-off between the amount of task processing and the amount of electric power storage; On the issue Decompose the problem to obtain subproblems. The problem of succubi , Sub-problems Optimization sub-problems for processor resource allocation in the case of known task migration policies Optimization sub-problems for processor resource allocation in the case of known task migration policies The expression of the sub-problem is: ， The constraint condition is: ， Sub-problems For the task migration optimization sub-problem in the case of a given processor resource allocation policy , the expression of the sub-problem is: ， The constraint condition is: 、 ， wherein, denotes the runtime of the kth processor of the photovoltaic edge computing node n at the tth time slot, which is specified as: ， wherein, indicates that the task is completed at time slot t the number of remaining processor cycles required. 8.The method of claim 7, wherein: The processor resource allocation optimization sub-problem and the task migration optimization sub-problem are jointly solved by using the preset multi-agent reinforcement learning algorithm to obtain an optimized processor resource allocation strategy and a task migration strategy, including: The state of each photovoltaic edge computing node is constructed, including: State of photovoltaic edge computing node n at time slot t is represented as: ， wherein, represents the data volume size of a task , represents the decision of a user to offload a task to a photovoltaic edge computing node n at a time slot , represents the decision of a photovoltaic edge computing node n to migrate a task to a photovoltaic edge computing node m at a time slot ; wherein, for sub-problem 1, the state of the photovoltaic edge computing node n at time slot t is represented as: ， For sub-problem 2, the state of the photovoltaic edge computing node n at time slot t is represented as: ， The action of each photovoltaic edge computing node is constructed, including: Actions taken by photovoltaic edge computing node n at time slot t are: ， wherein the action made by the photovoltaic edge computing node n at time slot t for sub-problem 1 is: ， Actions made by the photovoltaic edge computing node n at time slot t for sub-problem 2 are: ， The reward of each photovoltaic edge computing node is constructed: Photovoltaic edge computing node n's reward at time slot t is: ， wherein, R1(t) denotes the reward of sub-problem 1 for the t-th time slot photovoltaic edge computing node n, which is calculated as: ， R2(n, t) = R2(n, t) = R2(n, t) = R2(n, t) = R2(n, t) = R2(n, t) = R2(n, t) = R2(n, t) ， where F is a one-bit binary number that takes the value 1 if the expression inside the brackets is true, and 0 otherwise; the set of tasks that arrive at photovoltaic edge computing node n at time slot t, denotes the number of photovoltaic edge computing nodes; the kth processor core of photovoltaic edge computing node m processes task the required delay, denotes the photovoltaic edge computing node the kth processor is used for processing tasks at time slot t the decision; First, initialize the evaluation network for subproblem 1. and target network and the corresponding evaluation network parameters and target network parameters Reinitialize the critic target network for subproblem 2 and critic evaluation network And the corresponding critic target network parameters And critics evaluate network parameters Then initialize the actor target network for subproblem 2. and actor evaluation network and the corresponding actor target network parameters And actor evaluation network parameters ; The following steps are repeatedly executed until the sub-problem 1 and the sub-problem 2 converge: Based on the state of photovoltaic edge computing node n in time slot t Make decisions and take actions for sub-problem 1 in turn. Obtain the reward for subproblem 1 And observe the photovoltaic edge computing node n in The state of the time slot Generate data And put it into the sub-experience replay pool of sub-problem 1; H data are sampled from the experience replay pool for the sub-problem 1, and a loss function is calculated: The loss function of sub-problem 1 The calculation formula is: ， wherein, represents the function expectation value corresponding to the H data pairs of the sample, is the state corresponding to the h th sample of the photovoltaic edge computing node n for sub-problem 1, is the action taken by the photovoltaic edge computing node n for the h th sample of the sample, and the function value of the sub-problem 1 target network of the photovoltaic edge computing node n under the h th sample is calculated according to the following formula: ， wherein, is the next time slot state in the h-th sample for sub-problem 1 sampled by photovoltaic edge computing node n, is the action taken by photovoltaic edge computing node n for sub-problem 1 in the h-th sample for sub-problem 1 sampled by photovoltaic edge computing node n, is the discount factor, is the reward obtained by node n in sub-problem 1 when the state is and the action is , is the output value of the target network of sub-problem 1 of node n when the parameters are and the state is and the action is ; and the evaluation network of sub-problem 1 is updated according to the gradient descent method. Based on the observation status of photovoltaic edge computing node n in time slot t The task transfer decision output by sub-problem 1 determines the action to be taken. Obtain the reward for subproblem 2. And observe the photovoltaic edge computing node n in The state of the time slot Generate data And put it into the experience replay pool of subproblem 2; For sub-problem 2, sample H samples from the experience replay pool and compute the critic evaluation network loss function The formula is: ， wherein, is the global state data corresponding to is the state corresponding to the h-th sample of the actor n for sub-problem 2, is the action taken by the node n in sub-problem 2 when the model parameters are is the state is the action actor evaluation network output value, is calculated as: ， wherein, is the global state data corresponding to is the next time slot state in the h-th sample of photovoltaic edge computing node n for sub-problem 2, is the action made by photovoltaic edge computing node n in sub-problem 2 when the model parameters are is the state is the action actor target network output value; Computing cumulative policy function values of an actor evaluation network , the formula is: ， wherein, For sub-problem 2, the photovoltaic edge computing node n makes an action when the model parameter is state is action critic evaluates network output value; For Using the policy gradient method, With respect to the gradient is calculated as: ， wherein denotes the derivative with respect to with respect to the parameter a gradient operation is performed, The actor evaluation network and the critic evaluation network are updated: critic evaluates network parameters and actor evaluates network parameters through loss function gradient updates: ， wherein denotes the operation of with respect to the parameter a gradient operation is performed, is a discount factor; ， wherein denotes the operation of with respect to the parameter a gradient operation is performed, is a discount factor; The critic target network parameters and the actor target network parameters adopt a soft update mode: ， ， wherein, , are the updated weights of the critic and actor network parameters, respectively.

9. A task processing device for photovoltaic edge computing nodes, characterized in that, The application is applied to a photovoltaic edge computing node network system, including: A task processing queue generation module is configured to generate a task processing queue of each photovoltaic edge computing node based on a user task offloading queue and a migration task receiving queue. An optimization problem construction module is configured to construct a processor resource allocation optimization sub-problem and a task migration optimization sub-problem based on a pre-constructed service delay model, a service energy consumption model, and an electric energy storage amount model, with the goal of matching a task processing amount of each photovoltaic edge computing node with a current power supply capacity. A task processing module is configured to jointly solve the processor resource allocation optimization sub-problem and the task migration optimization sub-problem by using a preset multi-agent reinforcement learning algorithm to obtain an optimized processor resource allocation strategy and a task migration strategy.

10. A task processing system for photovoltaic edge computing nodes, characterized in that, The storage medium is configured to store instructions. The processor is configured to operate according to the instructions to perform the method according to any one of claims 1-8. The storage medium is configured to store instructions. The processor is configured to operate according to the instructions to perform the method according to any one of claims 1-8.

Images