A microgrid edge scheduling and agent lightweight clipping method
By applying deep reinforcement learning and model tailoring techniques in micronet groups, we construct edge scheduling and lightweight cutting methods of agents in micronet groups, and solve the challenges of dynamic migration of computing resources in micronet groups and lightweighting of agents in edges in micronet groups, and realize the The efficient use of resources and energy reduces model size and data transmission delay.
Patent Information
- Application Number
- CN202210549822.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-20
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-05-20
AI Technical Summary
In the micronet group, how to build an efficient dynamic migration system for computing resources and lightweight edge agents to make full use of large-scale renewable energy, reduce the power waste rate of distributed power generation and improve power supply reliability has become an urgent issue.
Using micronet group edge scheduling and lightweight cutting methods for agents, through deep reinforcement learning and model tailoring technology, a micronet group including power terminals and edge devices is built, training and tailoring of task allocation models is carried out, and the resource allocation strategy prediction is realized. and execution, optimize task offloading and energy allocation decisions.
Realize dynamic scheduling and model optimization of computing resources, reduce model size, reduce resource requirements, improve the energy and resource utilization efficiency of micronet groups, is suitable for efficient operation in complex environments, and avoids large amounts of data transmission and accuracy losses. .
Smart Images

Figure CN114936078B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer networks, and in particular relates to a microgrid edge scheduling and intelligent body lightweight clipping method. Background Art
[0002] With the rapid development of social economy and population growth, the problem of energy supply has become increasingly prominent. In order to meet the needs of sustainable social development, distributed energy and renewable energy distributed near users have gradually become a research hotspot in the field of energy technology. Distributed power generation is represented by photovoltaic systems and wind power generation. Due to its advantages of economy, environmental protection, and flexibility, it has been widely used in recent years. However, the output of distributed power generation is intermittent and random. The large-scale penetration of distributed power generation will have an adverse impact on the safety and stability of the power system, such as power quality problems, which brings unprecedented challenges to the stable operation and safety control of the power system. In order to coordinate the contradiction between distributed power generation and the large power grid, reduce the power abandonment rate of distributed power generation, and improve power supply reliability and power quality, the concept of microgrid group is proposed. However, even for microgrid groups, there are still obstacles to fully absorb and fully utilize large-scale renewable energy. In this mode, how to build an efficient and complete dynamic migration system of computing resources at the edge and how to further lightweight edge intelligent agents have become an urgent problem to be solved. Summary of the invention
[0003] In view of the above technical problems, the present invention proposes a method for microgrid edge scheduling and intelligent agent lightweight tailoring. By supporting the application of edge agent micro-intelligent algorithm lightweight tailoring technology in microgrid scenarios, dynamic migration scheduling of microgrid operation computing resources can be achieved, and further lightweight tailoring design of edge agents can be performed. In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:
[0004] A microgrid edge scheduling and intelligent agent lightweight tailoring method, comprising the following steps:
[0005] S1, construct a microgrid group including power terminals and edge devices, and a task allocation model is set in the edge device k;
[0006] S2, set the total training rounds T, the initial training round number t=0, initialize the local training model of each power terminal, the sparsity range, the aggregation weight of the local training model, and the experience replay memory of the power terminal;
[0007] S3, the power terminal trains the task allocation model based on the deep reinforcement learning method, and prunes the local training model based on the model pruning method. The edge device k aggregates the trained model and updates the task allocation model.
[0008] S4, predicting a resource allocation strategy based on the updated task allocation model and with the goal of maximizing long-term benefit expectations, wherein the resource allocation strategy includes a task offloading decision and an energy allocation decision;
[0009] S5, the power terminal executes the task according to the resource allocation strategy predicted in step S4.
[0010] The step S3 comprises the following steps:
[0011] S3.1, execute t=t+1, assign the random set of M power terminals to the set S t ;
[0012] S3.2, Set S t Each power terminal in downloads the task allocation model from the edge device k and assigns it to the local training model of the power terminal;
[0013] S3.3, the power terminal senses and updates the experience replay memory;
[0014] S3.4, the power terminal trains the local training model using a deep reinforcement learning method, and trims the trained model parameters based on a model trimming method;
[0015] S3.5, uploading the parameters of the local training model trimmed in step S3.4 and the number of times the power terminal performs local training to the edge device k;
[0016] S3.6, the edge device k counts the number of local training times of each power terminal respectively;
[0017] S3.7, edge device k aggregates the received model parameters according to the statistical local training times;
[0018] S3.8, determine whether t<T, if so, return to step S3.1, otherwise execute step S4.
[0019] The step S3.4 comprises the following steps:
[0020] S3.4.1, the set S t Initializing a neural network of each power terminal in the network, wherein the neural network includes a plurality of fully connected layers and a plurality of convolutional layers;
[0021] S3.4.2, reduce the parameters of the fully connected layer of the neural network;
[0022] S3.4.3, set the sparsity of each fully connected layer according to the sparsity range;
[0023] S3.4.4, arrange the parameters of each fully connected layer in order from small to large, and then The smaller parameter value is set to 0, where represents the i-th fully connected layer fc The sparsity of the layer, The i-th fully connected layer fc The number of parameters of the layer;
[0024] S3.4.5, set the sparsity of each layer of the accumulation layer according to the sparsity range;
[0025] S3.4.6, prune the number of filters in each convolutional layer.
[0026] In step S4, the expression for maximizing the expected long-term benefit is:
[0027]
[0028] In the formula, U(X, Φ) represents the maximum long-term benefit expectation, I represents the total number of time slices, u(·) represents the direct utility, X1 represents the initial network state variable, X0 represents the value of the initial network state, Φ(X i ) indicates network status X i The decision function of X represents the set of network states.
[0029] The network status X i The expression is:
[0030]
[0031] In the formula, Represents the task queue under time slice i, represents the energy queue in time slice i, s i represents the connection relationship between the power terminal and the edge device in the i time slice, g i represents the channel gain in time slice i, represents the channel gain set, N represents the number of edge devices, Indicates the maximum length of the task queue. Indicates the maximum length of the energy sequence.
[0032] Direct utility u(X i ,Φ(X i )) is calculated as:
[0033]
[0034] Where w1 represents the task execution delay d i The weight of w2 represents the task queuing delay ρ i The weight of w3 represents the number of tasks abandoned η iThe weight of w4 represents the cost of occupying edge nodes φ i The weight of .
[0035] The task execution delay d i The calculation formula is:
[0036]
[0037] In the formula, represents the time consumption of local execution task in time slice i, h i represents the switching delay in time slice i, represents the time cost of transmitting the task from the local to the corresponding edge device in the i-th time slice, d s represents the delay caused when the task is offloaded to the edge device for execution, e i represents the number of energy units allocated in time slice i, c i represents the decision of task offloading in time slice i, Represents a collection of edge devices.
[0038] The switching delay h in the i time slice i The expression is:
[0039]
[0040] Where σ represents the connection switching delay, s i Indicates the connection relationship between the power terminal and the edge device in time slice i.
[0041] The time consumption of executing tasks locally under the i time slice The calculation formula is:
[0042]
[0043] In the formula, f i Indicates the CPU frequency allocated to the microgrid in time slice i.
[0044] The occupied edge node cost φ i The calculation formula is:
[0045]
[0046] In the formula, π represents the price per unit time of occupying edge nodes, h i represents the switching delay in time slice i, represents the time cost of transmitting the task from the local to the corresponding edge device in the i-th time slice, d s It represents the delay caused when the task is offloaded to the edge device for execution, and δ represents the duration of the time slice.
[0047] The number of tasks discarded n i The calculation formula is:
[0048]
[0049] In the formula, represents the task queue under time slice i, δ represents the duration of the time slice, Indicates the number of tasks generated in time slice i, Indicates the maximum length of the task queue.
[0050] The task queuing delay ρ i The calculation formula is:
[0051]
[0052] In the formula, Indicates the task queue under time slice i.
[0053] Beneficial effects of the present invention:
[0054] By building an intelligent entity including power terminals and edge devices, training models based on observation data on each power terminal, uploading updated model parameters to edge devices for subsequent model aggregation, and trimming models during local training, it is possible to reduce the training burden and meet the needs of model optimization, and minimize accuracy loss; the optimized task allocation model can enable the microgrid to save energy and resources better and more effectively. The lightweight model trims unnecessary parameters to reduce their impact on the model. The size of the lightweight model is reduced by about 10%, which is more suitable for deployment in actual application scenarios. It solves the problem that traditional algorithms have huge model data in the original period and require too many resources, which will bring heavy resource pressure to the backbone network and is not conducive to the deployment of algorithm models. It can better support the efficient operation of microgrids in complex environments; while ensuring accurate and efficient decision-making, it avoids the transmission of a large amount of original data, reduces the amount of data transmitted during model training, and reduces the data transmission delay, giving full play to the adaptive and distributed decision-making advantages of edge intelligence technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0056] Figure 1 Schematic diagram of the microgrid scenario.
[0057] Figure 2 This is a comparison chart of task execution methods.
[0058] Figure 3 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] Deep reinforcement learning (DRL): Deep reinforcement learning provides a method for automatically training decision models. It is a combination of traditional reinforcement learning and deep neural network concepts, and reinforcement learning technology can be applied to large-scale data spaces. An important branch of deep reinforcement learning is the method based on action value estimation, which learns the value of the action and then selects the action based on its estimated value. In this branch, the most common method is Q-learning, which is widely used in many research fields. Deep reinforcement learning technology is applicable to many scenarios, such as controlling physical systems, playing games and interacting with users, etc. Reinforcement learning will first explore, which is what we often call trial and error. It will try different action decisions, apply the action to the environment to obtain the benefits of the action, and constantly adjust itself according to the effect. Deep learning uses a method of expressing the characteristics of data through a multi-layer neural network, and learns the characteristic information of the data through a neural network. The product of the combination of the two, deep reinforcement learning, adds the decision-making ability of reinforcement learning while retaining the perception ability of deep learning, so as to play a greater advantage in some fields and show better results.
[0061] Model pruning: Although large neural network models are very powerful, they are computationally intensive and memory intensive, consuming considerable storage, memory bandwidth, and computing resources. For embedded mobile applications, these resource requirements are often prohibitive and difficult to deploy on the system. Therefore, in order to solve this problem, we need to optimize and compress the model. Common methods for model compression and acceleration include: low-rank decomposition, model pruning, quantization, lightweight model design, and knowledge distillation. Pruning is widely used in model lightweighting. Pruning is a method to reduce the storage and computation required by a neural network by an order of magnitude without affecting its accuracy. The most important thing in pruning is to try to balance accuracy and complexity, reduce redundant parts and keep the most important parts. Pruning is mainly divided into two ways, pruning weights and pruning neurons. The former will not destroy the original network structure, and the latter will cause a significant change in the structure of the model. Therefore, pruning neurons has a greater impact on the model, while pruning weights has a smaller impact on the accuracy of the model. The pruning problem is actually a question of how to judge the importance of parameters, and then prune unimportant parameters, and their impact on the effect can be ignored, so as to achieve the purpose of lightweight model.
[0062] A microgrid edge scheduling and agent lightweight pruning method, such as Figure 3 As shown, the following steps are included:
[0063] S1, construct a microgrid group including power terminals and edge devices, and a task allocation model is set in the edge device k;
[0064] like Figure 1 As shown in FIG. 1 , the microgrid group is composed of several power terminals and several edge devices, and all power terminals can be connected to all edge devices. The edge devices are assembled using express, And k∈N, all edge devices form an edge cluster, each edge device is an edge node, and each edge node can serve any power terminal in the microgrid group. Among them, the edge device with the strongest computing power in the edge cluster can be selected as edge device k to store the task allocation model to aggregate the models trained by all power terminals in the microgrid group. The collection of power terminals adopts express, The computing tasks generated by the power terminal are represented by (μ, v), where μ represents the size of the transmission data required to load the computing task, and v represents the number of CPU cycles required to process the computing task, such as Figure 2As shown, these computing tasks are completed through the following two methods: one is to execute locally on the power terminal, which can avoid the delay caused by data transmission; the other is to load it to the edge device with a channel bandwidth of WHz for execution, so that the richer capabilities of the edge device can be used to perform computing tasks, that is, tasks. Which method to use needs to be weighed according to the situation.
[0065] There is a maximum length of The task queue stores tasks in a first-in, first-out (FIFO) manner. When the task queue is full, new tasks cannot be saved. At the same time, there is also a maximum length of The energy sequence is used to save energy units. The microgrid group needs to make decisions in the i time slice (c i , e i ), the duration of each time slice is δ seconds, where c i represents the decision of task offloading in time slice i, as shown in the following formula:
[0066]
[0067] e i represents the number of energy units allocated in time slice i, which will affect the CPU frequency and data transmission rate of the devices in the microgrid. i = 0, the computing task will not be executed and will remain in the task queue. i >0, the task will be executed.
[0068] S2, set the total training rounds T, the initial training round number t = 0, initialize the local training model of each power terminal, the sparsity range (γ min , γ max ), the aggregate weights of the local training model, and the experience replay memory;
[0069] S3, the power terminal trains the task allocation model based on the deep reinforcement learning method, and prunes the local training model based on the model pruning method. The edge device k aggregates the trained model and updates the task allocation model, including the following steps:
[0070] S3.1, execute t=t+1, assign the random set of M power terminals to the set S t ;
[0071] S3.2, Set S t Each power terminal in downloads the task allocation model from the edge device k with the strongest computing power and assigns it to the local training model;
[0072] S3.3, the power terminal senses and updates the experience replay memory;
[0073] The maximum size of the experience replay memory is M max , if the amount of accumulated data is greater than M max , the old data is deleted to ensure that the new data can be retained. The current network state, selected action, benefit, and the newly formed network state after the action is executed are stored in the experience replay memory for easy use during model training.
[0074] S3.4, the power terminal trains the local training model using a deep reinforcement learning method, and trims the trained model parameters based on a model trimming method, including the following steps:
[0075] S3.4.1, the set S t Initializing a neural network of each power terminal in the network, wherein the neural network includes a plurality of fully connected layers and a plurality of convolutional layers;
[0076] S3.4.2, reduce the parameters of the fully connected layer of the neural network;
[0077] There are m1 layers of fully connected layers in the neural network, and the parameter set of each layer is After cutting branches The number of parameters per layer is The final conditions that need to be met for clipping are:
[0078]
[0079] In the formula, represents the i-th fully connected layer after pruning fc A collection of layer parameters, represents the i-th fully connected layer fc The parameter set of the layer, and 1≤i fc ≤m1, m1 represents the number of fully connected layers, represents the parameter set of the m1th layer of the pruned fully connected layer, Represents the parameter set of the m1th layer of the fully connected layer.
[0080] S3.4.3, set the sparsity of each layer of the fully connected layer. The setting formula is:
[0081]
[0082] In the formula, represents the i-th fully connected layer fc The sparsity of the layer, γ max represents the maximum sparsity, γ min Indicates the minimum sparsity.
[0083] S3.4.4, arrange the parameters of each fully connected layer in order from small to large, and then The smaller parameter value is set to 0;
[0084] S3.4.5, set the sparsity of each convolutional layer. The setting formula is:
[0085]
[0086] In the formula, Represents the i-th convolutional layer c The sparsity of the layer.
[0087] S3.4.6, prune the number of filters in each convolutional layer;
[0088] The pruning requirements are:
[0089]
[0090] Where n l Indicates the number of filter kernels, F u represents the set of filters of each convolutional layer that are pruned, ||K|| represents the L2 norm of K, K represents the kernel in the filter, and l represents the label of the convolutional layer.
[0091] Since each convolutional layer of the neural network provides f three-dimensional filters Yes l ×k×k three-dimensional matrix; each three-dimensional filter F consists of n l 2D kernel K∈R k×k Composition, R k×k is a k×k two-dimensional matrix. The relative importance of filters in each layer is measured by calculating the sum of the absolute weights of the filters using formula (2). By pruning the convolutional layer, less useful filters can be removed to improve computational efficiency while minimizing the loss of accuracy.
[0092] Since two networks are required for the training of the intelligent agent model, and parameters need to be transferred between the two networks, it is necessary to ensure that the structure of the two networks remains unchanged, otherwise there will be problems in the transfer of parameters. Therefore, between neuron pruning and weight pruning, weight pruning is better. Weight pruning is to eliminate unnecessary values in the weight tensor, discard weights that do not seriously affect the performance of the model, reduce the parameters involved in the calculation, and thus reduce the number of operations, so as to achieve the effect of reducing the size of the model. This application reduces the parameters of some neural network layers in the model and removes unnecessary filters from some neural network layers, so as to maintain the model accuracy while reducing the size of the model and achieve more convenient deployment.
[0093] S3.5, uploading the parameters of the local training model trimmed in step S3.4 and the number of times the power terminal performs local training to the edge device k with the strongest computing power;
[0094] S3.6, the edge device k with the strongest computing power counts the number of local trainings for each power terminal;
[0095] S3.7, the edge device k with the strongest computing power aggregates the received model parameters according to the statistical local training times;
[0096] The formula for the k-aggregation of edge devices with the strongest computing power is:
[0097]
[0098] In the formula, θ t+1 represents the task allocation model after the tth round of aggregation, represents the number of times the power terminal d is trained in the current t round cycle, and d∈S t , A t represents the sum of the training times of M power terminals in the current t rounds, Represents the parameters of the local training model after the tth round of training.
[0099] S3.8, determine t<T, if yes, return to step S3.1, otherwise execute step S4;
[0100] S4, predicting a resource allocation strategy based on the updated task allocation model and with the goal of maximizing long-term benefit expectations, wherein the resource allocation strategy includes a task offloading decision and an energy allocation decision;
[0101] The expression for maximizing the expected long-term benefit is:
[0102]
[0103] In the formula, U(X, Φ) represents the maximum long-term benefit expectation, I represents the total number of time slices, u(·) represents the direct utility under time slice i, X1 represents the initial network state variable, X0 represents the value of the initial network state, Φ(X i ) indicates network status X i The decision function of X represents the set of network states.
[0104] The network status X i The expression is:
[0105]
[0106] In the formula, Represents the task queue under time slice i, represents the energy queue in time slice i, s i represents the connection relationship between the power terminal and the edge device in the i time slice, g i represents the channel gain in time slice i, represents the channel gain set.
[0107] In each time slice, when a task is taken out or generated, the task queue will be updated. The expression of the updated task queue is:
[0108]
[0109] Where, d i Indicates that the task execution is delayed. Indicates the number of tasks generated in time slice i, which are randomly generated according to Bernoulli distribution in each time slice. Represents the task queue under the i+1 time slice.
[0110] Similarly, the energy queue is updated in each time slice, and the expression of the updated energy queue is:
[0111]
[0112] In the formula, represents the energy queue under the i+1 time slice, It represents the number of energy units obtained by the power terminal in time slice i, which is randomly generated according to Poisson distribution in each time slice.
[0113] The connection relationship s between the power terminal and the edge device in the i time slice i It can be expressed as:
[0114]
[0115] In the formula, c i-1 represents the decision of task offloading in the i-1 time slice, s i-1 Indicates the connection relationship between the power terminal and the edge device in the i-1 time slice.
[0116] The direct utility of the i-th time slice is the task execution delay d i , the number of tasks abandoned η i 、Task queuing delay ρ i and the occupied edge node cost φ i Several indicators are jointly determined, and customized designs can be made for different goals. For example, if the system focuses on task execution delay d i , the task execution delay d can be increased i The weight of , thereby increasing its proportion in the entire utility. As an evaluation of the performance of the resource allocation strategy, its expression is:
[0117]
[0118] Where w1 represents the task execution delay d i The weight of w2 represents the task queuing delay ρ i The weight of w3 represents the number of tasks abandoned η i The weight of w4 represents the cost of occupying edge nodes φ i w1, w2, w3, and w4 are the weights of the target device, i.e., the device that generates the request, for different indicators. The values are determined by the device type of the target device and the application services it carries.
[0119] The task execution delay d i The calculation formula is:
[0120]
[0121] Where, d i Indicates the runtime delay of the task in time slice i, represents the time consumption of local execution task in time slice i, h i represents the switching delay in time slice i, represents the time cost of transmitting the task from the local to the corresponding edge device in the i-th time slice, d s It represents the delay incurred when the task is offloaded to the edge device for execution.
[0122] The switching delay h in the i time slice i The expression is:
[0123]
[0124] Where σ represents the connection switching delay, s i Indicates the connection relationship between the power terminal and the edge device in time slice i.
[0125] The time consumption of executing tasks locally under the i time slice The calculation formula is:
[0126]
[0127] In the formula, f i Indicates the CPU frequency allocated to the microgrid in time slice i.
[0128] The CPU frequency f allocated by the microgrid in the i time slice i The calculation formula is:
[0129]
[0130] Where τ is a constant that depends on the chip structure. Indicates the maximum frequency upper limit of the CPU chip.
[0131] The time overhead of transmitting the task from the local to the corresponding edge device in the i time slice The calculation formula is:
[0132]
[0133] In the formula, r i Indicates the data transmission rate in time slice i.
[0134] The data transmission rate r under the i time slice i The calculation formula is:
[0135]
[0136] In the formula, represents the channel gain between the power terminal and the edge device in the i-th time slice, represents the transmission power in the i-th time slice, i represents the sum of the power of interference and noise, and W represents the channel bandwidth.
[0137] The transmission power in the i time slice The calculation formula is:
[0138]
[0139] In the formula, Indicates the maximum transmission power.
[0140] According to equations (17), (18) and (19), the minimum transmission time satisfies:
[0141]
[0142] In the formula, represents the decision c of task offloading in time slice i i The corresponding channel gain, Indicates the minimum transmission time in time slice i.
[0143] Due to the limited computing resources of edge nodes, this application further defines the cost of occupying edge nodes to avoid excessive use of edge nodes, and defines π∈R + As the price per unit time of occupying edge nodes, R + is a set of positive real numbers, and the cost of occupying edge nodes is φ i The calculation formula is:
[0144]
[0145] The task queuing delay ρ i The calculation formula is:
[0146]
[0147] When the task queue is full, the newly generated tasks cannot be stored in the task queue, resulting in task loss. The task discard number η i The calculation formula is:
[0148]
[0149] The resource allocation strategy is expressed as:
[0150]
[0151] In the formula, Represents a collection of resource allocation strategies.
[0152] S5, the power terminal executes according to the resource allocation strategy predicted in step S4.
[0153] The embodiment of the present application also provides a microgrid group edge scheduling and intelligent agent lightweight tailoring device, including a power terminal and an edge device, wherein the power terminal is provided with a training module and a tailoring module, and the edge device is provided with an aggregation module and a decision module;
[0154] A training module is used for the power terminal to train the local training model using a reinforcement learning method according to the task allocation model parameters received from the edge device k, and the trained model and the number of local training times are sent to the cutting module;
[0155] The trimming module is used to trim the parameters of the received trained local training model, and send the trimmed local training model parameters and the number of local training times to the aggregation module;
[0156] Aggregation module: aggregates the trimmed local training model parameters according to the received local training times and updates the task allocation model;
[0157] Decision module: used to generate a resource allocation strategy including task offloading decision and energy allocation decision according to the updated task allocation model.
[0158] An embodiment of the present application also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the above-mentioned microgrid edge scheduling and intelligent agent lightweight trimming method are executed.
[0159] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned microgrid edge scheduling and intelligent agent lightweight tailoring method are executed. Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, the embodiment of the above-mentioned microgrid edge scheduling and intelligent agent lightweight tailoring method can be executed.
[0160] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for microgrid edge scheduling and agent lightweight tailoring, characterized in that: It includes the following steps: S1. Construct a microgrid group including power terminals and edge devices, and a task allocation model is set in edge device k; S2. Set the total number of training rounds T, the initial number of training rounds t = 0, initialize the local training models of each power terminal, the sparsity range, the aggregation weights of the local training models, and the experience replay memory of the power terminals; S3. The power terminal trains the task allocation model based on the deep reinforcement learning method, and prunes the local training model based on the model pruning method. Edge device k aggregates the trained models and updates the task allocation model; S4. According to the updated task allocation model, predict the resource allocation strategy with the goal of maximizing the long-term benefit expectation. The resource allocation strategy includes task offloading decisions and energy allocation decisions; S5. The power terminal executes tasks according to the resource allocation strategy predicted in step S4; The step S3 includes the following steps: S3.1, execute t=t+1, assign the random set of M power terminals to the set S t ; S3.2, Set S t Each power terminal in downloads the task allocation model from the edge device k and assigns it to the local training model of the power terminal; S3.
3. The power terminal senses and updates the experience replay memory; S3.
4. The power terminal trains the local training model using the deep reinforcement learning method and prunes the trained model parameters based on the model pruning method; S3.
5. Upload the parameters of the local training model pruned in step S3.4 and the number of local training times of the power terminal to edge device k; S3.
6. Edge device k separately counts the number of local training times of each power terminal; S3.
7. Edge device k aggregates the received model parameters according to the counted number of local training times; S3.
8. Judge whether t < T. If so, return to step S3.
1. Otherwise, execute step S4; The step S3.4 includes the following steps: S3.4.1, the set S t Initializing a neural network of each power terminal in the network, wherein the neural network includes a plurality of fully connected layers and a plurality of convolutional layers; S3.4.
2. Prune the parameters of the fully connected layer of the neural network; The conditions that the pruning needs to meet are: In the formula, m1 represents the number of fully connected layers in the neural network, represents the i-th fully connected layer after pruning fc A collection of layer parameters, Represents the i-th fully connected layer of the neural network fc The parameter set of the layer, and 1≤i fc ≤m1, represents the parameter set of the m1th layer of the pruned fully connected layer, Represents the parameter set of the m1th layer of the fully connected layer of the neural network, W1 ′ represents the parameter set of the first layer of the fully connected layer after pruning, and W1 represents the parameter set of the first layer of the fully connected layer of the neural network, that is, the fully connected layer before pruning; S3.4.
3. Set the sparsity of each layer of the fully connected layer according to the sparsity range. The corresponding setting formula is: In the formula, represents the i-th fully connected layer fc The sparsity of the layer, γ max represents the maximum sparsity, γ min represents the minimum sparsity; S3.4.4, arrange the parameters of each fully connected layer in order from small to large, and then The smaller parameter value is set to 0, where is the i-th fully connected layer fc The number of parameters of the layer; S3.4.
5. Set the sparsity of each layer of the convolutional layer according to the sparsity range. The corresponding setting formula is: In the formula, Represents the i-th convolutional layer c The sparsity of the layer; S3.4.
6. Prune the number of filters of each layer of the convolutional layer. The pruning requirements are: Where n l Indicates the number of filter kernels, F u represents the set of filters of each convolutional layer that are pruned, ||K|| represents the L2 norm of K, K represents the kernel in the filter, and l represents the label of the convolutional layer.
2. The microgrid edge scheduling and agent lightweight tailoring method according to claim 1 is characterized in that: In step S4, the expression for maximizing the long-term benefit expectation is: In the formula, U(X,Φ) represents the maximum long-term benefit expectation, I represents the total number of time slices, u(·) represents the direct utility, X1 represents the initial network state variable, X0 represents the value of the initial network state, Φ(X i ) indicates network status X i The decision function of X represents the set of network states.
3. The microgrid edge scheduling and agent lightweight tailoring method according to claim 2 is characterized in that: The network status X i The expression is: In the formula, Represents the task queue under time slice i, represents the energy queue in time slice i, s i represents the connection relationship between the power terminal and the edge device in the i time slice, g i represents the channel gain in time slice i, represents the channel gain set, N represents the number of edge devices, Indicates the maximum length of the task queue. Indicates the maximum length of the energy sequence.
4. The microgrid edge scheduling and agent lightweight tailoring method according to claim 2 is characterized in that: Direct utility u(X i ,Φ(X i )) is calculated as: Where w1 represents the task execution delay d i The weight of w2 represents the task queuing delay ρ i The weight of w3 represents the number of tasks abandoned η i The weight of w4 represents the cost of occupying edge nodes φ i The weight of .
5. The microgrid edge scheduling and agent lightweight tailoring method according to claim 4 is characterized in that: The task execution delay d i The calculation formula is: In the formula, represents the time consumption of local execution task in time slice i, h i represents the switching delay in time slice i, represents the time cost of transmitting the task from the local to the corresponding edge device in the i-th time slice, d s represents the delay caused when the task is offloaded to the edge device for execution, e i represents the number of energy units allocated in time slice i, c i represents the decision of task offloading in time slice i, Represents a collection of edge devices; The switching delay h in the i time slice i The expression is: Where σ represents the connection switching delay, s i Indicates the connection relationship between the power terminal and the edge device in the i-th time slice; The time consumption of executing tasks locally under the i time slice The calculation formula is: In the formula, f i Indicates the CPU frequency allocated to the microgrid in time slice i.
6. The microgrid edge scheduling and agent lightweight tailoring method according to claim 4 is characterized in that: The occupied edge node cost φ i The calculation formula is: In the formula, π represents the price per unit time of occupying edge nodes, h i represents the switching delay in time slice i, represents the time cost of transmitting the task from the local to the corresponding edge device in the i-th time slice, d s It represents the delay caused when the task is offloaded to the edge device for execution, and δ represents the duration of the time slice.
7. The microgrid edge scheduling and agent lightweight tailoring method according to claim 4 is characterized in that: The number of tasks discarded n i The calculation formula is: In the formula, represents the task queue under time slice i, δ represents the duration of the time slice, Indicates the number of tasks generated in time slice i, Indicates the maximum length of the task queue.
8. The microgrid edge scheduling and agent lightweight tailoring method according to claim 4 is characterized in that: The task queuing delay ρ i The calculation formula is: In the formula, Indicates the task queue under time slice i.
