A micro-grid data processing method and system based on federated learning

Abstract

The application discloses a micro-grid data processing method and system based on federal learning, comprising: classifying a plurality of micro-grid data obtained in each time slot to obtain a data difference threshold value, and comparing a plurality of first data difference values with the data difference threshold value respectively; obtaining a plurality of first transmission data to be trained which have a plurality of second data difference values greater than the data difference threshold value and have no repetition according to the data correlation being less than a correlation threshold value; establishing an optimization target of minimizing transmission time according to the first transmission data, and solving the optimization target according to deep reinforcement learning, and distributing the obtained centralized training result to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training according to the centralized training result; the data transmission amount can be reduced, and the communication quality can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart grid, and particularly relates to a micro-grid data processing method and system based on federated learning. BACKGROUND

[0002] With the development of 5G communication network technology and artificial intelligence technology, the construction of smart grid wireless communication network has been greatly increased, promoting the two-way flow of information flow and energy flow. With the popularization of new power systems, the rapid development of computing power networks and energy storage systems, distributed clean energy has been widely used. A large number of emerging bottom-layer devices such as clean energy are connected to the grid, and an open micro-grid is constructed to participate in the operation and regulation of the grid. The past centralized power system regulation has changed into a two-way control of smart two-way power and information flow, providing more reliable and flexible energy supply for users. Micro-grid can be considered as a regional cluster containing various distributed energy, energy storage devices and various loads, and there is a small data center to support the operation and maintenance of micro-grid. Traditional micro-grid can be divided into grid-connected mode and island mode, the former micro-grid is connected to the medium and low voltage distribution network, and the latter is a self-organized small-scale power network. With the demand for renewable energy, more and more micro-grids are integrated into the distribution network and put into operation.

[0003] In the micro-grid environment, the data of various bottom-layer nodes (i.e. supply and demand parties) are efficiently processed by using computing power networks, which can support stable, efficient and safe operation of various power grids. The micro-grid center controller and various bottom-layer nodes, such as clean energy generation nodes and power load nodes, need to interact with a large amount of data. The existing micro-grid power dispatching system model needs the bottom-layer nodes to transmit all the original data collected by the system to the center controller, and the center controller performs unified calculation. This will slow down the convergence speed of the overall power dispatching optimization model, and in addition, the centralized processing of the platform's power operation business system data faces the risk of data leakage. The federated learning strategy is characterized by distributed processing and centralized training, which effectively solves the data protection problem. Each bottom-layer node trains the model locally, and then uploads the training results (i.e. local variables) to the center control terminal, and the center control terminal updates the global variables of the optimization model, and returns them to the bottom-layer nodes for repeated iteration. The existing federated learning strategy directly uploads all the training results to the center control terminal in the task processing of the bottom-layer nodes, that is, the control terminal receives the processing data transmitted by the bottom-layer nodes at fixed intervals.

[0004] However, due to the following three characteristics of the underlying node data transmission, the efficiency of the existing federated learning strategy is not high. The characteristics of the underlying node data transmission include: 1) the amount of data to be transmitted is large, resulting in a large load of the controller. The underlying nodes include clean energy such as solar photovoltaic panels, which need to transmit as much data as possible to the central controller to obtain more detailed system data and obtain the global optimization value of the system. 2) There is obvious heterogeneity between nodes, and the existing federated learning strategy cannot well handle heterogeneous data. The amount of data collected by each underlying node and the available computing resources differ significantly, making it extremely difficult for the central controller to perform global model uniform iteration. For example, a certain underlying node may have completed multiple training of the local model, and the intermediate value accuracy of the local model parameters obtained is high; compared with, a certain node due to the large amount of computing resources and data to be processed, it may only complete a few training, and the intermediate value accuracy of the local model parameters obtained is low. The central controller receives the training results of each node, which differ greatly. If some local optimization model with poor training effect is directly integrated into the global optimal model framework, it will affect the convergence speed of the global optimization. 3) There is similarity between the data transmitted by the nodes. In some similar areas, the data to be transmitted by the nodes have certain similarities, and processing repeated similar data will increase the learning load of the federated learning strategy, and will not improve the learning efficiency. For example, the clean energy output of solar panels in the same area is similar. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a micro-grid data processing method and system based on federated learning, which can reduce the amount of data transmission and improve communication quality.

[0006] In a first aspect, the present application provides a micro-grid data processing method based on federated learning, comprising:

[0007] Classify the plurality of input data obtained for each time slot to obtain a data difference threshold value, and compare a plurality of first data difference values with the data difference threshold value respectively; wherein a first data difference value is obtained by a bottom node according to the distributed training result of the deep learning network;

[0008] According to the plurality of second data difference values greater than the data difference threshold value, a plurality of first transmission data to be trained are obtained, which have a data correlation less than a correlation threshold value and no repetition;

[0009] According to the first transmission data, an optimization objective of minimizing the transmission time is established, and the optimization objective is solved according to deep reinforcement learning, and the centralized training result obtained is distributed to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training according to the centralized training result.

[0010] The application classifies the input data of each time slot, can obtain a dynamic data difference threshold value, can adapt to a high dynamic change communication environment, and takes data exceeding the data difference threshold value as transmission data, further de-duplicates the transmission data, can reduce the transmission amount of repeated values, improve the utilization rate of computing resources, and trains the collected transmission data according to deep reinforcement learning, can further adapt to a high dynamic change channel state, and further improves the communication quality.

[0011] Further, the first transmission data to be trained, which is obtained according to a plurality of second data difference values greater than the data difference threshold value and has a data correlation less than a correlation threshold value and no repetition, comprises:

[0012] The second transmission data to be selected is obtained according to a plurality of second data difference values greater than the data difference threshold value, and the first transmission data is obtained according to a comparison between a correlation coefficient of every two second transmission data adjacent in position and a correlation threshold.

[0013] The application calculates a correlation coefficient of every two adjacent transmission data, can remove repeated values of repeated data with higher similarity of corresponding transmission data due to position adjacency, can accurately remove repeated values, and further reduces the data transmission amount and improves the quality of communication data.

[0014] Further, the first transmission data is obtained according to a comparison between a correlation coefficient of every two second transmission data adjacent in position and a correlation threshold, comprising:

[0015] A group of transmission data is removed from every two second transmission data greater than the correlation threshold to obtain third transmission data without repetition, and every two second transmission data less than the correlation threshold is integrated with the third transmission data to obtain the first transmission data.

[0016] Further, the plurality of first data difference values are compared with the data difference threshold value respectively, and the method further comprises:

[0017] When exceeding a preset iteration threshold value, a second data difference value less than the data difference threshold value is obtained, and fourth transmission data corresponding to the second data difference value is integrated with the first transmission data to obtain fifth transmission data to be trained, so that the fifth transmission data is concentratedly trained.

[0018] The application concentratesly trains transmission data exceeding a preset iteration threshold value, can prevent overfitting caused by always using transmission data exceeding a data difference threshold value for concentrated training, and can improve the quality of communication data.

[0019] Further, the solving the optimization target according to the deep reinforcement learning comprises:

[0020] According to the dynamic remaining computing resources and transmission environment, the deep reinforcement learning with initialized parameters is trained to obtain a centralized training result; wherein the centralized training result comprises a communication decision and a channel selection decision.

[0021] Further, the training by the deep reinforcement learning with initialized parameters comprises:

[0022] An action space, a state space and a reward function are established; wherein the state space comprises available computing resources, channel conditions, transmission power and data volume to be transmitted;

[0023] An action is obtained according to a network of the deep reinforcement learning; wherein the action comprises a communication decision and a channel selection decision;

[0024] Parameters of the network are updated according to the action and a value function in a direction of a minimum loss function to obtain a trained deep reinforcement learning.

[0025] Further, the reward function can be expressed as:

[0026]

[0027] wherein I is a number of bottom nodes, α i,j is a communication decision of the bottom node i, t i,j is a transmission delay of the bottom node i.

[0028] Preferably, the optimization target can be expressed as:

[0029]

[0030] s.t.α i,j ∈{0,1},β i,j ∈{0,1}

[0031] R i ≤R max ,

[0032] C i ≤C max ,

[0033] wherein α i,j is a communication decision of the bottom node i, t i,j is a transmission delay of the bottom node i, β i,j is a channel selection decision of the bottom node i, R i is a data transmission rate of the bottom node i, R max is a maximum allowed data transmission threshold value, and Ci is the remaining computing resource of the bottom node i, C max is the maximum allowed computing resource threshold, N and I are the number of time slots and the number of bottom nodes respectively.

[0034] Preferably, the data transmission rate can be represented as:

[0035]

[0036] wherein β i,j is the channel selection decision of the bottom node i, B represents the overall available bandwidth, K i represents the number of bottom nodes participating in data transmission, P i represents the transmission power of the bottom node, g i represents the channel transmission gain of the bottom node i, N0B represents Gaussian white noise, and ω represents mutual interference between transmission channels.

[0037] In a second aspect, the present application provides a micro-grid data processing system based on federated learning, comprising:

[0038] A data difference value acquisition module is configured to classify a plurality of micro-grid data obtained in each time slot to obtain a data difference threshold value, and compare a plurality of first data difference values with the data difference threshold value respectively; wherein a first data difference value is obtained by a bottom node according to a distributed training result of a deep learning network.

[0039] A screening module is configured to obtain a plurality of first transmission data to be trained with data correlation less than a correlation threshold value and no repetition according to a plurality of second data difference values greater than the data difference threshold value.

[0040] A communication resource optimization module is configured to establish an optimization target of minimizing transmission time according to the first transmission data, and solve the optimization target according to deep reinforcement learning, and distribute the obtained centralized training result to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training according to the centralized training result. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a communication schematic diagram of a micro-grid wireless communication network model provided by an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of a micro-grid data interaction model based on a federated learning framework provided by an embodiment of the present application;

[0043] Figure 3 is a flowchart of a micro-grid data processing method based on federated learning provided by the present application;

[0044] Figure 4 is a structural schematic diagram of a micro-grid data processing system based on federated learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0046] Referring to Figure 1 is a communication schematic diagram of a micro-grid wireless communication network model provided by an embodiment of the present application. The micro-grid wireless communication network model includes one central controller and a plurality of bottom nodes. The bottom nodes specifically include: a gateway and a smart meter of a residence, and various clean energy, such as a photovoltaic panel. The entire micro-grid wireless communication network model is operated in a time slot form. In each time slot, the central controller collects various information of the bottom nodes through the gateway, predicts the clean energy output and power load of the time slot, and performs global power scheduling, so as to achieve the system power supply and demand balance of the micro-grid.

[0047] The traditional centralized deep learning algorithm needs the bottom nodes to upload all the data to the cloud, and the cloud uniformly trains to obtain the optimal solution. However, this will cause the data leakage of the bottom nodes. The conventional residential power consumption data involves personal privacy and needs to be protected. In the power data interaction between the micro-grid central controller and the bottom nodes, it is not suitable to directly upload the original power consumption data. Federated learning is a typical distributed machine learning framework. The bottom nodes can perform distributed model training locally. Without interacting the local original data, only the intermediate results or model training parameters are interacted, and finally the data sharing and calculation are completed under the premise of protecting the local data.

[0048] However, the available computing resources and the amount of data to be processed between the underlying nodes differ greatly, which leads to differences in the accuracy of the localized computing model parameters processed by different nodes. If the cloud directly fuses them, it will seriously affect the overall performance of the system. From the data to be transmitted, the data to be transmitted by the nodes in the same area is similar, for example, the power generation of photovoltaic panels in the same area is similar, and the user power consumption law in the same cell also has similarities. Therefore, the application optimizes the selection of the data to be transmitted to reduce the data transmission amount and improve the data interaction efficiency. From the available communication resources for transmission, the time-varying characteristics of the wireless communication channel directly affect the communication quality, and a suitable deep learning strategy needs to be selected to adapt to the dynamically changing time-varying channel conditions. Based on this, the patent intends to use a heterogeneous federated learning framework to support the data interaction between the microgrid central controller and the underlying nodes.

[0049] Based on the above analysis, the application proposes a microgrid wireless communication network transmission strategy based on a federated learning framework, constructs an asynchronous federated learning framework to preliminarily optimize the training results between the underlying nodes, compares the accuracy of the training result parameters with the accuracy of the training model optimized by the center end globally, and fuses the transmission model with a higher accuracy than the predetermined value. At the same time, the correlation analysis method is used to analyze the correlation of the transmission data between different underlying nodes. Nodes with high correlation can choose one node to upload. After the selection of the underlying nodes and the transmission data, a communication resource allocation strategy based on deep reinforcement learning is further constructed, which greatly improves the convergence speed of the system to achieve global optimization, and realizes the deep fusion of the computing power network and the power network.

[0050] Referring to Figure 2 is a schematic diagram of a microgrid data interaction model based on a federated learning framework provided by the embodiment of the application. Frequent data interaction between the center controller and the underlying nodes relies on the federated learning framework. In each time slot, the underlying nodes first use local computing resources to perform distributed training on the local model. After the distributed training is completed, the results are transmitted to the center controller. The center controller optimizes and adjusts the global model after obtaining the transmission data of the underlying nodes, and then returns the global model parameters to the underlying nodes. Then, the underlying nodes train the localized model after obtaining the global optimization parameters of the center controller. Repeat iteration until global convergence optimization is achieved.

[0051] Referring to Figure 3 is a flowchart of a microgrid data processing method based on a federated learning provided by the application, which includes steps S11-S13, and specifically comprises:

[0052] In step S11, the plurality of micro-grid data obtained in each time slot is classified to obtain a data difference threshold value, and a plurality of first data difference values are compared with the data difference threshold value respectively; wherein a first data difference value is obtained by a bottom node according to the distributed training result of the deep learning network.

[0053] It is worth noting that the micro-grid system is controlled in time slots, so N time slots n={1, 2,...N} can be defined, and each time slot has a size of δ; the bottom node set is i={1, 2,...I}. All power users need to complete their entire tasks within a total length of T time. After completing the local model training, the bottom node i in the nth time slot needs to transmit the micro-grid data to the central controller. The micro-grid data includes the distributed training result of the bottom node and the energy data, and the energy data includes the clean energy data of the solar photovoltaic panel.

[0054] Specifically, at the beginning of each time slot, the central controller calculates the global optimization model and parameters, and classifies the micro-grid data according to the K-means clustering method of unsupervised learning to obtain the data difference threshold value. The central controller sends the initialized global model and initial parameters to the bottom nodes, and the bottom nodes respectively perform distributed training on the local model according to the available computing resources and the collected global model and initial parameters, and take the maximum mean difference value of the calculated distributed training result as the data difference value. The bottom node sends the data difference value to the central controller, and the central controller compares a plurality of data difference values with the data difference threshold value respectively, and allows the bottom node corresponding to the data difference value greater than the data difference threshold value to send the distributed training result to the central controller.

[0055] It is worth noting that, in order to prevent overfitting, when the preset iteration threshold value is exceeded, a second data difference value smaller than the data difference threshold value is obtained, and the fourth transmission data corresponding to the second data difference value is integrated with the first transmission data to obtain fifth transmission data to be trained, so as to perform centralized training according to the fifth transmission data.

[0056] In other words, when the preset iteration threshold value is exceeded, in the next iteration process, the bottom node will allow the distributed training result to be sent, and the distributed training result is integrated with the fourth transmission data. The integrated fifth transmission data is still non-repetitive transmission data, and the transmission data includes the distributed training result of each bottom node and the energy data. The central controller optimizes the global distributed training result, and sends the iteration updated centralized training result to all bottom nodes, and according to the collected energy data, the clean energy output and the power load are predicted, and the global power dispatching is performed, so as to achieve the system power supply and demand balance of the micro-grid.

[0057] The application adopts centralized training according to transmission data exceeding a preset iteration threshold value, can prevent overfitting caused by centralized training using transmission data exceeding a data difference threshold value all the time, and thus can improve the quality of communication data.

[0058] It is worth noting that in the microgrid, the transmission data of the bottom nodes in the same area has similarity. For example, the output of clean energy of the solar photovoltaic panels in the same area in the same time period is similar. Based on this, part of the bottom nodes can choose to transmit only part of the data, minimizing the data transmission delay without affecting the overall system performance. In general, first, the output data of the bottom nodes allowed to upload data is analyzed using a correlation analysis method to obtain repeated output values between the bottom nodes; then the output data of each bottom node is divided into repeated data and data that must be transmitted; a joint optimization problem is constructed to process the repeated data of each bottom node, and only the processed eigenvalues need to be transmitted, which will consume the computing resources of the bottom nodes, and the transmission environment is also dynamically changing. Based on this, the bottom nodes in the heterogeneous federated learning framework can select to process data or not according to the dynamic node remaining computing resources and transmission environment, that is, the communication decision of the bottom nodes is made, that is, the bottom nodes are allowed to communicate to transmit or not allowed to communicate, and the overall data transmission delay is minimized, and finally the data interaction strategy with the shortest delay is obtained.

[0059] In step S12, a plurality of first transmission data to be trained are obtained according to a plurality of second data difference values greater than the data difference threshold value and having a data correlation less than a correlation threshold value and no repetition.

[0060] Specifically, the second transmission data to be selected is obtained according to a plurality of second data difference values greater than the data difference threshold value, and the first transmission data is obtained according to a comparison between the correlation coefficient of each pair of second transmission data adjacent in position and the correlation threshold.

[0061] The application calculates the correlation coefficient of each pair of adjacent transmission data, can remove the repeated values of the repeated data with higher similarity of corresponding transmission data due to the position adjacency, can accurately remove the repeated values, and thus can reduce the data transmission amount and improve the quality of communication data.

[0062] The first transmission data is obtained according to a comparison between the correlation coefficient of each pair of second transmission data adjacent in position and the correlation threshold, including: removing a group of transmission data from each pair of second transmission data greater than the correlation threshold to obtain third transmission data without repetition, and integrating each pair of second transmission data less than the correlation threshold with the third transmission data to obtain the first transmission data.

[0063] It is worth mentioning that the bottom nodes complete data collection and training according to the global model and initial parameters assigned by the center controller, and output second transmission data. The transmission data between adjacent bottom nodes is transmitted and analyzed in pairs. When the correlation is greater than the preset correlation threshold, the transmission data between two bottom nodes has great similarity and can be regarded as repeated data. At this time, the bottom node transmission data is divided into two categories according to whether it is repeated, including: repeated data and non-repeated data.

[0064] Preferably, the correlation coefficient is calculated according to the Pearson correlation coefficient.

[0065] Preferably, the transmission data can be expressed as:

[0066]

[0067] wherein, is one of the two groups of repeated data, [s i,j+1 ,…,s i,N ] is non-repeated data. For the data interaction between the center controller and the bottom nodes in the federated learning framework, the center controller only needs to receive a group of original repeated data, and the repeated data of the remaining bottom nodes can be processed locally under the condition that the bottom node computing resources are sufficient. The center controller only needs to receive a group of original repeated data, and the remaining bottom nodes can process the results of further processing, that is, the eigenvalues, and transmit them to the center controller.

[0068] Preferably, the transmission delay of the repeated data can be expressed as:

[0069]

[0070] wherein, C i is the available resource, α i,j ∈{0,1} is the communication decision of the bottom node i, 0 represents not performing transmission data sending, and 1 represents performing transmission data sending, represents the output data amount of the bottom node i after further processing, is the output data amount of the bottom node i without further processing, obviously, R i represents the transmission rate of the bottom node i to the center controller.

[0071] Step S13, according to the first transmission data, an optimization target of minimizing transmission time is established, and the optimization target is solved according to deep reinforcement learning. The centralized training result obtained is distributed to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training according to the centralized training result.

[0072] It is worth mentioning that in the given federated learning framework, the center controller builds the optimization objective and solves the optimization problem, and feeds back the optimization result to the underlying nodes. Specifically, the center controller as the decision center optimizes the communication decision and channel selection decision according to the dynamically changing available computing resources of the underlying nodes and the transmission link speed between the underlying nodes and the center controller, with the goal of minimizing the overall underlying node data transmission delay.

[0073] Preferably, the optimization objective can be represented as:

[0074]

[0075] s.t.α i,j ∈{0,1},β i,j ∈{0,1}

[0076] R i ≤R max ,

[0077] C i ≤C max ,

[0078] wherein α i,j is the communication decision of the underlying node i, t i,j is the transmission delay of the underlying node i, β i,j is the channel selection decision of the underlying node i, R i is the data transmission rate of the underlying node i, R max is the maximum allowed data transmission threshold value, C i is the remaining computing resources of the underlying node i, C max is the maximum allowed computing resource threshold value, N and I are the number of time slots and the number of underlying nodes, respectively.

[0079] Preferably, the data transmission rate can be represented as:

[0080]

[0081] wherein β i,j is the channel selection decision of the underlying node i, B represents the overall available bandwidth, K i represents the number of underlying nodes participating in data transmission, P i represents the transmit power of the underlying node, g i represents the channel transmission gain of the underlying node i, N0B represents the Gaussian white noise, and ω represents the mutual interference between transmission channels.

[0082] The main difficulty in solving the optimization objective is that the available computing resources of the bottom nodes dynamically change, and the transmission environment between the bottom nodes and the central controller is also dynamically changing; in addition, the problem is a typical mixed integer programming problem, which can be optimized and solved by deep reinforcement learning.

[0083] Solving the optimization objective according to deep reinforcement learning includes: training by deep reinforcement learning of initialized parameters according to dynamic remaining computing resources and transmission environment to obtain a centralized training result; wherein the centralized training result includes communication decision and channel selection decision.

[0084] Specifically, the training by deep reinforcement learning of initialized parameters includes: establishing an action space, a state space and a reward function; wherein the state space includes available computing resources, channel conditions, transmission power and data amount to be transmitted; obtaining an action according to a network of deep reinforcement learning; wherein the action includes communication decision and channel selection decision; updating parameters of the network according to the action and a value function in the direction of a minimum loss function to obtain trained deep reinforcement learning.

[0085] Preferably, deep Q-learning (DQN) is used to solve the optimization objective.

[0086] DQN (Deep Q-Learning) is a common method of deep reinforcement learning, which is used for solving optimization problems and can be regarded as a standard Markov decision process. In order to better describe the solving method of DQN, the state space, action space and reward function are first defined.

[0087] Since the dynamic available computing resources of the bottom nodes are directly related to the channel conditions, and the transmission power and the data amount to be transmitted of the bottom nodes also affect the final communication decision and channel selection decision. Therefore, the state space can be represented as: [C i ,g i ,P i ,S i,j ], wherein C i is the dynamic available computing resources of the bottom nodes, g i is the channel condition, P i is the transmission power, and S i,j is the transmission data amount of the data to be transmitted.

[0088] In the DQN model, the bottom nodes make decisions according to the real-time changing environment. The decision variable is a set of binary variables, which respectively represent whether to communicate and whether to select an access channel, i.e. whether to process a task and whether to select an access channel. Therefore, the action space can be represented as: a i =[αi,j ,β i,j ],α i,j is the communication decision of the underlying node i, β i,j is the channel selection decision of the underlying node i.

[0089] According to the determined optimization target is to minimize the task processing delay. To this end, the optimization target is used as a reward function in the DQN model.

[0090] The reward function can be expressed as:

[0091]

[0092] wherein I is the number of underlying nodes, α i,j is the communication decision of the underlying node i, t i,j is the transmission delay of the underlying node i.

[0093] When the reward is maximum, the best action a i * When the underlying node is in state s i , if action a i is taken, it will get reward r i and next time state s i+1 , denoted as

[0094] In order to evaluate the reward, the Q function Q(s t , a t ) is defined as:

[0095]

[0096] wherein Q new (s i , a i ) represents the new Q function value of the current time slot, ζ represents the learning rate, r i represents the reward, γ represents the discount factor, and Q'(s', a') is the Q value under the given new state and action. The update formula of the Q function is expressed as:

[0097]

[0098] wherein γ is the discount factor, ζ is the learning rate, Q(s i+1 , a i+1 ) is the Q value of the next iteration, r i is the reward function, and the DQN can use a neural network to simulate the Q function, which can be approximated as: Q(s i , a i ) = Q(s i , a i; theta), where theta represents parameters. i i ) and Q(s i i ; theta), the difference between which is expressed by a mean squared error loss function, which can be expressed as:

[0099]

[0100] where m is the number of samples for centralized training, and theta i is the parameter set of the Q network.

[0101] where y i represents the Q value of the target network, which is expressed as:

[0102]

[0103] where theta is the parameter set of the target network, and the loss function is minimized by gradient update to update the Q network parameters, and the parameters theta are updated in the direction of the minimum loss function, which can be expressed as

[0104]

[0105] where, represents the gradient value of the parameters theta.

[0106] Finally, the bottom layer nodes transmit relevant data to the central controller of the micro-grid according to the optimized solution, the central controller unifies the transmission data of each bottom layer node and processes it, and then returns the global model and global parameters obtained by the centralized training model to each bottom layer node, and then iterates repeatedly, so that the model between the cloud and the edge cluster with the minimum resource consumption under the condition of not affecting the system performance is finally obtained, to support power system scheduling.

[0107] The present application proposes a micro-grid wireless communication network transmission strategy of a federal learning framework to support the integration of micro-grids into new power systems, which can optimize and filter the output data of the bottom layer nodes to adapt to the dynamic computing resources and computing demands of the bottom layer nodes, minimize the data upload delay of the bottom layer nodes, improve the overall performance of the system, and reduce the data transmission amount and improve the quality of communication data.

[0108] Referring to Figure 4 ​​Fig. 1 is a structural schematic diagram of a micro-grid data processing system based on federated learning provided by an embodiment of the present application, comprising a data difference value acquisition module 41, a screening module 42 and a communication resource optimization module 43.

[0109] The data difference value acquisition module 41 obtains a data difference threshold value according to the micro-grid data of each time slot, compares a plurality of first data difference values with the data difference threshold value, and sends the comparison result to the screening module 42. The screening module 42 obtains a plurality of first transmission data to be trained with data correlation less than a correlation threshold value and no repetition according to the comparison result of a plurality of second data difference values greater than the data difference threshold value, and sends the first transmission data to the communication resource optimization module 43 for resource optimization. After receiving the first transmission data, the communication resource optimization module 43 establishes an optimization optimization target, solves according to deep reinforcement learning, and distributes the obtained centralized training result to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training of their own local models according to the centralized training result.

[0110] The data difference value acquisition module 41 is used for classifying a plurality of micro-grid data obtained for each time slot to obtain a data difference threshold value, and comparing a plurality of first data difference values with the data difference threshold value respectively; wherein a first data difference value is obtained by a bottom node according to the distributed training result of a deep learning network.

[0111] Comparing a plurality of first data difference values with the data difference threshold value respectively further comprises: when exceeding a preset iteration threshold value, obtaining a second data difference value less than the data difference threshold value all the time, and integrating fourth transmission data corresponding to the second data difference value with the first transmission data to obtain fifth transmission data to be trained, so as to perform centralized training according to the fifth transmission data.

[0112] The screening module 42 is used for obtaining a plurality of first transmission data to be trained with data correlation less than a correlation threshold value and no repetition according to a plurality of second data difference values greater than the data difference threshold value.

[0113] Specifically, it comprises obtaining second transmission data to be selected according to a plurality of second data difference values greater than the data difference threshold value, and obtaining first transmission data according to the comparison of the correlation coefficient of every two second transmission data adjacent in position with the correlation threshold. Wherein, obtaining first transmission data according to the comparison of the correlation coefficient of every two second transmission data adjacent in position with the correlation threshold comprises: removing a group of transmission data from every two second transmission data greater than the correlation threshold to obtain third transmission data with no repetition, and integrating every two second transmission data less than the correlation threshold with the third transmission data to obtain first transmission data.

[0114] The communication resource optimization module 43 is configured to establish an optimization target of minimizing transmission time according to the first transmission data, and solve the optimization target according to deep reinforcement learning, and distribute a centralized training result obtained to all bottom nodes, so that the bottom nodes perform distributed execution and distributed training according to the centralized training result.

[0115] In order to solve the problem of massive data transmission of bottom nodes in a micro-grid system, the micro-grid data processing system based on federated learning is provided. In order to improve the data interaction efficiency between the central controller and the bottom nodes in the micro-grid, an asynchronous federated learning strategy is proposed. Under the support of cloud-edge fusion architecture, the local model training accuracy of the bottom nodes is used to screen the nodes, and a global model parameter iteration strategy is constructed at the control end to improve the efficiency of the fusion network of the central controller and the bottom nodes. At the same time, the correlation analysis is used to analyze the data between the nodes, and part of the data is selected to participate in the global model iteration, so as to realize the deep fusion of the computing power network and the power network. In addition, after the bottom nodes and the transmission data are selected, the communication resource allocation strategy of deep reinforcement learning is proposed, which can cope with the dynamic changes of transmission demand and channel, and maximize the transmission efficiency of the bottom nodes and the micro-grid central controller.

[0116] Preferably, the optimization target can be represented as:

[0117]

[0118] s.t.α i,j ∈{0,1},β i,j ∈{0,1}

[0119] R i ≤R max ,

[0120] C i ≤C max ,

[0121] Wherein, α i,j is the communication decision of the bottom node i, t i,j is the transmission delay of the bottom node i, β i,j is the channel selection decision of the bottom node i, R i is the data transmission rate of the bottom node i, R max is the maximum allowed data transmission threshold value, C i is the remaining computing resource of the bottom node i, C max is the maximum allowed computing resource threshold value, and N and I are the number of time slots and the number of bottom nodes, respectively.

[0122] Preferably, the data transmission rate can be represented as:

[0123]

[0124] wherein, β i,j is the channel selection decision of the bottom node i, B represents the overall available bandwidth, K i represents the number of bottom nodes participating in data transmission, P i represents the transmission power of the bottom node, g i represents the channel transmission gain of the bottom node i, N0B represents Gaussian white noise, and ω represents mutual interference between transmission channels.

[0125] The main difficulty in solving the optimization objective is that the available computing resources of the bottom node dynamically change, and the transmission environment between the bottom node and the central controller is also dynamically changed; in addition, the problem is a typical mixed integer programming problem, which can be optimized and solved by deep reinforcement learning.

[0126] According to the deep reinforcement learning, the optimization objective is solved, including: according to the dynamic remaining computing resources and the transmission environment, training is performed through the deep reinforcement learning of the initialized parameters to obtain a centralized training result; wherein the centralized training result includes a communication decision and a channel selection decision.

[0127] Specifically, the training through the deep reinforcement learning of the initialized parameters includes: establishing an action space, a state space and a reward function; wherein the state space includes available computing resources, channel conditions, transmission power and data to be transmitted; according to the network of the deep reinforcement learning, an action is obtained; wherein the action includes a communication decision and a channel selection decision; according to the action and the value function, the parameters of the network are updated in the direction of the minimum loss function to obtain the trained deep reinforcement learning.

[0128] The reward function can be represented as:

[0129]

[0130] wherein, I is the number of bottom nodes, α i,j is the communication decision of the bottom node i, t i,j is the transmission delay of the bottom node i.

[0131] The application classifies the input data of each time slot, can obtain a dynamic data difference threshold value, can adapt to a high dynamic change communication environment, and takes data exceeding the data difference threshold value as transmission data, further de-duplicates the transmission data, can reduce the transmission amount of repeated values, improve the utilization rate of computing resources, and trains the collected transmission data according to the deep reinforcement learning, can further adapt to a high dynamic change channel state, and further improves the communication quality.

[0132] Those skilled in the art will appreciate that embodiments of the application can also provide a computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0133] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0134] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0136] The above description is only preferred embodiments of the application. It should be pointed out that for those skilled in the art, without departing from the technical principles of the application, a number of improvements and modifications can be made, which should be considered as falling within the scope of the application.

Claims

1. A microgrid data processing method based on federated learning, characterized in that, include: Multiple microgrid data acquired in each time slot are classified to obtain a data difference threshold value, and multiple first data difference values ​​are compared with the data difference threshold value respectively; wherein, a first data difference value is obtained by a bottom node based on the distributed training results of a deep learning network, and the comparison of multiple first data difference values ​​with the data difference threshold value respectively further includes: when the data difference value exceeds a preset iteration threshold value, a second data difference value that is always less than the data difference threshold value is obtained, and the fourth transmission data corresponding to the second data difference value is integrated with the first transmission data to obtain a fifth transmission data to be trained, so as to perform centralized training based on the fifth transmission data; Based on multiple second data difference values ​​that are greater than the data difference threshold, multiple first transmission data to be trained with data correlation less than the correlation threshold and no repetition are obtained. Based on the first transmitted data, an optimization objective to minimize transmission time is established, and the optimization objective is solved using deep reinforcement learning. The obtained centralized training results are distributed to all bottom-level nodes so that the bottom-level nodes can perform distributed execution and distributed training based on the centralized training results.

2. The microgrid data processing method based on federated learning as described in claim 1, characterized in that, The step of obtaining multiple sets of first transmission data for training that have a correlation threshold less than a correlation threshold and no repetition based on multiple second data difference values ​​greater than the data difference threshold includes: Based on multiple second data difference values ​​that are greater than the data difference threshold, the second transmission data to be selected is obtained, and based on the comparison of the correlation coefficients of two adjacent second transmission data with the correlation threshold, the first transmission data is obtained.

3. The microgrid data processing method based on federated learning as described in claim 2, characterized in that, The first transmission data is obtained by comparing the correlation coefficients and correlation thresholds of two adjacent pairs of second transmission data, including: Remove one pair of transmission data from each pair of second transmission data that is greater than the correlation threshold to obtain a third transmission data without repetition. Then, integrate the pair of second transmission data that is less than the correlation threshold with the third transmission data to obtain the first transmission data.

4. The microgrid data processing method based on federated learning as described in claim 1, characterized in that, Solving the optimization objective based on deep reinforcement learning includes: Based on the dynamic remaining computing resources and transmission environment, deep reinforcement learning with initialization parameters is used for training to obtain centralized training results; wherein, the centralized training results include communication decisions and channel selection decisions.

5. The microgrid data processing method based on federated learning as described in claim 4, characterized in that, The training via deep reinforcement learning with initialized parameters includes: Establish an action space, a state space, and a reward function; wherein, the state space includes available computing resources, channel conditions, transmit power, and the amount of data to be transmitted; Based on a deep reinforcement learning network, actions are obtained; wherein, the actions include communication decisions and channel selection decisions; The parameters of the network are updated according to the action and value function in the direction of the minimum loss function to obtain the trained deep reinforcement learning.

6. The microgrid data processing method based on federated learning as described in claim 5, characterized in that, The reward function is expressed as follows: ， in, This represents the number of underlying nodes. For the bottom layer node Communication decisions, For the bottom layer node Transmission delay.

7. The microgrid data processing method based on federated learning as described in claim 4, characterized in that, The optimization objective is expressed as: ， s.t. , ， ， in, For the bottom layer node Communication decisions, For the bottom layer node Transmission delay, For the bottom layer node Channel selection decision, For the bottom layer node Data transmission rate, This is the maximum allowed data transmission threshold. For the bottom layer node The remaining computing resources This is the maximum allowed computing resource threshold. and These represent the number of time slots and the number of underlying nodes, respectively.

8. The microgrid data processing method based on federated learning as described in claim 7, characterized in that, The data transmission rate is expressed as: ， in, For the bottom layer node Channel selection decision, This represents the total available bandwidth. This indicates the total number of underlying nodes participating in the overall data transmission. This indicates the transmit power of the underlying node. Represents the underlying node Channel transmission gain, This represents Gaussian white noise. This indicates mutual interference between transmission channels.

9. A microgrid data processing system based on federated learning, characterized in that, include: The data difference value acquisition module is used to classify multiple microgrid data acquired in each time slot to obtain a data difference threshold value, and compare multiple first data difference values ​​with the data difference threshold value respectively; wherein, a first data difference value is obtained by a bottom node based on the distributed training results of a deep learning network, and the comparison of multiple first data difference values ​​with the data difference threshold value respectively further includes: when the data difference value exceeds a preset iteration threshold value, acquiring a second data difference value that is always less than the data difference threshold value, and integrating the fourth transmission data corresponding to the second data difference value with the first transmission data to obtain a fifth transmission data to be trained, so as to perform centralized training based on the fifth transmission data; The filtering module is used to obtain multiple first transmission data to be trained that have a data correlation less than the correlation threshold and no repetition, based on multiple second data difference values ​​that are greater than the data difference threshold. The communication resource optimization module is used to establish an optimization objective that minimizes the transmission time based on the first transmission data, solve the optimization objective based on deep reinforcement learning, and distribute the obtained centralized training results to all bottom-level nodes so that the bottom-level nodes can perform distributed execution and distributed training based on the centralized training results.

Images