Electric power multi-modal data sharing method based on federal learning

Abstract

The invention provides an electric power multi-modal data sharing method based on federated learning, and the method comprises the steps: obtaining and carrying out the preprocessing operation of the electric power multi-modal data, initializing a federated learning framework, distributing the federated learning framework to each participating node, training a local model of the federated learning based on the electric power multi-modal data, and carrying out the training of the local model of the federated learning. Each participating node sends the parameters of the local model obtained by training to the central server or the coordination node for aggregation operation to obtain global model parameters, the central server or the coordination node distributes the global model parameters back to each participating node, and each participating node updates the local model according to the global model parameters and carries out training of the next iteration period to obtain the global model parameters. And repeating the above steps until a preset stop condition is met, ending iteration, obtaining a global model, evaluating the global model, if an evaluation result meets a preset requirement, completing training, otherwise, repeating the steps until the training is completed. The model performance in the power field can be improved.

Description

Technical Field

[0001] The present invention relates to the field of power data technology, and in particular to a method for sharing power multimodal data based on federated learning. Background Art

[0002] As a key energy source for various social activities, the country's annual electricity consumption data can directly reflect the level of social and economic development in my country. Conducting research on power big data and fully extracting the social and commercial value of power data have become the direction of information development in the power industry. The key points of power multimodal big data processing include: 1. Data collection and fusion. The power system involves multiple links such as power generation, transmission, distribution, and power consumption, and each link will generate a large amount of data. These data include: electrical data: such as voltage, current, power, frequency, etc. Equipment data: such as equipment model, operating status, fault information, etc. Environmental data: such as temperature, humidity, wind speed, etc. Personnel data: such as inspection records, maintenance records, etc. 2. Data processing and analysis. Power multimodal big data processing requires cleaning, preprocessing, feature extraction, model training, analysis and prediction of the collected data.

[0003] Power multimodal data fusion is a crucial link in the power system. It involves the effective integration of data from different sources and modes to provide more comprehensive and accurate information to support the operation, maintenance, analysis and decision-making of the power system. At present, there are three main fusion methods for power multimodal data fusion: data-level fusion, decision-level fusion and feature-level fusion. Among them, data-level fusion: multiple independent data sets are fused into a single feature vector, which is then input into the machine learning classifier. This method can make full use of the rich information of the original data, but may face challenges in data redundancy and feature extraction. In practical applications, front-end fusion is often combined with feature extraction methods, such as principal component analysis (PCA) and maximum relevance minimum redundancy algorithm (mRMR) to eliminate redundant information and extract key features. Decision-level fusion: The output scores (decisions) of the classifiers trained separately for different modal data are fused. This method can retain the independence and characteristics of each modal data and synthesize them at the decision level. Ensemble learning is a typical representative of decision-level fusion. It improves the accuracy and robustness of classification by combining the outputs of multiple classifiers. Feature-level fusion: It lies between data-level fusion and decision-level fusion and can be performed at any stage of data processing. Feature-level fusion has great flexibility and can select appropriate fusion positions and strategies according to specific application scenarios and requirements.

[0004] However, there are two main technical disadvantages of the existing power data analysis methods: 1. The existing data processing methods rely on centralized data storage and processing, which brings inconvenience to data sharing and privacy protection; 2. The existing power data processing methods cannot adapt to the data fusion analysis and processing requirements of a large amount of heterogeneous multimodal power data.

[0005] Therefore, it is necessary to design a method for sharing power multimodal data based on federated learning. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for sharing power multimodal data based on federated learning.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A first aspect of the present invention provides a method for sharing electric power multimodal data based on federated learning, comprising:

[0009] Acquire multimodal power data and perform preprocessing operations on it;

[0010] Initialize the federated learning framework, determine the nodes participating in the federated learning, establish a secure communication channel, and distribute the federated learning framework to each participating node;

[0011] The local model of federated learning is trained based on the power multimodal data of participating nodes after preprocessing operations;

[0012] Each participating node sends the parameters of the local model obtained through training to the central server or coordination node, which aggregates the received parameters to obtain the global model parameters.

[0013] The central server or coordination node distributes the global model parameters back to each participating node. Each participating node updates the local model based on the received global model parameters and conducts training for the next iteration cycle.

[0014] Repeat the above steps until the preset stop condition is met, end the iteration, and obtain the global model;

[0015] The global model is evaluated. If the evaluation result meets the preset requirements, the training is completed. If not, the local model training, model parameter aggregation, and global model update and distribution are repeated until a global model that meets the preset requirements is obtained.

[0016] Preferably, the power multimodal data is obtained and preprocessed, specifically:

[0017] The participating nodes acquire multimodal data from different power data sources and perform cleaning, denoising and normalization operations on the acquired data.

[0018] Preferably, the federated learning framework includes a feature preprocessing module, a secondary feature extraction module, an attention module and an MLP module. The feature preprocessing module is used to extract features of multimodal data based on a pre-trained neural network to obtain preliminary features; the secondary feature extraction module is used to learn the temporal features and probability distribution information of the preliminary features to obtain multimodal features; the attention module is used to process multimodal features through an attention mechanism; the MLP module is a multilayer perceptron, which is used to output the final prediction result.

[0019] Preferably, the feature preprocessing module is provided with three pre-trained neural networks, namely MobileNetV2, MobileBERT and MFCC, wherein MobileNetV2 is used for video feature extraction, MobileBERT is used for text feature extraction, and the MFCC is used for audio feature extraction.

[0020] Preferably, the central server or coordination node performs aggregation operation on the received parameters through FedAvg algorithm.

[0021] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0022] The present invention provides a method for sharing electric power multimodal data based on federated learning, including obtaining electric power multimodal data, performing preprocessing operations on the data, initializing a federated learning framework, determining nodes participating in the federated learning, establishing a secure communication channel, distributing the federated learning framework to each participating node, training the local model of the federated learning based on the electric power multimodal data of the participating nodes after the preprocessing operation, each participating node sends the parameters of the trained local model to a central server or a coordination node, the central server or the coordination node aggregates the received parameters to obtain global model parameters, the central server or the coordination node distributes the global model parameters back to each participating node, each participating node updates the local model according to the received global model parameters, performs training for the next iteration cycle, repeats the above steps until a preset stop condition is met, ends the iteration, obtains the global model, evaluates the global model, completes the training if the evaluation result meets the preset requirements, and repeats the local model training, model parameter aggregation, and global model update and distribution if it does not meet the preset requirements until a global model that meets the preset requirements is obtained. The present invention can improve the model performance in the electric power field by using distributed data while protecting privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0025] Figure 2 is an algorithm logic diagram of an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the federated learning framework structure;

[0027] Figure 4 It is a structural diagram of the secondary feature extraction module;

[0028] Figure 5 Schematic diagram comparing single-modal training and multi-modal training. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The purpose of the present invention is to provide a method for sharing electric power multimodal data based on federated learning, which can improve the model performance in the electric power field by using distributed data while protecting privacy.

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Figure 1 A flow chart of a method provided by an embodiment of the present invention, Figure 2 is an algorithm logic diagram of an embodiment of the present invention, such as Figure 1 and Figure 2 As shown, the present invention provides a method for sharing electric power multimodal data based on federated learning, comprising:

[0033] Step 100: Acquire power multimodal data and perform preprocessing operations on it;

[0034] Step 200: Initialize the federated learning framework, where the schematic diagram of the federated learning framework structure is as follows: Figure 3As shown, the nodes participating in federated learning are determined, a secure communication channel is established, and the federated learning framework is distributed to each participating node;

[0035] Step 300: training a local model of federated learning based on the power multimodal data of the participating nodes after the preprocessing operation;

[0036] Step 400: Each participating node sends the parameters of the local model obtained through training to the central server or the coordination node, and the central server or the coordination node aggregates the received parameters to obtain global model parameters;

[0037] Step 500: The central server or coordination node distributes the global model parameters back to each participating node, and each participating node updates the local model according to the received global model parameters and performs training for the next iteration cycle;

[0038] Step 600: Repeat the above steps until a preset stop condition is met, and then the iteration ends to obtain a global model;

[0039] Step 700: Evaluate the global model. If the evaluation result meets the preset requirements, the training is completed. If not, repeat the local model training, model parameter aggregation, and global model update and distribution until a global model that meets the preset requirements is obtained.

[0040] In step 100, the multimodal power data is acquired and preprocessed, specifically:

[0041] Participating nodes acquire multimodal data from different power data sources and perform preprocessing operations such as cleaning, denoising and normalization on the acquired data to ensure data quality and consistency. In addition, features related to power prediction, analysis or control can also be extracted as needed.

[0042] In step 200, the federated learning framework is initialized, specifically:

[0043] Initialize the federated learning framework and set the parameters of the federated learning system, including learning rate, number of iterations, communication protocol, etc.

[0044] The federated learning framework includes a feature preprocessing module, a secondary feature extraction module, an attention module and an MLP module. The feature preprocessing module is used to extract features of multimodal data based on a pretrained neural network to obtain preliminary features; the secondary feature extraction module is used to learn the temporal features and probability distribution information of the preliminary features to obtain multimodal features; the attention module is used to process multimodal features through an attention mechanism; the MLP module is a multilayer perceptron, which is used to output the final prediction results.

[0045] 1. Introduce the feature preprocessing module. After feature preprocessing, the processed output features can be obtained. Among them, l represents the size of the data sequence, which is determined by the size of the input data itself, and d is the data dimension, which is related to the feature extraction method used. For example, after 60 seconds of video data is input into the MobileNetV2 pre-trained network, the data input into the downstream deep neural network The 2-second audio data can be input into the downstream deep neural network by resampling to 16000Hz, frame length 40, frame shift 20, and MFCC with Mel filter group 80. The text data needs to be determined according to the division of the word segmenter and the length of the text. The data dimension of the data using the MobileBERT model after processing is 512, which is determined by the word segmenter. The data sequence size and data dimension of other data are also determined as above. It should be noted that the downstream deep neural network can process data of different sequence sizes, but the data dimension needs to be the same. The feature preprocessing module recorded in this application is mainly equipped with three pre-trained neural networks, namely MobileNetV2, MobileBERT and MFCC, which are used for different feature extraction of three modalities of video (image), text and audio respectively;

[0046] MobileNetV2 is used for video feature extraction. MobileNetV2 is a lightweight convolutional neural network, which is usually used for tasks such as image classification and target detection. Although it is designed for images, it can be applied to video feature extraction by frame-by-frame processing (decomposing the video into a series of image frames);

[0047] MobileBERT is used for text feature extraction. MobileBERT is a lightweight variant of BERT (Bidirectional Encoder Representations from Transformers) for natural language processing tasks. MobileBERT can be used to extract text features. First, the text needs to be preprocessed, including word segmentation, vocabulary building, and conversion of text into token sequences. The preprocessed text is input into MobileBERT, and its output is obtained as the feature representation of the text, which is usually achieved by taking the output of a certain layer in the BERT model or the output of the entire model.

[0048] MFCC is used for audio feature extraction. First, the audio signal needs to be preprocessed, including framing, windowing, and calculating the Fourier transform of each frame. Then, the MFCC of each frame can be calculated, which involves converting the result of the Fourier transform to the Mel frequency scale and applying the discrete cosine transform (DCT) to extract the cepstral coefficients.

[0049] Second, the secondary feature extraction module is introduced. The secondary feature extraction module learns the temporal features and probability distribution information from different modal data through the CNN-GRU neural network contained in the module. The network structure of this module is as follows: Figure 4 shown.

[0050] 3. Introduction to the attention module. In order to better integrate data from different sources in the power multimodal federated learning framework, the attention module uses the attention mechanism to process multimodal features. The attention mechanism calculates weights so that the model can assign different importance to different positions. This weight calculation method not only makes the model more flexible, but also improves the model's attention to key information. The specific calculation formula is:

[0051] X c =concate(X1,X2,...,X n ) (1)

[0052] Att=softmax((X c W1+B1)W2+B2) (2)

[0053] V=Att T X c (3)

[0054] Where X1, X2, ..., X n is the input from different CNN-GRU neural networks, V is the attention output, W1, W2, B1, B2 are learnable parameters, concate is the vector concatenation operation function, and softmax is the normalized exponential function.

[0055] 4. Introduce the MLP module. The MLP module is a common multi-layer perceptron responsible for outputting the final prediction results.

[0056] Federated learning is a machine learning method in which multiple clients (such as embedded devices, sensors, or other devices) collaborate to train a model without directly sharing their local data. This setting is critical for privacy protection and data security because it avoids the centralized storage of sensitive data on a central server. Unlike common practices, the deep neural networks (MobileNetV2 and MobileBERT) in data preprocessing are no longer updated when the local model is updated and the model is aggregated, because the main role of these neural networks is preliminary feature extraction, and these neural networks have been trained on the original dataset to extract the features of the corresponding modality well;

[0057] The central server or coordination node performs aggregation operation on the received parameters through the FedAvg algorithm;

[0058] The FedAvg algorithm is introduced. The goal of the FedAvg algorithm is to optimize the neural network objective function in the following formula (4), which is the sum of a finite number of objective functions;

[0059]

[0060] For a machine learning problem, let f i (w) = loss(x i ,y i ; w) then it means that under the condition of given model parameter w, about the i-th sample (x i ,y i ) loss, assuming there are K clients, P k Represents the index set of the data points of the kth client, n k =|P k | is the amount of data owned by the kth client. The above objective function, i.e., formula (4), can be rewritten as:

[0061]

[0062] In order to solve the objective function (5), the FedAvg algorithm uses a method equivalent to aggregating the gradients calculated by each client, as shown in formula (6). Each client calculates its own current model parameter w based on its local data. t After the update is performed, the central server performs a weighted average of the updated parameters of each client.

[0063]

[0064] The specific algorithm flow is shown in Table 1. The present invention uses the Fed algorithm in Table 1 to aggregate local servers from different regions on the server side.

[0065] Table 1 FedAvg algorithm flow

[0066]

[0067] In step 600, the above steps are repeated until a preset stop condition is met, the iteration ends, and a global model is obtained, which is specifically:

[0068] Repeat steps 200 to 500 until a preset stop condition is met (for example, the maximum number of iterations is reached). Parameters such as learning rate and communication frequency can be adjusted according to actual needs to optimize model performance and communication efficiency.

[0069] In step 700, the global model is evaluated. If the evaluation result meets the preset requirements, the training is completed. If not, the local model training, model parameter aggregation, and global model update and distribution are repeated until a global model that meets the preset requirements is obtained. Specifically,

[0070] The global model is evaluated, including its performance on the validation set or the test set. Whether the evaluation result meets the preset requirements is determined based on the performance. If so, the training is completed. If not, the local model training, model parameter aggregation, and global model update and distribution in steps 200 to 500 are repeated until a global model that meets the preset requirements is obtained.

[0071] The present invention provides an embodiment to verify the method provided by the present invention. During the experiment, three public data sets UCI-HAR, MELD and CREMA-D were used, as well as 600,000 multimodal sensor data generated by using a sine function plus Gaussian noise fitting. A total of four data sets were used for the experiment, and the last data was recorded as HEC, and experimental results of different data sets can be obtained;

[0072] Table 2 lists the performance of the method on different data sets, and adds a comparative experiment without using the attention mechanism to fuse multimodal features; the results show that in most data sets, the method of the present invention has excellent performance, and the attention mechanism can improve the performance of multimodal fusion in federated learning for power-related data;

[0073] Table 2 Experimental results on different datasets

[0074]

[0075] In addition, the present invention provides a schematic diagram for comparing single-modal training and multi-modal training, such as Figure 5 As shown in the figure, the performance comparison of single-modal training and multi-modal fusion tested in UCI-FAR, HEC, CREMA-D and MELD datasets shows that the method proposed in the present invention can well aggregate data information from different modalities, so that the model can learn more comprehensive and in-depth information, thereby improving the performance and generalization ability of the model.

[0076] The present invention allows multiple power system participants (such as power companies, smart meter users, etc.) to jointly train machine learning models without sharing original data by sharing model parameters, which can effectively protect user privacy and commercial secrets and comply with data protection regulations;

[0077] The present invention extracts text, video, and audio features respectively through pre-trained MobileBERT, MobileNetV2 networks, and MFCC audio feature extraction methods, converts different modal features into unified structured data, solves the problem of inconsistent structure of heterogeneous power multimodal data, and maps high-dimensional data to low-dimensional space for easy subsequent processing.

[0078] The present invention integrates power multimodal data from different sources through the CNN-GRU network, and can train a more accurate and robust model, which is of great help for prediction and optimization tasks in the power system (such as load forecasting, fault detection, demand response, etc.);

[0079] The attention mechanism used in the present invention can better capture long-distance dependencies by learning the attention weights between different time steps to obtain a feature combination that is more in line with the task objectives. Compared with the self-attention mechanism in the common Transformer, the method proposed in the present invention has a lower number of parameters and can run on clients with lower computing power.

[0080] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0081] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for sharing electric power multimodal data based on federated learning, characterized in that: include: Acquire multimodal power data and perform preprocessing operations on it; Initialize the federated learning framework, determine the nodes participating in the federated learning, establish a secure communication channel, and distribute the federated learning framework to each participating node; The local model of federated learning is trained based on the power multimodal data of participating nodes after preprocessing operations; Each participating node sends the parameters of the local model obtained through training to the central server or coordination node, which aggregates the received parameters to obtain the global model parameters. The central server or coordination node distributes the global model parameters back to each participating node. Each participating node updates the local model based on the received global model parameters and conducts training for the next iteration cycle. Repeat the above steps until the preset stop condition is met, end the iteration, and obtain the global model; The global model is evaluated. If the evaluation result meets the preset requirements, the training is completed. If not, the local model training, model parameter aggregation, and global model update and distribution are repeated until a global model that meets the preset requirements is obtained.

2. The method according to claim 1, characterized in that Obtain power multimodal data and perform preprocessing operations on it, specifically: The participating nodes acquire multimodal data from different power data sources and perform cleaning, denoising and normalization operations on the acquired data.

3. The method according to claim 1, characterized in that The federated learning framework includes a feature preprocessing module, a secondary feature extraction module, an attention module and an MLP module. The feature preprocessing module is used to extract features of multimodal data based on a pretrained neural network to obtain preliminary features; the secondary feature extraction module is used to learn the temporal features and probability distribution information of the preliminary features to obtain multimodal features; the attention module is used to process multimodal features through an attention mechanism; the MLP module is a multilayer perceptron, which is used to output the final prediction results.

4. The method according to claim 3, characterized in that The feature preprocessing module is provided with three pre-trained neural networks, namely MobileNetV2, MobileBERT and MFCC, wherein MobileNetV2 is used for video feature extraction, MobileBERT is used for text feature extraction, and the MFCC is used for audio feature extraction.

5. The method according to claim 1, characterized in that The central server or coordination node performs aggregation operation on the received parameters through the FedAvg algorithm.

Images