Power image data augmentation method based on federal learning

Abstract

The invention relates to a federated learning-based power image data augmentation method. The method comprises the following steps of obtaining an original image of a substation power inspection system; performing image processing on the original image, eliminating interference and influence of noise in the original image, constructing an image data set, and performing training on the generative adversarial network; based on the trained generative adversarial network model weight of the transformer substation, uploading to a central server for aggregation training, and issuing a final training result to the transformer substation; and the generator generates new image data by using the weight information issued by the central server. Through the generative adversarial network method based on federated learning, on the premise of protecting data privacy and security of each transformer substation, data augmentation is performed on power image data, image annotation is not required for the synthesized image, manpower and material resource cost is saved, meanwhile, the synthesis method is scientific and reasonable, augmentation result quality is high, and the method is suitable for large-scale popularization and application. Efficiency is high, the scheme is safe, and feasibility is high.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for augmenting power image data based on federated learning. Background Art

[0002] As an important infrastructure for people's livelihood and national defense construction, the power grid plays a decisive role in the transmission and distribution of electric energy. In recent years, with the continuous expansion of the scale of my country's power grid, the area covered by transmission lines and substations has become larger and larger. Therefore, it is particularly important to ensure the safe operation of the power grid. Helicopters, drones and intelligent robots are gradually becoming popular in power inspections due to their high outdoor operation efficiency, low operation risk and flexible operation. They have brought a large amount of inspection images and video data to the power system, which effectively ensures the safety of the power grid compared to traditional inspection methods. However, due to the current level of intelligence, front-line staff can only manually analyze and identify the collected data to obtain the final inspection report. This processing method is inefficient and has high labor costs, and can no longer meet actual needs.

[0003] With the development and advancement of artificial intelligence technology, the research on image recognition methods using deep learning in power fault diagnosis is in full swing. Unlike traditional machine learning, deep learning requires a large number of positive samples for training, so that the machine can learn enough features to achieve "learning from one example and applying it to other cases", thereby preventing the phenomenon of overfitting that may occur when the data set is too small. However, since helicopters, drones and intelligent robots have not been promoted and applied for a long time, the image data collection volume of some fault types (such as damaged bird protection facilities, dirty insulators, etc.) is small and cannot meet the needs of deep learning. Summary of the invention

[0004] The purpose of the embodiment of the present application is to provide a method for augmenting power image data based on federated learning, which uses a generative adversarial network based on federated learning to perform data expansion, solves the problems of small data volume due to data privacy and security issues in the past, and performs massive augmentation of power image data. The generated images are of high quality and high efficiency to meet the needs of deep learning.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] The present application embodiment provides a method for augmenting power image data based on federated learning, comprising the following steps:

[0007] 1) Obtain the original image of the substation power inspection system;

[0008] 2) Perform image processing on the original image to eliminate the interference and influence of noise in the original image, build an image dataset, and train it on the generative adversarial network;

[0009] 3) Based on the weights of the generative adversarial network model trained by the substation, it is uploaded to the central server for aggregate training, and the final training results are sent to the substation;

[0010] 4) The generator uses the weight information sent by the central server to generate new image data.

[0011] Obtain the original image of the substation power inspection system, including:

[0012] 1) Obtain real image data captured in the substation within a preset time period;

[0013] 2) Evenly distribute the preset time period to each time period throughout the year to ensure that the acquired images include all industrial control protocol services as much as possible;

[0014] 3) All the acquired samples are used to create a dataset called real images in substations.

[0015] Construct an image information dataset and train it on a generative adversarial network, including:

[0016] The image processing in S2 includes image denoising, defogging, feature extraction and feature enhancement;

[0017] The image denoising includes Gaussian filtering and median filtering.

[0018] The Gaussian filter eliminates Gaussian noise in the image data by performing weighted averaging on the image data, including:

[0019] Scan each pixel in the image with a template;

[0020] The weighted average grayscale value of the pixels in the neighborhood determined by the template is used to replace the value of the central pixel of the template; the median filtering includes:

[0021] Replace the value of a point in the image data with the median value of each point in a neighborhood of the point;

[0022] Make the surrounding pixel values ​​close to the true value and eliminate isolated impulse noise.

[0023] The dehazing process includes histogram equalization and homomorphic filtering:

[0024] Histogram equalization adjusts the contrast of the image through the histogram to increase the global contrast of the image;

[0025] Homomorphic filtering transforms images, removes multiplicative noise, increases image contrast, and standardizes brightness.

[0026] The feature extraction includes:

[0027] The feature is constructed by using the directional gradient histogram to calculate and count the gradient direction histogram of the local area of ​​the image;

[0028] Feature extraction combined with support vector machine classifier to classify the target image;

[0029] The feature enhancements in S3 include:

[0030] The target core area is sampled multiple times by sliding the window;

[0031] Obtain preliminary enhanced target data and enhance image features.

[0032] The generative adversarial network in S4 includes a generator network and a discriminator network. Data is generated by the generator network, and the discriminator network identifies the data. The generator network is guided by the learned data features to generate data and synthesize with the background image to generate new image data. The generator network is a 7-layer convolutional neural network, and its network structure is the first convolutional layer of the generator, the first non-linear activation function of the generator, the second convolutional layer of the generator, the second non-linear activation function of the generator, the third convolutional layer of the generator, the third non-linear activation function of the generator, and the generator fully connected layer. The discriminator network is a 7-layer convolutional neural network, and its network structure is the first convolutional layer of the discriminator, the first non-linear activation function of the discriminator, the second convolutional layer of the discriminator, the second non-linear activation function of the discriminator, the third convolutional layer of the discriminator, the third non-linear activation function of the discriminator, and the discriminator fully connected layer.

[0033] Based on the model weights trained by the substation, upload them to the central server for aggregate training and send them to the substation, including:

[0034] 1) The grid edge device downloads the global model Wt-1 from the central server;

[0035] 2) The grid edge device trains the local command data to obtain the local model Wt,k, and the local model of the Kth grid edge device in the tth round of communication is updated;

[0036] 3) Each grid edge device uploads the local confidential model update to the central server;

[0037] 4) After the central server receives the encrypted weights of each grid edge device, it first performs a decryption operation and then performs a weighted aggregation operation to obtain the global model Wt. The global model of the tth round of communication is updated. The objective function of the central server is as follows:

[0038]

[0039] Where m is the total number of grid edge devices participating in the training, n is all grid edge devices, Nk is the amount of data of the Kth edge device, and Fk(w) is the local objective function of the kth device:

[0040]

[0041] Where dk is the local data set of the kth edge device, f(w) = a(x, y, w) is the loss function generated by the model with parameter W for the instance (x, y) in the data set dk. The loss function generated by all instances in dk is divided by the total data volume of the power grid edge device K, which is the average loss function of the edge device. The loss function is inversely proportional to the model accuracy. The model weight of the edge device is updated as follows:

[0042]

[0043] The model aggregation update of the communication center server in round t is as follows:

[0044]

[0045] Compared with the prior art, the beneficial effects of this application are: for the synthesized images, there is no need to annotate the images, which saves manpower and material costs. At the same time, the synthesis method is scientific and reasonable, the augmentation results are of high quality and high efficiency, the scheme is highly feasible, and data privacy is protected based on federated learning. The example of the power image data augmentation method based on the generative adversarial network defines the network structure of the generator network G and the discriminator network D in the generative adversarial network, including 7 layers including convolutional layers, non-activation functions and fully connected layers. The number of network layers is small, the amount of calculation required is low, and the generation and optimization of new image data can be effectively completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a flow chart of the method of the present invention;

[0048] Figure 2 This is a flow chart of the federated learning framework in the present invention. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0050] The terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0051] The terms "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and should not be understood as indicating or implying relative importance, nor should they be understood as requiring or implying any such actual relationship or order between these entities or operations.

[0052] like Figure 1-2 As shown, the method for augmenting power image data based on federated learning described in the present invention includes the following steps:

[0053] S1: Obtain the original image of the substation power inspection system;

[0054] S2: Process the original image to eliminate the interference and influence of noise in the original image.

[0055] Build an image dataset and train it on a generative adversarial network;

[0056] S3: Based on the weights of the generative adversarial network model trained by the substation, upload it to the central server for aggregate training, and send the final training results to the substation;

[0057] S4: The generator generates new image data using the weight information sent by the central server;

[0058] The present invention utilizes a generative adversarial network. When the amount of original image data collected is small, the original image is processed and enhanced, and then new image data that is realistic enough is generated and synthesized in the generative adversarial network. These newly generated image data are then added to the original data set to achieve the purpose of augmenting power image data and provide a massive data foundation for deep learning. The generative adversarial network method is adopted, and the augmentation results are of high quality, high efficiency, and strong feasibility. For the synthesized images, there is no need to perform image annotation, which effectively saves manpower and material costs.

[0059] Furthermore, the acquisition of the original image includes images collected by fixed cameras, drones, helicopters or intelligent robots. Compared with traditional manual collection methods, the acquisition of original images by intelligent devices such as fixed cameras, drones, helicopters or intelligent robots has high outdoor operation efficiency, low operation risk and strong operation flexibility.

[0060] Furthermore, the image processing in S2 includes image denoising, defogging, feature extraction and image cropping. By performing denoising, defogging and other processing operations on the original image, the interference and influence of noise in the original image are eliminated, a high-quality digital image is obtained, and the clarity of the original image is improved, which is conducive to accurate recognition in subsequent steps and improves the accuracy of the results.

[0061] Furthermore, the image denoising includes Gaussian filtering and median filtering, and the Gaussian filtering eliminates Gaussian noise in the image data by performing weighted averaging on the image data, including:

[0062] Use a template to scan each pixel in the image; use the weighted average grayscale value of the pixels in the neighborhood determined by the template to replace the value of the central pixel of the template.

[0063] Specifically, Gaussian filtering is the process of weighted averaging the entire image. The value of each pixel is obtained by weighted averaging the value of itself and other pixels in the neighborhood, thereby eliminating the interference and influence of Gaussian noise in the original image and improving the quality of the original image. The median filter includes:

[0064] Replace the value of a point in the image data with the median value of all points in a neighborhood of the point; make the surrounding pixel values ​​close to the true value and eliminate isolated impulse noise.

[0065] Specifically, the median filtering algorithm is relatively simple and easy to implement with hardware. While filtering out noise, it can also protect the edge of the image so that it is not blurred.

[0066] Furthermore, the dehazing includes histogram equalization and homomorphic filtering: histogram equalization adjusts the contrast of the image through a histogram to increase the global contrast of the image.

[0067] Specifically, when the contrast of useful data of the image is quite close. The histogram equalization method can make the brightness better distributed on the histogram, so it can be used to enhance the local contrast without affecting the overall contrast. Homomorphic filtering transforms the image, removes multiplicative noise, increases image contrast and standardizes brightness. Homomorphic filtering converts the original image signal through nonlinear mapping to a different domain where a linear filter can be used, and then maps it back to the original domain after the operation. Convert the original complex operation into a relatively simple operation with the same efficiency, and keep the relevant properties unchanged.

[0068] Furthermore, the generative adversarial network in S2 includes a generator network and a discriminator network. The generator network generates data, and the discriminator network identifies the data. The learned data features guide the generator network to generate data and synthesize it with the background image to generate new image data. The mutual game learning between the generator network and the discriminator network produces a fairly good output, and finally the generator network generates image data that is difficult for the discriminator network to judge true or false.

[0069] Furthermore, the generator network is a 7-layer convolutional neural network, and its network structure is convolutional layer, non-linear activation function, convolutional layer, non-linear activation function, convolutional layer, non-linear activation function, and fully connected layer in sequence; the discriminator network is a 7-layer convolutional neural network, and its network structure is convolutional layer, non-linear activation function, convolutional layer, non-linear activation function, convolutional layer, non-linear activation function, and fully connected layer in sequence. The network structures of the generator network and the discriminator network are designed to be a 7-layer convolutional neural network that alternates between convolutional layers, non-linear activation functions, and fully connected layers. The number of network layers is small, the amount of computation required is low, and the generation and optimization of new image data can be effectively completed. The specific structural hierarchy of the generator network and the discriminator network is shown in the following table:

[0070] Generator Network Structure

[0071] Number of layers structure 1 Convolution layer: convolution kernel 5*5, channel 64 2 Non-linear activation function: Relu 3 Convolution layer: convolution kernel 3*3, channel 128 4 Non-linear activation function: ReLu 5 Convolution layer: convolution kernel 3*3. Channel 256 6 Non-linear activation function: tanh 7 Fully connected layer

[0072] Discriminator network structure

[0073]

[0074] Further, S3 is based on the weights of the generative adversarial network model trained by the substation, uploaded to the central server for aggregate training, and the final training results are sent to the substation. The grid edge device downloads the global model Wt-1 from the central server, and the grid edge device trains the local instruction data to obtain the local model Wt,k (the local model update of the Kth grid edge device in the tth round of communication). Each grid edge device uploads the local confidential model update to the central server. After the central server receives the encrypted weights of each grid edge device, it first performs a decryption operation and then performs a weighted aggregation operation to obtain the global model Wt (the global model update of the tth round of communication). The objective function of the central server is as follows:

[0075]

[0076] Where m is the total number of grid edge devices participating in the training, n is all grid edge devices, Nk is the amount of data of the Kth edge device, and Fk(w) is the local objective function of the kth device:

[0077]

[0078] Where dk is the local data set of the kth edge device, and f(w) = a(x, y, w) is the loss function generated by the model with parameter W for the instance (x, y) in the data set dk. The loss function generated by all instances in dk is then divided by the total data volume of the grid edge device K to obtain the average loss function of the edge device. The loss function is inversely proportional to the model accuracy. The model weight of the edge device is updated as follows:

[0079]

[0080] The model aggregation update of the communication center server in round t is as follows:

[0081]

[0082] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for augmenting power image data based on federated learning, characterized in that: The steps include: 1) Obtain the original image of the substation power inspection system; 2) Perform image processing on the original image to eliminate the interference and influence of noise in the original image, build an image dataset, and train it on the generative adversarial network; 3) Based on the weights of the generative adversarial network model trained by the substation, it is uploaded to the central server for aggregate training, and the final training results are sent to the substation; 4) The generator uses the weight information sent by the central server to generate new image data.

2. The method for augmenting power image data based on federated learning according to claim 1, characterized in that: Obtain the original image of the substation power inspection system, including: 1) Obtain real image data captured in the substation within a preset time period; 2) Evenly distribute the preset time period to each time period throughout the year to ensure that the acquired images include all industrial control protocol services as much as possible; 3) All the acquired samples are used to create a dataset called real images in substations.

3. The method for augmenting power image data based on federated learning according to claim 1, characterized in that: Construct an image information dataset and train it on a generative adversarial network, including: The image processing in S2 includes image denoising, defogging, feature extraction and feature enhancement; The image denoising includes Gaussian filtering and median filtering.

4. The method for augmenting power image data based on federated learning according to claim 3, characterized in that: The Gaussian filter eliminates Gaussian noise in the image data by performing weighted averaging on the image data, including: Scan each pixel in the image with a template; Use the weighted average gray value of the pixels in the neighborhood determined by the template to replace the value of the central pixel of the template; The median filtering includes: Replace the value of a point in the image data with the median value of each point in a neighborhood of the point; Make the surrounding pixel values ​​close to the true value and eliminate isolated impulse noise.

5. The method for augmenting power image data based on federated learning according to claim 3, characterized in that: The dehazing process includes histogram equalization and homomorphic filtering: Histogram equalization adjusts the contrast of the image through the histogram to increase the global contrast of the image; Homomorphic filtering transforms images, removes multiplicative noise, increases image contrast, and standardizes brightness.

6. The method for augmenting power image data based on federated learning according to claim 3, characterized in that: The feature extraction includes: The feature is constructed by using the directional gradient histogram to calculate and count the gradient direction histogram of the local area of ​​the image; Feature extraction combined with support vector machine classifier to classify the target image; The feature enhancements in S3 include: The target core area is sampled multiple times by sliding the window; Obtain preliminary enhanced target data and enhance image features.

7. The method for augmenting power image data based on federated learning according to claim 1, characterized in that: The generative adversarial network in S4 includes a generator network and a discriminator network. Data is generated by the generator network, and the discriminator network identifies the data. The generator network is guided by the learned data features to generate data and synthesize with the background image to generate new image data. The generator network is a 7-layer convolutional neural network, and its network structure is the first convolutional layer of the generator, the first non-linear activation function of the generator, the second convolutional layer of the generator, the second non-linear activation function of the generator, the third convolutional layer of the generator, the third non-linear activation function of the generator, and the generator fully connected layer. The discriminator network is a 7-layer convolutional neural network, and its network structure is the first convolutional layer of the discriminator, the first non-linear activation function of the discriminator, the second convolutional layer of the discriminator, the second non-linear activation function of the discriminator, the third convolutional layer of the discriminator, the third non-linear activation function of the discriminator, and the discriminator fully connected layer.

8. The method for augmenting power image data based on federated learning according to claim 1, characterized in that: Based on the model weights trained by the substation, upload them to the central server for aggregate training and send them to the substation, including: 1) The grid edge device downloads the global model Wt-1 from the central server; 2) The grid edge device trains the local command data to obtain the local model Wt,k, and the local model of the Kth grid edge device in the tth round of communication is updated; 3) Each grid edge device uploads the local confidential model update to the central server; 4) After the central server receives the encrypted weights of each grid edge device, it first performs a decryption operation and then performs a weighted aggregation operation to obtain the global model Wt. The global model of the tth round of communication is updated. The objective function of the central server is as follows: Where m is the total number of grid edge devices participating in the training, n is all grid edge devices, Nk is the amount of data of the Kth edge device, and Fk(w) is the local objective function of the kth device: Where dk is the local data set of the kth edge device, f(w) = a(x, y, w) is the loss function generated by the model with parameter W for the instance (x, y) in the data set dk. The loss function generated by all instances in dk is divided by the total data volume of the power grid edge device K, which is the average loss function of the edge device. The loss function is inversely proportional to the model accuracy. The model weight of the edge device is updated as follows: The model aggregation update of the communication center server in round t is as follows:

Images