Electrical equipment fault image expansion method and device and computer equipment

Abstract

The invention relates to an electrical equipment fault image expansion method and device and computer equipment. The method comprises the steps that an electrical equipment fault image set is acquired, the electrical equipment fault image set serves as input, a trained image expansion model is called, an expanded target electrical equipment fault image set is obtained, the image expansion model is obtained by training a pre-constructed generative adversarial network on the basis of a historical electrical equipment fault image set, and the target electrical equipment fault image set is obtained; a generative network and a discriminant network in the pre-constructed generative adversarial network are symmetrical, a preset regularization relative loss function is used as a loss function in the training process, and the generative adversarial network is trained by adopting a zero center gradient penalty strategy. By adopting the method, the image expansion quality can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image augmentation, and particularly to a method, device, computer device, storage medium, and computer program product for augmenting electrical equipment fault images. Background Art

[0002] With the development of the power system, the types, numbers, and structures of electrical equipment have become increasingly complex. When an electrical equipment fails, it may not only cause large-scale power outages but also affect the stability of the entire power grid. Therefore, in recent years, many researchers have begun to attempt to perform fault diagnosis and prediction based on the fault images of electrical equipment.

[0003] In traditional solutions, data augmentation techniques are usually used for image augmentation. For example, some transformations are performed on the original image to create more image data. These transformations can include image flipping, rotation, distortion, affine transformation, scaling, contrast adjustment, chromaticity adjustment, etc., to increase the diversity of the images.

[0004] However, data augmentation techniques are essentially relatively simple geometric and color transformations of the original image. These transformation methods are relatively fixed and patterned, while the actual electrical equipment fault conditions are complex and variable. It is difficult for traditional solutions to simulate various complex situations that may occur in reality, that is, the quality of the augmented images is relatively low. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for augmenting electrical equipment fault images that can improve the quality of image augmentation in view of the above technical problems.

[0006] In a first aspect, the present application provides a method for augmenting electrical equipment fault images. The method includes:

[0007] Obtain a set of electrical equipment fault images;

[0008] Using the set of electrical equipment fault images as input, call a trained image augmentation model to obtain an augmented set of target electrical equipment fault images;

[0009] Wherein, the image augmentation model is trained based on a historical set of electrical equipment fault images for a pre-constructed generative adversarial network. The generative network and discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularized relative loss function is used as the loss function, and the generative adversarial network is trained using a zero-centered gradient penalty strategy.

[0010] In a second aspect, the present application also provides an apparatus for augmenting electrical equipment fault images. The apparatus includes:

[0011] A data acquisition module, configured to acquire a set of electrical equipment fault images;

[0012] An image augmentation module, configured to take the set of electrical equipment fault images as an input, call a trained image augmentation model, and obtain an augmented set of target electrical equipment fault images;

[0013] Wherein, the image augmentation model is obtained by training a pre-constructed generative adversarial network based on a set of historical electrical equipment fault images. The generative network and the discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularized relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is adopted to train the generative adversarial network.

[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the embodiment of the above electrical equipment fault image augmentation method are implemented.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the embodiment of the above electrical equipment fault image augmentation method are implemented.

[0016] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the embodiment of the above electrical equipment fault image augmentation method are implemented.

[0017] The above electrical equipment fault image augmentation method, device, computer equipment, storage medium, and computer program product are different from the fixed and patterned image augmentation methods in traditional solutions. This application performs image augmentation based on a trained image augmentation model, and the trained image augmentation model is obtained by training a preset generative adversarial network. In the generative adversarial network, the generator network is responsible for generating new images, and the discriminator network is responsible for judging whether the generated images are real. During the training process based on the historical electrical equipment fault set, the generator network and the discriminator network confront and optimize each other, making the finally generated augmented images more real and diverse, capable of simulating the complex and changeable electrical equipment fault situations in reality, and improving the image quality of the target electrical equipment fault image set after augmentation. Moreover, the generator network and the discriminator network in the pre-constructed generative adversarial network are symmetric, and this symmetric structural design helps to improve the stability of the model and the subsequent training effect. During the training process of the image augmentation model, a preset regularization relative loss function is used as the loss function, and this function is mathematically proven to have local convergence, which can improve the stability of the model training process. At the same time, a zero-centered gradient penalty strategy is also adopted during the training process to optimize the training process, which can reduce problems such as gradient disappearance and gradient explosion during the model training process, improve the convergence speed and stability of the model, and obtain an image augmentation model with better training effect. Therefore, when applying the trained image augmentation model, the image quality of the target electrical equipment fault image set after augmentation can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a diagram of the application environment of the electrical equipment fault image augmentation method in an embodiment;

[0019] Figure 2 It is a schematic flowchart of the electrical equipment fault image augmentation method in an embodiment;

[0020] Figure 3 It is a schematic flowchart of the steps for training the pre-constructed generative adversarial network in an embodiment;

[0021] Figure 4 It is a schematic flowchart of the electrical equipment fault image augmentation method in another embodiment;

[0022] Figure 5 It is a schematic flowchart of the electrical equipment fault image augmentation method in a detailed embodiment;

[0023] Figure 6 It is a schematic diagram of the model architecture of the image augmentation model in an embodiment;

[0024] Figure 7 It is a block diagram of the structure of the electrical equipment fault image augmentation device in an embodiment;

[0025] Figure 8 It is the internal structure diagram of a computer device in an embodiment. Specific embodiments

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The electrical equipment fault image augmentation method provided by the embodiments of the present application can be applied to, for example Figure 1 the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers.

[0028] Specifically, a pre-trained image augmentation model is pre-stored in the data storage system of the server 104. The image augmentation model is obtained by an operator training a pre-constructed generative adversarial network based on a historical electrical equipment fault image set. The generative network and the discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularization relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is adopted to train the generative adversarial network. When applying the pre-trained image augmentation model, it can be that the operator uploads the collected electrical equipment fault image set to the server 104 through the terminal 102. The server 104 uses the electrical equipment fault set as the input, calls the pre-trained image augmentation model, and performs image augmentation on the electrical equipment fault image set to obtain the augmented target electrical equipment fault image set.

[0029] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0030] In one embodiment, as Figure 2 shown, an electrical equipment fault image augmentation method is provided. Taking the method applied to Figure 1 the server 104 therein as an example for description, it includes the following steps:

[0031] S200, obtain an electrical equipment fault image set.

[0032] Among them, the electrical equipment failure image set includes multiple electrical equipment failure images, which record information such as the appearance and state of the electrical equipment when a failure occurs.

[0033] The acquisition method of the electrical equipment failure images is not limited. For example, image acquisition devices such as cameras can be deployed in the power system to capture the operating state of the electrical equipment in real time. When a failure of the electrical equipment is detected, the failure moment and images for a period of time afterwards of the electrical equipment are automatically recorded to obtain the electrical equipment failure images. In addition, it can also be to retrieve the images taken and saved by workers when electrical equipment failed in the past from the equipment maintenance database of the power system, or to simulate electrical equipment failure experiments and take electrical equipment failure images under different failures in the experimental environment.

[0034] Furthermore, in order to improve the image quality of the electrical equipment failure image set, the collected electrical equipment failure images can be initially screened to remove obviously blurred, incorrect or duplicate images, and operations such as contrast enhancement and denoising can also be performed on the remaining images, and finally the electrical equipment failure image set is formed.

[0035] S400: Using the electrical equipment failure image set as the input, call the trained image augmentation model to obtain the augmented target electrical equipment failure image set. Among them, the image augmentation model is obtained by training a pre-constructed generative adversarial network based on the historical electrical equipment failure image set. The generative network and the discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularization relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is used to train the generative adversarial network.

[0036] Among them, the image augmentation model is obtained by training a pre-constructed generative adversarial network based on the historical electrical equipment failure image set. The symmetry of the generative network and the discriminative network in the pre-constructed generative adversarial network means that the generative network and the adversarial network have the same structure, and there are also corresponding relationships in the data processing flow and operation generality. The generative network in the generative adversarial network is responsible for generating new electrical equipment failure images according to the input noise or other data, and the discriminative network is responsible for judging whether the input image is a real electrical equipment failure image or false data generated by the generative network. The two optimize their own performance through continuous confrontation and game.

[0037] During the training process, a preset regularized relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is adopted to train the generative adversarial network. The regularized relative loss function is a function used to measure the difference between the images generated by the generative network and the real images, and then quantifies this difference into a loss value. The loss value can be used to guide the adjustment of the parameters of the generative network and the discriminative network during the training process, so that the electrical equipment fault images finally generated by the model can be closer to the real electrical equipment fault images. The regularization characteristics of the regularized relative loss function help reduce the risk of overfitting during the model training process and improve the generalization ability of the image augmentation model. The zero-centered gradient penalty strategy is a strategy for constraining the gradient of the discriminative network during the training process of the image augmentation model. By punishing the gradient norms of the discriminative network on real silk data and generated data, the risk of abnormal gradients in the discriminative network can be reduced, and the training stability of the image augmentation model can be improved.

[0038] Exemplarily, since the trained image augmentation model is a model trained on a historical electrical equipment fault image set, it has the ability to augment the input electrical equipment fault images. The augmented target electrical equipment fault set includes not only the real electrical equipment fault images collected, but also the electrical equipment fault images generated by the image augmentation model. These newly generated electrical equipment fault images are related to the input electrical equipment fault images but have diversity. Specifically, the electrical equipment fault image set obtained in the above steps can be input into the already trained image augmentation model. The image augmentation model internally performs operations such as transformation and generation on the input electrical equipment fault images according to the patterns and features it has learned, and finally outputs a series of new electrical equipment fault images. These newly generated images and the real images together constitute the augmented target electrical equipment fault image set.

[0039] It can be understood that the trained image augmentation model has been pre-stored in the data storage system of the server and has been deployed to a suitable operating environment and is in a callable state. Moreover, the augmented target electrical equipment fault image set can be screened and intelligently evaluated to check whether there are abnormalities or unreasonableness in the electrical equipment fault images generated by the model, and the images with poor quality can be screened out or regenerated.

[0040] The above electrical equipment fault image augmentation method is different from the fixed and patterned image augmentation methods in traditional solutions. This application performs image augmentation based on a trained image augmentation model, and the trained image augmentation model is obtained by training a preset generative adversarial network. In the generative adversarial network, the generator is responsible for generating new images, and the discriminator is responsible for judging whether the generated images are real. During the training process based on the historical electrical equipment fault set, the generator and the discriminator confront each other and continuously optimize, making the finally generated augmented images more real and diverse, capable of simulating complex and variable electrical equipment fault situations in reality, and improving the image quality of the target electrical equipment fault image set after augmentation. Moreover, the generator and the discriminator in the pre-constructed generative adversarial network are symmetric, and this symmetric structural design helps to improve the stability of the model and subsequent training effects. During the process of training the image augmentation model, a preset regularized relative loss function is used as the loss function, and this function is mathematically proven to have local convergence, which can improve the stability of the model training process. At the same time, a zero-centered gradient penalty strategy is also adopted during the training process to optimize the training process, which can reduce problems such as gradient disappearance and gradient explosion during the model training process, improve the convergence speed and stability of the model, and obtain an image augmentation model with better training effects. Therefore, when applying the trained image augmentation model, the image quality of the target electrical equipment fault image set after augmentation can be further improved.

[0041] In one embodiment, as Figure 3 shown, the image augmentation model is trained based on the following steps:

[0042] S110, Obtain a historical electrical equipment fault image set.

[0043] S120, Input a random variable into the generator in the pre-constructed generative adversarial network to obtain a simulated electrical equipment fault image set.

[0044] S130, Input the simulated electrical equipment fault image set and the historical electrical equipment fault image set into the discriminator of the pre-constructed generative adversarial network to obtain a discrimination result.

[0045] S140, Based on the discrimination result and the preset regularized relative loss function, determine the loss value of the discriminator network.

[0046] S150, Based on the loss value of the discriminator network, adopt a zero-centered penalty strategy, adjust the gradient norm of the discriminator network, and update the network parameters of the generator and the discriminator.

[0047] S160, Judge whether the preset training end condition is reached.

[0048] S170. If not, return to S120 until a preset training end condition is reached, and a trained image augmentation model is obtained.

[0049] Among them, the historical electrical equipment fault image set includes multiple historical electrical equipment fault images collected in the past. These images record the feature information such as the appearance and structure of the electrical equipment in different fault states. A random variable is a set of unruly numerical sequences. In a generative adversarial network, a random variable can be used as the initial input of the generative network, prompting the generative network to generate a simulated electrical equipment fault image set based on this random information. A preset regularized relative loss function is a mathematical function used to measure the difference between the prediction result of the discriminative network and the real situation. The loss value is a scalar value calculated according to the loss function, reflecting the deviation degree between the current prediction result of the discriminative network and the ideal result.

[0050] Exemplarily, random variables can be randomly generated by a random number generator. It can be understood that the format, dimension, and distribution of these random variables should meet the requirements of the pre-constructed generative adversarial network for input data. The random variables are input into the generative network of the pre-constructed generative adversarial network. The generative network internally includes convolutional layers, activation functions, etc., and can perform a series of processes such as feature extraction on the random variables. Finally, the generative network generates a series of simulated electrical equipment fault images, constituting a simulated electrical equipment fault image set. Then, the generated simulated electrical equipment fault image set and the historical electrical equipment fault image set are input into the discriminative network for authenticity discrimination. The discriminative network receives the images in the simulated electrical equipment fault image set and the historical electrical equipment fault image set. For each input image, the discriminative network can output the probability that the image belongs to a real image to obtain a discrimination result. The gradient norm is an index to measure the magnitude of the gradient. During the training process of a neural network, the gradient indicates the direction in which the loss function decreases fastest. By adjusting the gradient norm, the step size and direction of network parameter update can be controlled, reducing the situation of gradient disappearance or gradient explosion.

[0051] For the real images in the historical electrical equipment fault image set, the ideal output of the discriminative network should be close to judging as real images; for the images in the simulated electrical equipment fault image set, the ideal output should be judging as simulated images. The regularized relative loss function calculates the loss value by comparing the actual output of the discriminative network with these ideal outputs. For example, the discriminative network outputs the probability that the input image is a real image as the discrimination result. The regularized relative loss function can be expressed by Equation (1) as follows:

[0052] (1)

[0053] In Equation (1), represents the regularized relative loss function, represents the simulated electrical equipment fault images generated by the generation network, and respectively represent the outputs of the discriminative network when the input images are real images and simulated images, is the input of the generation network, represents the expectation of a random variable that follows the distribution and represents the expectation of the real samples that follow the real data distribution and represents the Sigmoid function, which is a mathematical function and can be expressed as Equation (2):

[0054] (2)

[0055] Based on Equation (1) and Equation (2), the loss value of the discriminative network can be calculated. Further, based on the loss value of the discriminative network, the gradient of the discriminative network can be calculated by means of the backpropagation algorithm, etc., and then the zero-centered gradient penalty strategy is used to adjust the gradient norm of the discriminative network. If the gradient norm is too large, it may cause the network parameters of the generation network and the discriminative network to be updated too violently. At this time, based on the zero-centered gradient penalty strategy, the gradient is scaled or corrected to keep it within a reasonable range, and the network parameters of the discriminative network are updated based on the adjusted gradient. For the generation network, since the adjustment target of the network parameters of the generation network is to make the generated simulated images "fool" the discriminative network as much as possible, the network parameters of the generation network can be indirectly adjusted by adjusting the network parameters of the discriminative network.

[0056] The above steps are one round of training for the pre-constructed generative adversarial network. In fact, it needs to be trained for multiple rounds to obtain the trained image augmentation network. After each round of training ends, the server will determine whether the preset training end condition is met. The preset training end condition can be reaching the preset number of training rounds, the loss value converging within a certain range, the quality index of the generated images reaching the preset standard, etc. If the preset training end condition is not met, the pre-constructed generative adversarial network will continue to be iteratively trained. The generation network generates new simulated electrical equipment fault images again, and the discriminative network discriminates between the simulated images and the real images again, calculates the loss value, adjusts the gradient norm, updates the parameters, etc., and repeats this iteration until the preset training end condition is met to obtain the trained image augmentation model.

[0057] In this embodiment, a pre-constructed generative adversarial network is used to generate a set of simulated electrical equipment fault images with random variables as inputs. Due to the randomness of the random variables and the learning ability of the generative network, the generated simulated images are diverse in features and appearance, capable of simulating various different fault scenarios and features. Moreover, by combining the regularized relative loss function and the zero-centered penalty strategy, it is possible to effectively reduce situations such as gradient vanishing and gradient explosion during the training process of the generative adversarial network, which helps the parameters of the generative network and the discriminative network to converge smoothly during training, improves the stability and efficiency of model training, obtains a better image augmentation model, and enhances the image generation efficiency and image generation accuracy.

[0058] In one embodiment, the generative network and the discriminative network each include a transition layer and a residual block. The transition layer is used to adjust the size of the input electrical equipment fault image and the feature map channels, and the residual block is used to extract features from the input electrical equipment fault image.

[0059] Among them, the transition layer plays a connecting role in the neural network architecture and is used to adjust the size of the input electrical equipment fault image and the feature map channels. Adjusting the image size can make the image adapt to the processing requirements of different network layers, and adjusting the feature map channels can regulate the quantity and representation ability of feature information in the neural network. The residual block is a neural network structural unit that can directly add the input to the output after operations such as convolution, which can alleviate the problems of gradient vanishing or gradient explosion that occur during the training of deep neural networks.

[0060] Exemplarily, in the image augmentation network, the generative network and the discriminative network adopt a completely symmetric design, each containing 25 million parameters, and the architecture of the image augmentation network is extremely simple. At each resolution stage, there is a transition layer and two residual blocks. The transition layer consists of bilinear resampling and an optional 1*1 convolution, which is used to change the spatial size and feature map channels of the image. Among them, bilinear sampling is an image scaling technique that can smoothly change the size of the image, and the 1*1 convolution can flexibly adjust the number of feature map channels. Through these two operations, preprocessing of the input data in the spatial and channel dimensions is achieved, enabling the data to meet the processing requirements of subsequent network layers.

[0061] The residual block is used to perform five operation steps on the input data, which are convolution operation with a 1*1 convolution kernel, calling the activation function, convolution operation with a 3*3 convolution kernel, calling the activation function again, and finally convolution operation with a 1*1 convolution kernel. The activation function can be Leaky ReLU. The convolution operations in the residual block do not include bias terms, which can reduce the situation of gradient disappearance and can better extract image features. Considering the characteristics of the 1*1 convolution operation, without changing the spatial resolution of the image, it can efficiently adjust the number of channels of the feature map and enhance the feature expression ability. The 3*3 convolution operation in the residual block can compress the number of channels of the feature map to one-fourth of the original, which can retain key features while compressing the number of channels of the feature map and can also reduce the subsequent calculation amount.

[0062] It should be noted that at the 4*4 resolution stage, the transition layer in the image augmentation model can be replaced by a specific module, the transition layer in the generation network can be replaced by the base layer, and the transition layer in the discriminant network can be replaced by the classification head. The input of the base layer of the generation network is a random variable, and the random variable undergoes a linear transformation through the linear layer, affecting the generation of the 4*4 feature map. Thus, the generation network can generate specific base features according to the random variable, providing a data basis for generating higher-resolution images subsequently. The classification head of the discriminant network can perform a global 4*4 depth convolution operation on the input image, thereby removing the spatial range information of the input image and integrating the features in the spatial dimension, and then processing through the linear layer to finally generate the discriminant result.

[0063] In this embodiment, the transition layer can flexibly adjust the size and the number of channels of the feature map of the input electrical equipment fault image, improving the adaptability of the image augmentation model to the electrical equipment fault image. The residual block can effectively alleviate the problems of gradient disappearance and gradient explosion during the neural network training process, which means that the image augmentation network can perform more detailed feature extraction on the electrical equipment fault image, capture more complex and abstract fault features, thus facilitating the generation of more realistic electrical equipment fault images and improving the generation quality of the electrical equipment images.

[0064] In one embodiment, the residual block includes a plurality of sequentially connected convolutional layers. Before S120, the method further includes: initializing the parameters of the last convolutional layer in the residual block to zero, and initializing the other convolutional layers except the last convolutional layer based on the number of the residual blocks.

[0065] Among them, before training the pre-constructed generative adversarial network, it needs to be initialized. Regarding the residual block, which includes multiple sequentially connected convolutional layers, initializing the parameters of the last convolutional layer to zero means that, without considering the bias term, the residual block will not produce a substantial change to the input feature information, that is, the input and output are basically the same. In this way, the generative adversarial network can be more stable in the initial stage of training, and there will be no large fluctuations in the output caused by the random initialization of the parameters of the last convolutional layer, which affects the training of the entire generative adversarial network.

[0066] Specifically, in the initial stage of model training, the generative adversarial network does not yet understand the distribution and features of the data. At this time, making the output of the residual block as consistent as possible with the input can enable the generative adversarial network to focus on learning how to extract and process features through the previous convolutional layers, rather than introducing too much uncertainty due to the random parameters of the last convolutional layer. As the training progresses, the parameters of the last convolutional layer will gradually learn how to further process the features extracted by the previous convolutional layers, thereby improving the training effect of the model. Further, the parameters of the convolutional layers except the last one in the residual block can be initialized to , where L represents the number of residual blocks. Initializing the convolutional layers according to the number of residual blocks in this way helps to control the parameter update amplitude during the training process of the model and reduce the variance explosion caused by improper initialization. It should be noted that when using the above method for initialization, adding too many bias terms or setting learnable multipliers should be avoided as much as possible. This is because too many bias terms may introduce unnecessary parameter adjustments, and learnable multipliers will also cause instability in the model during training, increasing the complexity and training difficulty of the model. Only by initializing in the above way can the variance explosion problem caused by the lack of normalization be effectively solved, and the stability and efficiency of model training be improved.

[0067] In this embodiment, by initializing the parameters of the last convolutional layer in the residual block to zero and initializing other convolutional layers according to the number of residual blocks, the generative adversarial network can be more stable in the initial stage of training, control the parameter update amplitude during the training process of the model, reduce the variance explosion caused by improper initialization, improve the stability and efficiency of model training, and further improve the image augmentation efficiency and stability.

[0068] In one embodiment, S150 includes: adjusting the gradient norm of the discriminative network on the real historical electrical equipment fault images and adjusting the gradient norm of the discriminative network on the simulated electrical equipment fault images generated by the generative network based on the loss value of the discriminative network.

[0069] Continuing from the above embodiments, by adjusting the gradient norm of the discriminative network, the discriminative network can better learn how to distinguish real historical electrical equipment fault images from the simulated electrical equipment fault images generated by the generative network. At the same time, it also helps to optimize the generative network to generate more realistic simulated images, thereby improving the performance of the image augmentation network.

[0070] Exemplarily, on real historical electrical equipment fault images, the gradient norm of the discriminative network can be adjusted based on Equation (3):

[0071] (3)

[0072] In Equation (3), represents the penalty on the gradient norm of the discriminative network on real historical electrical equipment fault images, represents the expectation of the data distribution that real samples (historical electrical equipment fault images in this embodiment) follow, is the gradient norm of the discriminator on x, represents the penalty factor used to control the intensity of the penalty.

[0073] On the simulated electrical equipment fault images generated by the generative network, the gradient norm of the discriminative network can be adjusted based on Equation (4):

[0074] (4)

[0075] In Equation (4), represents the penalty on the gradient norm of the discriminative network on the generated simulated electrical equipment fault images, represents the expectation of the data distribution that the generated simulated samples (simulated electrical equipment fault images in this embodiment) follow, represents the generative network, are the parameters of the generative network, represents the discriminative network on the generated simulated electrical equipment fault images and represents the penalty factor used to control the intensity of the penalty.

[0076] In this embodiment, by means of the gradient penalty norm, the situation where the gradient of the discriminative network is too large or too small can be reduced. An overly large gradient may cause the parameter update to be too drastic, resulting in unstable model training or even non-convergence. An overly small gradient will lead to an overly slow learning speed of the model. By controlling the gradient norm on the real samples and simulated samples respectively, the training process can be made more stable, and the generative adversarial network can more robustly learn the features of the real data and simulated data. Consequently, the image augmentation network obtained through training can also generate more realistic electrical equipment fault images, improving the quality of image augmentation.

[0077] In one embodiment, as Figure 4 shown, after S400, it further includes:

[0078] S510. For each target electrical equipment fault image in the set of target electrical equipment fault images, compare the target electrical equipment fault image with the set of electrical equipment fault images, determine the similarity between the target electrical equipment fault image and each electrical equipment fault image, and select the maximum similarity from multiple similarities.

[0079] S520. Screen out the target electrical equipment fault images with the maximum similarity greater than the preset similarity threshold from the set of target electrical equipment fault images, and update the augmented set of target electrical equipment fault images.

[0080] Following the above steps, for each target electrical equipment fault image in the set of target electrical equipment fault images, calculate the similarity between it and each electrical equipment fault image in the set of electrical equipment fault images. For example, using the cosine similarity algorithm, convert the target electrical equipment fault image and the electrical equipment fault image into feature vectors respectively, and obtain the similarity by calculating the cosine value between the two vectors. For each target electrical equipment fault image, select the maximum value from the multiple similarities calculated between it and all electrical equipment fault images. This maximum similarity represents the degree of similarity between the target image and the most similar image in the original image set.

[0081] Compare the calculated maximum similarity with the preset similarity threshold. If the maximum similarity is greater than the preset similarity threshold, it indicates that the target electrical equipment fault image is too similar to a certain image in the set of electrical equipment fault images and may be just a minor variant of the original image, with limited contribution to enhancing the diversity of the augmented dataset. Therefore, screen out such target electrical equipment fault images from the set of target electrical equipment fault images. After the above processing for each image in the set of target electrical equipment fault images, an updated augmented set of target electrical equipment fault images is obtained. The images retained in this set have sufficient differences from the images in the original image set, thereby improving the diversity of the dataset.

[0082] In this embodiment, by screening out target images with extremely high similarity to the original images, redundant images similar to the original images in the augmented dataset are reduced. This enables the augmented set of target electrical equipment fault images to contain more images with different features and patterns, enriches the diversity of the augmented set of target electrical equipment fault images, and improves the quality of image augmentation.

[0083] To provide a clearer description of the electrical equipment fault image augmentation method provided in this application, the following is an explanation in conjunction with Figure 5 and One detailed embodiments. The detailed embodiments include the following steps:

[0084] S501: Obtain a set of historical electrical equipment fault images, initialize a pre-constructed generative adversarial network, initialize the parameters of the last convolutional layer in the residual block to zero, and initialize other convolutional layers except the last convolutional layer based on the number of residual blocks.

[0085] S502: Input a random variable into the generator network of the pre-constructed generative adversarial network to obtain a set of simulated electrical equipment fault images.

[0086] S503: Input the set of simulated electrical equipment fault images and the set of historical electrical equipment fault images into the discriminator network of the pre-constructed generative adversarial network to obtain a discrimination result, and determine the loss value of the discriminator network based on the discrimination result and a preset regularization relative loss function.

[0087] S504: Based on the loss value of the discriminator network, adjust the gradient norm of the discriminator network on real historical electrical equipment fault images and adjust the gradient norm of the discriminator network on simulated electrical equipment fault images generated by the generator network.

[0088] S505: Determine whether a preset training end condition is reached.

[0089] S506: If not, return to S502 until the preset training end condition is reached to obtain a trained image augmentation model.

[0090] S507: Obtain a set of electrical equipment fault images, use the set of electrical equipment fault images as input, and call the trained image augmentation model to obtain an augmented set of target electrical equipment fault images.

[0091] S508: For each target electrical equipment fault image in the set of target electrical equipment fault images, compare the target electrical equipment fault image with the set of electrical equipment fault images, determine the similarity between the target electrical equipment fault image and each electrical equipment fault image, and screen out the maximum similarity from multiple similarities.

[0092] S509, screen out the target electrical equipment fault images with the maximum similarity greater than the preset similarity threshold from the target electrical equipment fault image set, and update the expanded target electrical equipment fault image set.

[0093] It should be noted that the model architecture of the trained image expansion model in this embodiment can be as shown in the appendix Figure 6 as follows.

[0094] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0095] Based on the same inventive concept, an embodiment of the present application also provides an electrical equipment fault image expansion device for implementing the electrical equipment fault image expansion method involved above. The solution provided by this device to solve the problem is similar to the solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the electrical equipment fault image expansion device provided below can refer to the limitations on the electrical equipment fault image expansion method in the above text, and will not be repeated here.

[0096] In one embodiment, as Figure 7 shown, an electrical equipment fault image expansion device 700 is provided, including: a data acquisition module 710 and an image expansion module 720, where:

[0097] The data acquisition module 710 is used to acquire the electrical equipment fault image set.

[0098] The image expansion module 720 is used to take the electrical equipment fault image set as input, call the trained image expansion model, and obtain the expanded target electrical equipment fault image set.

[0099] Among them, the image expansion model is trained based on the historical electrical equipment fault image set for a pre-constructed generative adversarial network. The generative network and the discriminant network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularization relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is used to train the generative adversarial network.

[0100] In one embodiment, the electrical equipment fault image augmentation device 700 is further configured to obtain a set of historical electrical equipment fault images, input a random variable into a generator network in a pre-constructed generative adversarial network to obtain a set of simulated electrical equipment fault images, input the set of simulated electrical equipment fault images and the set of historical electrical equipment fault images into a discriminator network of the pre-constructed generative adversarial network to obtain a discrimination result, determine a loss value of the discriminator network based on the discrimination result and a preset regularization relative loss function, adopt a zero-centered penalty strategy based on the loss value of the discriminator network, adjust the gradient norm of the discriminator network, update the network parameters of the generator network and the discriminator network, and execute again the step of inputting the random variable into the generator network in the pre-constructed generative adversarial network until a preset training end condition is reached, thereby obtaining a trained image augmentation model.

[0101] In one embodiment, the generator network and the discriminator network each include a transition layer and a residual block. The transition layer is used to adjust the size of the input electrical equipment fault image and adjust the feature map channels, and the residual block is used to extract features from the input electrical equipment fault image.

[0102] In one embodiment, the residual block includes a plurality of convolutional layers connected in sequence. The electrical equipment fault image augmentation device 700 is further configured to initialize the parameters of the last convolutional layer in the residual block to zero, and initialize the other convolutional layers except the last convolutional layer based on the number of residual blocks.

[0103] In one embodiment, the electrical equipment fault image augmentation device 700 is further configured to adjust the gradient norm of the discriminator network on real historical electrical equipment fault images and adjust the gradient norm of the discriminator network on simulated electrical equipment fault images generated by the generator network based on the loss value of the discriminator network.

[0104] In one embodiment, the set of electrical equipment fault images includes multiple electrical equipment fault images. The electrical equipment fault image augmentation device 700 is further configured to, for each target electrical equipment fault image in the target set of electrical equipment fault images, compare the target electrical equipment fault image with the set of electrical equipment fault images, determine the similarity between the target electrical equipment fault image and each electrical equipment fault image, screen out the maximum similarity from multiple similarities, remove the target electrical equipment fault image with the maximum similarity greater than a preset similarity threshold from the target set of electrical equipment fault images, and update the augmented target set of electrical equipment fault images.

[0105] Each module in the above electrical equipment fault image augmentation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0106] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 8 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the trained image augmentation model. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements an electrical equipment fault image augmentation method.

[0107] Those skilled in the art can understand that Figure 8 the structure shown in

[0108] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0109] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the embodiment of the above electrical equipment fault image augmentation method.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps in the embodiment of the above electrical equipment fault image augmentation method.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0113] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0114] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An electrical equipment fault image expansion method, characterized in that, The method includes: Obtaining a set of electrical equipment fault images; Taking the set of electrical equipment fault images as input, calling a trained image augmentation model to obtain an augmented target set of electrical equipment fault images; Among them, the image augmentation model is trained based on a historical set of electrical equipment fault images for a pre-constructed generative adversarial network. The generative network and the discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularized relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is used to train the generative adversarial network.

2. The method according to claim 1, wherein The image augmentation model is trained based on the following steps: Obtaining a historical set of electrical equipment fault images; Inputting a random variable into the generative network of the pre-constructed generative adversarial network to obtain a set of simulated electrical equipment fault images; Inputting the set of simulated electrical equipment fault images and the historical set of electrical equipment fault images into the discriminative network of the pre-constructed generative adversarial network to obtain a discrimination result; Based on the discrimination result and a preset regularized relative loss function, determining the loss value of the discriminative network; Based on the loss value of the discriminative network, adopting a zero-centered penalty strategy, adjusting the gradient norm of the discriminative network, and updating the network parameters of the generative network and the discriminative network; Returning to the step of inputting the random variable into the generative network of the pre-constructed generative adversarial network until a preset training end condition is reached to obtain a trained image augmentation model.

3. The method according to claim 2, wherein The generative network and the discriminative network respectively include a transition layer and a residual block. The transition layer is used to adjust the size of the input electrical equipment fault image and adjust the feature map channels, and the residual block is used to extract features from the input electrical equipment fault image.

4. The method according to claim 3, characterized in that, The residual block includes a plurality of convolutional layers connected in sequence. Before inputting the preset random variable into the generative network of the generative adversarial network, the method further includes: Initializing the parameters of the last convolutional layer in the residual block to zero, and initializing other convolutional layers except the last convolutional layer based on the number of the residual blocks.

5. The method according to claim 2, wherein Adjusting the gradient norm of the discriminative network based on the loss value of the discriminative network includes: Based on the loss value of the discriminative network, adjusting the gradient norm of the discriminative network on the real historical electrical equipment fault images, and adjusting the gradient norm of the discriminative network on the simulated electrical equipment fault images generated by the generative network.

6. The method according to any one of claims 1 to 5, characterized in that, The set of electrical equipment fault images includes multiple electrical equipment fault images. After obtaining the augmented target set of electrical equipment fault images, the method further includes: For each target electrical equipment fault image in the target set of electrical equipment fault images, comparing the target electrical equipment fault image with the set of electrical equipment fault images, determining the similarity between the target electrical equipment fault image and each of the electrical equipment fault images, and screening out the maximum similarity from multiple similarities; Screen out the target electrical equipment fault images with the maximum similarity greater than the preset similarity threshold from the target electrical equipment fault image set, and update the expanded target electrical equipment fault image set.

7. An electrical equipment fault image expansion device, characterized in that, The device includes: A data acquisition module, configured to acquire an electrical equipment fault image set; An image expansion module, configured to use the electrical equipment fault image set as input, call a trained image expansion model, and obtain an expanded target electrical equipment fault image set; Wherein, the image expansion model is obtained by training a pre-constructed generative adversarial network based on a historical electrical equipment fault image set. The generative network and the discriminative network in the pre-constructed generative adversarial network are symmetric. During the training process, a preset regularized relative loss function is used as the loss function, and a zero-centered gradient penalty strategy is adopted to train the generative adversarial network.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images