A method, system, equipment, and medium for countering data expansion during power distribution network inspection.

Abstract

This invention discloses a method, system, device, and medium for adversarial augmentation of distribution network inspection data. The method includes: separating target equipment and defective components under the target equipment from a real-world distribution network inspection image to obtain a real component image; performing defect simulation on the real image of the target equipment to obtain a rendered component image; performing style transfer on the real component image and the content image to obtain a rendered composite image; training a generative adversarial network (GAN) based on the rendered composite image and the real component image; generating a defect local image containing defective equipment and defective components under the defective equipment corresponding to a given defect type based on the trained GAN; and replacing the defect local image with the corresponding equipment location in a normal distribution network inspection image to obtain an augmented sample of the distribution network inspection image containing defects. This achieves rapid and high-quality augmentation of distribution network line defect samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power distribution network inspection, in particular to a power distribution network inspection data adversarial expansion method, system, device and medium. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] Although the number of inspections generated by the unmanned aerial vehicle inspection of the power distribution network is large, some defects have a low probability of occurring in actual scenarios, and if they occur, they will have a serious impact on the overhead lines of the power distribution network. Therefore, first-line field inspectors will immediately carry out defect elimination when they find emergency defects, resulting in a small number of samples in some categories and difficulty in accumulation. Sample generation technology based on computer vision has been applied in the power transmission and other industries. The sample generation methods mainly include:

[0004] (1) 3D modeling rendering technology: scanning the point cloud data of the component through the point cloud scanning device, shooting the component image, using professional software for mapping, modeling, and directly pasting to the scene of the non-defective image through the puzzle method. However, the 3D modeling rendered component image and the background image have differences, and the human eye can recognize that it is a synthetic image.

[0005] (2) Image stitching: using a puzzle software to cover and deform the image target, and manufacturing each image. However, the image stitching efficiency is low.

[0006] (3) Adversarial generation: constructing a generator and a discriminator, and generating an image through multiple iterations. However, the images generated by the adversarial generation network are not all available and need to be identified and screened by humans.

[0007] (4) Real scene simulation: artificially manufacturing partial defects and placing them in the scene for simulation, such as damaging a perfect insulator and placing it in the actual scene for image acquisition. However, real scene simulation is low in efficiency and high in cost. SUMMARY

[0008] In order to solve the above problems, the present application provides a power distribution network inspection data adversarial expansion method, system, device and medium, which is applied to the expansion of defect samples in the power distribution network unmanned aerial vehicle sample library, and realizes the rapid and high-quality expansion of power distribution network line defect samples.

[0009] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0010] In a first aspect, the present application provides a power distribution network inspection data adversarial expansion method, comprising:

[0011] After acquiring real-time images of the power distribution network inspection and separating the target equipment and the defective components under the target equipment, images of the actual components are obtained.

[0012] Obtain physical images of the target device, and after defect simulation, obtain rendered component images;

[0013] Style transfer is performed using real component images as style images and rendered component images as content images to obtain a rendered composite image of a realistic style scene;

[0014] Based on rendered synthetic images and real component images, a generative adversarial network is trained to generate defect images according to a given defect type. Then, based on the trained generative adversarial network, a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type is generated.

[0015] Obtain normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining an expanded sample of distribution network inspection images containing defects.

[0016] As an alternative implementation, after deleting the background point cloud from the physical image of the target device through mesh segmentation, the texture information is automatically pasted onto the corresponding position of the point cloud through texture mapping, thereby obtaining a clean and colored 3D model of the physical image of the target device, and performing defect simulation on the 3D model.

[0017] As an alternative implementation method, after defect simulation, the defect simulation image is augmented by modifying the distance, rotating at different angles, and adjusting the lighting position and intensity, thereby obtaining the rendered component image.

[0018] As an alternative implementation, a generative adversarial network includes a discriminator and a generator. The rendered synthetic image and the real part image are used as the real image and semantic information of the discriminator, respectively. These are used together with the generated image and defect semantic information output by the generator as the input of the discriminator. The training is completed through interactive adversarial iteration between the generator and the discriminator.

[0019] As an alternative implementation, after replacing the defective local image with the corresponding equipment location in the normal inspection image of the power distribution network, image fusion is achieved through size transformation and edge smoothing processing.

[0020] As an optional implementation, the size transformation includes: obtaining the length and width of the region corresponding to the equipment in the normal inspection image of the power distribution network; using the length and width as target values, determining whether the size of the defective local image meets the target value; if the size of the defective local image is greater than the target value and the difference exceeds the excessive threshold, then the defective local image is downsampled; if the size of the defective local image is less than the target value and the difference exceeds the excessive threshold, then the defective local image is upsampled to obtain an image to be inserted with the same size.

[0021] As an alternative implementation, the edge smoothing process includes: defining a range of N pixels inside and outside the edge of the image to be inserted, and taking the average of M pixels surrounding each pixel as the value of that pixel, so as to achieve edge smoothing.

[0022] Secondly, the present invention provides a power distribution network inspection data countermeasure expansion system, comprising:

[0023] The real image acquisition module is configured to acquire real-scene images of power distribution network inspection, and after separating the target equipment and the defective components under the target equipment, obtain real component images;

[0024] The physical image rendering module is configured to acquire physical images of the target device and, after defect simulation, obtain rendered component images.

[0025] The style transfer module is configured to perform style transfer using real component images as style images and rendered component images as content images to obtain rendered composite images of real style scenes.

[0026] The adversarial generation module is configured to train a generative adversarial network to generate defect images based on a given defect type, using rendered synthetic images and real component images. The trained generative adversarial network then generates a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type.

[0027] The expansion module is configured to acquire normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining expanded samples of distribution network inspection images containing defects.

[0028] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0029] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] This invention innovatively proposes a data augmentation method for power distribution network inspection. It designs a component-level sample augmentation method to narrow the image generation range and improve the image generation speed. Through style transfer between real component images and rendered component images, it enhances the realism of 3D rendered images of power distribution equipment. It generates highly reliable local defect images through generative adversarial networks. By fitting the local defect images with normal inspection images, it quickly generates small sample defect images, expands the number of small sample defect images, and achieves rapid generation and amplification of defect samples.

[0032] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1 The flowchart is for the distribution network inspection data countermeasure expansion method provided in Embodiment 1 of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0038] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0039] Example 1

[0040] This embodiment provides a method for countering the expansion of distribution network inspection data, such as... Figure 1 As shown, it includes:

[0041] After acquiring real-time images of the power distribution network inspection and separating the target equipment and the defective components under the target equipment, images of the actual components are obtained.

[0042] Obtain physical images of the target device, and after defect simulation, obtain rendered component images;

[0043] Style transfer is performed using real component images as style images and rendered component images as content images to obtain a rendered composite image of a realistic style scene;

[0044] Based on rendered synthetic images and real component images, a generative adversarial network is trained to generate defect images according to a given defect type. Then, based on the trained generative adversarial network, a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type is generated.

[0045] Obtain normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining an expanded sample of distribution network inspection images containing defects.

[0046] In this embodiment, the real-scene image of the power distribution network inspection is a visible light image generated by the actual scene of the drone inspection of the overhead power distribution network. After separating the target equipment and the defective parts under the target equipment through the semantic segmentation algorithm, the real component image is obtained.

[0047] The visible light images include normal images and defect images, and the real component images can be separated from both normal and defect images.

[0048] The target devices include, for example, mounting plates and surge arresters. First, the regions of the target devices such as mounting plates and surge arresters are marked by curves to obtain a training set of target device instances. Then, based on the target device instance training set, an instance segmentation model is trained using a DeepLabv3 network to achieve automated separation of target devices such as mounting plates and surge arresters.

[0049] Taking the defect of "missing pin on the mounting plate" as an example, it is easier to find the mounting plate in a 6000*8000 pixel image than to find the bolt on the mounting plate directly. Therefore, the mounting plate is located first, and then the position of the bolt is located within the range of the mounting plate.

[0050] Defect images can be subdivided into different categories of defect groups according to the equipment name. In order to achieve a balanced distribution of samples, this embodiment expands the defect samples of equipment with a small number of defects; such as defects with a small number of defects, such as missing pins on mounting plates or detached covers on surge arresters.

[0051] First, obtain physical images of target equipment such as mounting plates and surge arresters, and then perform 3D modeling through operations such as mesh segmentation and texture mapping to construct 3D models of target equipment such as mounting plates and surge arresters;

[0052] Specifically, after deleting the background point cloud from the physical image through mesh segmentation, the texture information is automatically pasted onto the corresponding position of the point cloud, thereby obtaining a clean, colored 3D model of the target equipment such as the hanging plate and lightning arrester.

[0053] The image of the object was taken from a single photograph, not from an object on a power distribution line.

[0054] Then, perform defect simulation, such as simulating defects like missing pins on the mounting plate and detachment of the surge arrester cover (which can be achieved by manually removing the mounting plate bolts and pins, etc.).

[0055] By modifying the distance, rotating at different angles, adjusting the lighting position and intensity, the two-dimensional images of defects such as missing pins on the mounting plate and detached lightning arresters and protective covers are amplified. Images of usable equipment containing defects are manually selected, thus obtaining the rendered component images.

[0056] However, the resulting image still visually differs significantly from the actual mounting plate and surge arrester in the real-world scenario. Therefore, this embodiment uses the real component image as the style image and the rendered component image as the content image for style transfer to obtain a rendered composite image of the realistic style scene.

[0057] Specifically:

[0058] (1) Considering the pixel information of the RGB and HSV dual color spaces, the pixel information of the style image Represented as: Similarly, the pixel information of the content image can be obtained.

[0059] (2) Evaluate the content similarity between style images and content images:

[0060] (3) Using the Gram matrix, calculate the relationship between different response layers, that is, remove the influence of image content while retaining low-level features, and compare the style similarity between the two.

[0061]

[0062] Style similarity is represented as:

[0063]

[0064]

[0065] (3) Total Loss Function; The style image and content image are evaluated for similarity based on "content + style," using a loss function... To conduct an evaluation.

[0066] The content image and style image are input into the VGG19 network, and the feature maps of each layer of the network are visualized (reconstructed): a white noise image is used as input x into the VGG19 network, the response of the conv4_2 layer is used to calculate the content loss with the original content image, and the responses of the five layers "conv1_1, conv2_1, conv3_1, conv4_1, conv5_1" are used to calculate the style loss with the style image, and then the results are added together to obtain the total style loss. Finally, the total loss is obtained, and the gradient descent optimization method is used to solve for the minimum value of the loss function. x is continuously updated to finally obtain a rendered and synthesized image that fits the real style scene.

[0067] In this embodiment, the process of training a generative adversarial network to generate defect images based on rendered synthetic images and real part images includes:

[0068] Generative adversarial networks consist of a discriminator and a generator. The rendered synthetic image and the real part image are used as the real image and semantic information of the discriminator, respectively. These, together with the generated image and defect semantic information output by the generator, are used as the input of the discriminator. The training is completed through interactive adversarial iteration between the generator and the discriminator.

[0069] The generator uses a CNN as its kernel, takes 100*1 random noise as input, and outputs the generated image. ReLU and BN layers are used between two convolutional layers to speed up the training process and prevent the model from overfitting. Three convolutional layers are used to deepen the network. Finally, tanh is used as the activation function to output the generated image.

[0070] The discriminator also uses a CNN as its kernel. The input is a 64*64 real image and the generated image output by the generator. The output is the probability that the generated image is real or fake. The two convolutional layers are connected by a batch normalization layer and a leaky ReLU layer. The network depth is increased by three convolutional layers. Finally, the probability that the generated image is real or fake is output through the sigmoid activation function.

[0071] Therefore, after training, a local defect image containing the defective equipment and defective components under the defective equipment is generated based on the given defect type; taking the missing pin of the mounting plate-bolt as an example, the defective equipment is the mounting plate, and the defective component under the defective equipment is the bolt.

[0072] In this embodiment, a normal inspection image of the power distribution network is acquired, and a defective local image is replaced with the corresponding equipment location in the normal inspection image of the power distribution network. By adjusting the size and edge smoothing processing, the images are fused.

[0073] Specifically:

[0074] (1) Size transformation; obtain the length L and width W of the corresponding equipment area in the normal inspection image of the power distribution network. Using L and W as target values, determine whether the size of the defect local image meets the target value. If the size of the defect local image is greater than the target value and the difference exceeds the excessive threshold, then the defect local image is downsampled. If the size of the defect local image is less than the target value and the difference exceeds the excessive threshold, then the defect local image is upsampled to obtain the same size image to be inserted.

[0075] (2) Edge smoothing: Define a range of 10 pixels inside and outside the edge of the image to be inserted, and use the average value of the 15 pixels around each pixel as the value of that pixel to achieve edge smoothing of the image.

[0076] In this embodiment, images of overhead power lines of the power distribution network are acquired by drones, and a generator that integrates semantic information is used to expand small sample data. At the same time, it integrates data augmentation methods commonly used in deep learning, such as affine transformation, noise perturbation, and image blurring, as well as image annotation modules and intelligent annotation functions, to realize the amplified small sample data and automatically generate sample libraries in formats such as VOC and COCO, which facilitates data migration and model development.

[0077] Example 2

[0078] This embodiment provides a data countermeasure expansion system for distribution network inspection, including:

[0079] The real image acquisition module is configured to acquire real-scene images of power distribution network inspection, and after separating the target equipment and the defective components under the target equipment, obtain real component images;

[0080] The physical image rendering module is configured to acquire physical images of the target device and, after defect simulation, obtain rendered component images.

[0081] The style transfer module is configured to perform style transfer using real component images as style images and rendered component images as content images to obtain rendered composite images of real style scenes.

[0082] The adversarial generation module is configured to train a generative adversarial network to generate defect images based on a given defect type, using rendered synthetic images and real component images. The trained generative adversarial network then generates a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type.

[0083] The expansion module is configured to acquire normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining expanded samples of distribution network inspection images containing defects.

[0084] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0085] In further embodiments, the following is also provided:

[0086] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0087] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0088] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0089] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0090] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0091] Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in connection with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for countering data expansion during power distribution network inspection, characterized in that, include: After acquiring real-time images of the power distribution network inspection and separating the target equipment and the defective components under the target equipment, images of the actual components are obtained. Obtain physical images of the target device, and after defect simulation, obtain rendered component images; Style transfer is performed using real component images as style images and rendered component images as content images to obtain a rendered composite image of a realistic style scene. Specifically: Considering pixel information in both RGB and HSV color spaces, we obtain pixel information for the style image and the content image, and evaluate the content similarity between the style image and the content image. Using the Gram matrix, the influence of image content is removed while preserving low-level features, and the style similarity between style images and content images is compared. The total loss function is determined by evaluating the similarity between style images and content images in terms of content and style. The content images and style images are then input into the network, and the feature maps of each layer of the network are visualized and reconstructed. The minimum value of the total loss function is then solved, and finally, a rendered and synthesized image that fits the real style scene is obtained. Based on rendered synthetic images and real component images, a generative adversarial network is trained to generate defect images according to a given defect type. Then, based on the trained generative adversarial network, a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type is generated. Obtain normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining an expanded sample of distribution network inspection images containing defects.

2. The method for countering data expansion during power distribution network inspection as described in claim 1, characterized in that, After deleting the background point cloud from the physical image of the target device through mesh segmentation, the texture information is automatically pasted onto the corresponding position of the point cloud through texture mapping, thereby obtaining a 3D model of the physical image of the target device with color, and defect simulation is performed on the 3D model.

3. The method for countering data expansion during power distribution network inspection as described in claim 1, characterized in that, After replacing the defective local image with the corresponding equipment location in the normal inspection image of the power distribution network, image fusion is achieved through size transformation and edge smoothing.

4. The method for countering data expansion during power distribution network inspection as described in claim 3, characterized in that, The size transformation includes: obtaining the length and width of the region corresponding to the equipment in the normal inspection image of the power distribution network; using the length and width as target values, determining whether the size of the defective local image meets the target value; if the size of the defective local image is greater than the target value and the difference exceeds the excessive threshold, then the defective local image is downsampled; if the size of the defective local image is less than the target value and the difference exceeds the excessive threshold, then the defective local image is upsampled to obtain an image to be inserted with the same size.

5. The method for countering data expansion during power distribution network inspection as described in claim 4, characterized in that, The edge smoothing process includes: defining a range of N pixels inside and outside the edge of the image to be inserted, and taking the average value of the M pixels surrounding each pixel as the value of that pixel, so as to achieve edge smoothing.

6. A data countermeasure expansion system for power distribution network inspection, characterized in that, include: The real image acquisition module is configured to acquire real-scene images of power distribution network inspection, and after separating the target equipment and the defective components under the target equipment, obtain real component images; The physical image rendering module is configured to acquire physical images of the target device and, after defect simulation, obtain rendered component images. The style transfer module is configured to perform style transfer using real component images as style images and rendered component images as content images, resulting in a rendered composite image of a realistic style scene. Specifically: Considering pixel information in both RGB and HSV color spaces, we obtain pixel information for the style image and the content image, and evaluate the content similarity between the style image and the content image. Using the Gram matrix, the influence of image content is removed while preserving low-level features, and the style similarity between style images and content images is compared. The total loss function is determined by evaluating the similarity between style images and content images in terms of content and style. The content images and style images are then input into the network, and the feature maps of each layer of the network are visualized and reconstructed. The minimum value of the total loss function is then solved, and finally, a rendered and synthesized image that fits the real style scene is obtained. The adversarial generation module is configured to train a generative adversarial network to generate defect images based on a given defect type, using rendered synthetic images and real component images. The trained generative adversarial network then generates a local defect image containing the defective device and the defective component under the defective device corresponding to the given defect type. The expansion module is configured to acquire normal inspection images of the distribution network, replace the defective local images with the corresponding equipment locations in the normal inspection images of the distribution network, thereby obtaining expanded samples of distribution network inspection images containing defects.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

Images