A Method and System for Power Distribution Network Defect Sample Enhancement and High-Precision Identification with Small Samples Based on Generative Adversarial Networks
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ELECTRIC POWER RESEARCH INSTITUTE OF STATE GRID JIBEI ELECTRIC POWER CO LTD
- Filing Date
- 2026-03-05
- Publication Date
- 2026-06-02
Smart Images
Figure CN122135104A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent operation and maintenance of power distribution networks and computer vision technology, specifically involving a method and system for enhancing power distribution network defect samples and high-precision identification of small samples based on generative adversarial networks. Background Technology
[0002] As a crucial component of the power system, the condition of distribution network equipment directly impacts power supply reliability and operational safety. Insulators, clamps, and hardware in distribution networks are constantly exposed to complex environments, susceptible to surface defects such as cracks, breakage, corrosion, and burning due to factors like corona discharge, mechanical stress, and pollution. Failure to detect and address these defects promptly can lead to power outages or even grid paralysis. Therefore, efficient and accurate defect detection of distribution network equipment is of paramount importance for ensuring the safe and stable operation of the power grid.
[0003] Traditional power distribution network defect detection mainly relies on manual inspections, which suffers from low efficiency, high subjectivity, and difficulty in covering remote areas. With the popularization of drone inspections and machine vision technology, deep learning-based target detection methods are gradually being applied to the field of power distribution network defect identification. However, the following technical challenges still exist in real-world industrial scenarios: First, there is the problem of scarce defect samples and class imbalance. Defects in distribution network equipment are low-probability events, with normal samples far outnumbering defective samples, and the distribution of different types of defects (such as cracks, damage, and burns) is extremely uneven. Existing public datasets are mostly for single devices or specific scenarios, making it difficult to cover the diverse types and complex operating conditions of distribution network equipment. Deep learning models are prone to overfitting under conditions of scarce samples, and their ability to identify a minority of defect types is severely insufficient, resulting in a high false negative rate.
[0004] Secondly, multimodal data fusion is difficult. Single visible light images are easily interfered with in complex environments such as low light, haze, and nighttime, resulting in blurred defect features. Infrared thermal images can reflect abnormal equipment heating, and point cloud data can provide three-dimensional geometric information, but existing methods lack effective fusion mechanisms for multimodal data, making it difficult to fully utilize the complementary advantages of different modalities for defect identification.
[0005] Third, it is difficult to balance model accuracy and real-time performance. Industrial sites are mostly deployed on edge computing devices, which place stringent demands on model inference speed. Existing high-precision detection models have a large number of parameters and are computationally complex, making it difficult to run in real time on resource-constrained embedded platforms; while lightweight models, although having fast inference speed, lack sufficient accuracy in locating small-sized defects and have a high false detection rate.
[0006] Fourth, the ability to adapt to new types of defects is weak. Distribution network equipment is diverse in type and defect forms, and new types of defects not covered in the training set often appear in actual operation and maintenance. Traditional detection models lack the ability to quickly adapt to new categories, requiring the collection of a large number of new samples for retraining, which is difficult to meet the needs of dynamic changes in industrial sites.
[0007] To address the aforementioned issues, there is an urgent need for a distribution network defect detection method that can effectively expand defect samples, integrate multimodal information, balance accuracy and efficiency, and has the ability to quickly adapt to new categories. Summary of the Invention
[0008] The purpose of this invention is to overcome the shortcomings of the prior art and solve the technical problems of scarce samples, difficulty in multimodal data fusion, difficulty in balancing accuracy and real-time performance, and weak adaptability to new types of defects in distribution network defect detection. This invention provides a method and system for enhancing distribution network defect samples and high-precision identification of small samples based on generative adversarial networks.
[0009] In a first aspect, embodiments of this application provide a method for enhancing distribution network defect samples and achieving high-precision identification of small samples based on generative adversarial networks, the method comprising: Visible light images, infrared thermal images, and point cloud data of power distribution network equipment are collected and defect categories are labeled to obtain a multimodal small sample dataset that provides a benchmark for real sample distribution and multimodal alignment. Using the multimodal small sample dataset, a data augmentation module based on StarGANv2 was trained, and synthetic defect samples were generated using normal visible light images as the source domain. After screening, the augmented dataset was obtained. Based on geometric prior information, the visible light image is converted into an infrared image and a point cloud contour image through a style transfer module, and corresponding semantic segmentation labels are generated simultaneously to obtain multimodal training data. A two-stage defect identification model was built. In the first stage, a lightweight detection network was used to locate and trim distribution network equipment. In the second stage, a detection network that integrates high-order relationship modeling and multi-scale features was used for defect identification. The augmented dataset and multimodal training data are organized into a meta-learning task set, and the two-stage defect identification model is trained using meta-learning. The joint loss function of fusion detection and segmentation is optimized, and finally an identification model for identifying distribution network defects is obtained.
[0010] Secondly, embodiments of this application provide a distribution network defect sample enhancement and small-sample high-precision identification system based on generative adversarial networks, applied to the method described in the first aspect, the system comprising: The data acquisition module is used to collect visible light images, infrared thermal images and point cloud data of power distribution network equipment and label the defect categories to obtain a multimodal small sample dataset that provides a benchmark for real sample distribution and multimodal alignment. The data augmentation module is used to train a StarGANv2-based data augmentation module using the multimodal small sample dataset, and to generate synthetic defect samples using the normal visible light images in the dataset as the source domain, and obtain the augmented dataset after screening. The style transfer module is used to convert visible light images into infrared images and point cloud contour images based on geometric prior information, and simultaneously generate corresponding semantic segmentation labels to obtain multimodal training data. The model building module is used to build a two-stage defect identification model. The first stage uses a lightweight detection network to locate and trim distribution network equipment, and the second stage uses a detection network that integrates high-order relationship modeling and multi-scale features for defect identification. The training and output module is used to organize the augmented dataset and multimodal training data into a meta-learning task set, perform meta-learning training on the two-stage defect identification model, optimize the joint loss function of fusion detection and segmentation, and finally obtain an identification model for identifying distribution network defects.
[0011] Thirdly, embodiments of this application provide an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is configured to implement the distribution network defect sample enhancement and small sample high-precision identification method based on generative adversarial networks as described in the first aspect when executing the instructions.
[0012] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the distribution network defect sample enhancement and small sample high-precision identification method based on generative adversarial networks as described in the first aspect.
[0013] Compared with existing technologies, this invention has the following advantages: It generates diverse synthetic defect samples through StarGANv2 multi-domain style transfer, and constructs a high-quality augmented dataset by combining multiple indicators such as FID, LPIPS, and classification confidence for joint screening, effectively alleviating the problems of defect sample scarcity and class imbalance. The style transfer module based on geometric prior information converts visible light images into infrared and point cloud images, simultaneously generating semantic segmentation labels and constructing multimodal aligned training data, fully utilizing the complementary advantages of different modal data. In the two-stage detection framework, the first stage uses a lightweight network for rapid localization and cropping, while the second stage integrates high-order relation modeling and multi-scale features for precise defect localization, maintaining high accuracy while meeting the real-time requirements of edge deployment. Through meta-learning tasks, it organizes model-independent meta-learning training, enabling the model to quickly adapt to newly emerging defect categories with only a small number of new samples, without requiring complete retraining. It introduces a hard-to-fail example replay training strategy, constructing a high-quality hard-to-fail sample library through stability screening and performing hot-start iterative training, effectively enhancing the model's ability to suppress false targets. Attached Figure Description
[0014] Figure 1 This is a schematic flowchart of a method for enhancing distribution network defect samples and identifying high-precision small-sample defects based on generative adversarial networks, provided in an embodiment of this application.
[0015] Figure 2 This is a diagram illustrating the architecture of a distribution network defect sample enhancement and small-sample high-precision identification system based on generative adversarial networks, provided in one embodiment of this application.
[0016] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.
[0018] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.
[0019] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] Example 1
[0021] Figure 1 This is a schematic flowchart illustrating a method for enhancing distribution network defect samples and achieving high-precision identification of small samples based on generative adversarial networks, provided as an embodiment of this application. Figure 1 As shown, a method for enhancing distribution network defect samples and achieving high-precision identification of small samples based on generative adversarial networks includes: S101. Collect visible light images, infrared thermal images, and point cloud data of the distribution network equipment and label the defect categories to obtain a multimodal small sample dataset for providing a benchmark for real sample distribution and multimodal alignment.
[0022] Specifically, in this embodiment, a drone equipped with a visible light camera, an infrared thermal imager, and a lidar scanning device is first used to collect multi-angle data on distribution network equipment such as insulators, clamps, and fittings on the power transmission line. The collected image data includes visible light images reflecting the texture of the equipment's appearance, infrared thermal images reflecting the temperature distribution of the equipment, and point cloud data containing three-dimensional spatial coordinate information. Subsequently, professional annotators label the collected image data with defect categories, including cracks, damage, burning, and corrosion. Through the above process, a small sample dataset containing multimodal data and exhibiting realistic sample distribution characteristics is constructed, providing a foundation for subsequent data augmentation and model training.
[0023] S102. Using the multimodal small sample dataset, train a data augmentation module based on StarGANv2, and use the normal visible light images in it as the source domain to generate synthetic defect samples. After screening, the augmented dataset is obtained.
[0024] Specifically, in this embodiment, a generative adversarial network based on StarGANv2 is trained as a data augmentation module using the multimodal small sample dataset obtained in step S101. This network defines each defect type (such as cracks, damage, and burns) as an independent visual domain, and uses style transfer technology to convert normal visible light images into synthetic samples that present specific defect visual features, while maintaining the structure of the original image.
[0025] Specifically, in this embodiment, the process of training the StarGANv2-based data augmentation module is as follows: First, each defect type (e.g., cracks, damage, burning, corrosion, etc.) is defined as an independent visual domain. The StarGANv2 network includes a generator, a mapping network, a style encoder, and multiple discriminators. The mapping network is responsible for converting random noise vectors into style codes for the target domain. These style codes encode visual features unique to the target defect domain, such as the texture direction of cracks, the edge morphology of damage, and the color changes of burning. For the same defect domain, by inputting different random noise, the mapping network can generate diverse style codes, thus achieving morphological diversity within the same defect type. The generator takes a normal visible light image and the style codes as input and fuses the content features of the source domain image with the style features of the target domain through an adaptive instance normalization layer. During the fusion process, the adaptive instance normalization layer adjusts the mean and variance of the input features, so that the generated image retains the original image structure (e.g., device outlines, edges, etc.) while presenting the visual features of the target defect domain, thereby generating synthetic defect samples that look realistic and natural.
[0026] During training, a diversity loss is introduced to constrain the differences between samples generated by different style codes. This loss function encourages the generator to produce output images with significant visual differences when given the same content image but different style codes, thereby avoiding pattern collapse and ensuring the morphological diversity of generated defects. Through the above training process, the StarGANv2 data augmentation module can generate synthetic defect samples with specified defect types and diverse morphologies using normal visible light images as the source domain, providing high-quality candidate samples for subsequent augmentation dataset construction.
[0027] The optimization objective of training the StarGANv2-based data augmentation module can be expressed as follows: , in, The generator for StarGANv2 is responsible for converting the input image into a target domain image. As a mapping network for StarGANv2, it converts random noise into style codes for the target domain. The input is a normal visible light image. For the generated synthetic defect samples, Label the target defect domain, such as crack, breakage, burning, etc. It is random noise used to generate diverse style codes. Style codes encode the visual features of the target defect domain. The adversarial loss for StarGANv2 makes the generated images more realistic; For the style reconstruction loss of StarGANv2, ensure that the style code encodes the correct style information; To mitigate the diversity loss of StarGANv2, we encourage the generation of diverse images using different style codes for the same content; These are the weighting coefficients of the loss function.
[0028] Generator With normal visible light images and the style code As input, an adaptive instance normalization layer fuses the content features of the source domain image with the style features of the target domain, generating a synthetic sample that retains the original image structure but presents the visual features of the target defect. .pass Constraining different style codes The generated samples are differentiated to ensure the morphological diversity of the generated defects.
[0029] To ensure the quality of the generated samples, a multi-index joint screening strategy is adopted. Specifically, in this embodiment, the process of performing multi-index joint screening on the synthetic defective samples generated by StarGANv2 is as follows: First, the generated images are objectively evaluated using the FID score. This metric measures the similarity between the generated image and the real training set in the feature space of the pre-trained network by calculating the distribution distance between the two sets of images at the feature level. For each generated synthetic defect sample, its FID contribution value to the real defect sample set is calculated. If this value exceeds a preset threshold, it indicates that the generated image deviates significantly from the distribution of the real samples, and may have structural distortion or texture abnormalities, and is therefore discarded.
[0030] Specifically, to ensure the quality of the generated samples, a multi-index joint screening strategy is used to evaluate the generated images, where the FID score is calculated using the following formula: , in, The mean of the features of the real image in the feature space of the pre-trained network. To generate the feature mean of the image in the feature space of the pre-trained network, The covariance matrix of the real image features. To generate the covariance matrix of image features, The trace of the matrix is the sum of its diagonal elements. This is the square of the Euclidean distance. This formula is used to calculate the distribution distance between generated and real images in the feature space of the pre-trained network.
[0031] Secondly, the remaining images filtered by FID are evaluated using LPIPS perceptual similarity. This metric measures the similarity of images at the human visual perception level by calculating the distance between the generated image and the real training set in the deep feature space of the pre-trained network. The lower the LPIPS value, the closer the generated image is to the real image in visual perception. Images with LPIPS perceptual similarity below a preset threshold are considered to have unnatural or distorted visual characteristics and are therefore discarded.
[0032] Specifically, the LPIPS perceptual similarity calculation formula is as follows: , in, To generate an image, For real images, This represents the total number of layers in the pre-trained network. For the pre-trained network layer, For the first The height and width of the layer feature map For the spatial location index of the feature map, The generated image and the real image are respectively in the 1st... layer The feature vector of the location, For the first The learnable perceptual weight vector of a layer, where ⊙ represents element-wise multiplication. This is the square of the L2 norm. This formula is used to calculate the perceptual feature similarity between the generated image and the real image.
[0033] Then, a pre-trained defect classification network is used to evaluate the defect category confidence of the generated images. This classification network has been trained on real defect samples and can predict the defect category of the input image and output a confidence score. For each generated synthetic defect sample, it is input into the classification network. If the network outputs a target defect category confidence score lower than a preset threshold, it indicates that the defect features of the generated image are not clear enough or there is category confusion, and it is discarded.
[0034] Finally, based on the combined results of the three evaluation metrics, only synthetic samples that simultaneously meet the criteria of similar FID distribution, high LPIPS perceptual similarity, and high defect classification confidence are retained. This multi-metric joint automatic screening process ensures that the synthetic samples ultimately included in the augmented dataset possess clear defect features, realistic visual representation, and accurate category attributes, thus constructing a high-quality augmented dataset to expand the scale of the original small sample dataset and provide high-quality data support for subsequent model training.
[0035] S103. Based on geometric prior information, the visible light image is converted into an infrared image and a point cloud contour image through the style transfer module, and the corresponding semantic segmentation label is generated simultaneously to obtain multimodal training data.
[0036] Specifically, in this embodiment, based on the geometric prior information of the distribution network equipment, a style transfer module is constructed to convert visible light images into infrared images and point cloud contour images, including the following modules and steps: S103-1, Geometric Prior Mask Image Generation Module. Specifically, in this embodiment, prior information is first extracted based on the geometric model of the distribution network equipment. The geometric model of the distribution network equipment can be obtained from the equipment's CAD design drawings or high-precision 3D scanning data, which includes the equipment's precise dimensions, shape contours, and spatial relationships between its components. Based on this geometric prior information, a mask image corresponding to each visible light image is generated. This mask image is a binarized contour image, where areas with a pixel value of 1 correspond to the body area of the distribution network equipment, and areas with a pixel value of 0 correspond to the background area. The mask image accurately delineates the edge contours of the equipment, providing spatial constraints for subsequent style transfer.
[0037] S103-2, Recurrent Generative Adversarial Network (RGAN) Construction Module. A RGAN is constructed to achieve cross-modal conversion from visible light images to infrared images and point cloud contour images. This network contains two generators and two discriminators: the first generator converts the visible light image to an infrared image, and the second generator converts the infrared image back to a visible light image; the third generator converts the visible light image to a point cloud contour image, and the fourth generator converts the point cloud contour image back to a visible light image. This bidirectional mapping structure introduces cyclic consistency constraints to ensure that the images before and after conversion maintain structural consistency. The two discriminators are used to determine whether the generated infrared image and point cloud contour image are realistic. Through adversarial training between the generator and the discriminator, the generated infrared image possesses realistic temperature distribution characteristics, and the generated point cloud contour image possesses accurate three-dimensional spatial information.
[0038] Specifically, a recurrent generative adversarial network is constructed. This network uses a bidirectional mapping process constrained by a cycle consistency loss. The formula for calculating the cycle consistency loss is as follows: , in, For the first generator from domain A to domain B, such as visible light → infrared, For a second generator that generates data from domain B to domain A, such as infrared to visible light, For image samples in domain A, such as visible light images, For image samples in the B domain, such as infrared images, Let A be the image probability distribution in domain A. The probability distribution of the image in the B domain. , For the expected value, Using the L1 norm, calculate the absolute difference at the pixel level.
[0039] S103-3, Cross-modal conversion execution module. Specifically, in this embodiment, the visible light image obtained in step S101 and the mask image generated in step S103-1 are jointly input into the recurrent generative adversarial network. The mask image serves as a spatial location guide, causing the generator to focus on the device body region and avoid interference from the background region. Under the constraint of the mask image, the first generator converts the visible light image into an infrared image, and the device body region in the generated infrared image exhibits temperature distribution characteristics similar to those in the actual infrared image; under the constraint of the mask image, the second generator converts the visible light image into a point cloud contour image, and the device body region in the generated point cloud contour image exhibits accurate three-dimensional contour information.
[0040] S103-4, Semantic Segmentation Label Synchronous Generation Module. Specifically, in this embodiment, during the cross-modal conversion process, semantic segmentation labels are synchronously generated using the intermediate layer features of the generator. The intermediate layer features of the generator contain rich spatial and semantic information. A lightweight segmentation head network is attached to these features. This segmentation head consists of several convolutional layers and performs pixel-by-pixel classification on the intermediate layer features. The segmentation head outputs a segmentation probability map of the same size as the generated image, where each pixel is classified into categories such as device body, defect region, or background. In this way, while generating infrared images or point cloud contour images, semantic segmentation labels aligned to them at the pixel level are automatically obtained without the need for additional manual annotation.
[0041] S104. Build a two-stage defect identification model. In the first stage, a lightweight detection network is used to locate and trim distribution network equipment. In the second stage, a detection network that integrates high-order relationship modeling and multi-scale features is used for defect identification.
[0042] Specifically, in this embodiment, a two-stage defect identification model is built for accurate detection of defects in distribution network equipment. The first stage uses a lightweight detection network obtained by structurally compressing the baseline detection model through a heuristic model pruning strategy. Specifically, in this embodiment, a general target detection model is first selected as the baseline model. This model has good feature extraction capabilities and detection accuracy, but it has a large number of parameters and high computational complexity, making it difficult to run directly on edge devices in real time. The baseline model typically includes a backbone network for feature extraction, a neck network for multi-scale feature fusion, and multiple detection heads of different scales for detecting targets of different sizes.
[0043] The imaging characteristics of power distribution network equipment in wide-field-of-view images are analyzed. Because the flight altitude of UAVs during inspections is relatively fixed, the imaging scale variation of power distribution network equipment in the images is small, mainly concentrated within a specific size range, lacking the multi-scale, drastic variation characteristics common in general target detection tasks. Based on this observation, the structure of the baseline model is optimized according to the task-structure matching criterion. Specifically, the feature pyramid structure and its associated multi-scale detection heads are removed from the baseline model, retaining only the detection head that best matches the actual imaging scale of the power distribution network equipment. The feature pyramid structure was originally used to detect targets of different scales, but it is a redundant structure in this scenario; removing it significantly simplifies the model structure.
[0044] Channel pruning is performed on the retained network layers to further compress the model. First, the contribution of each convolutional layer to computation is analyzed. Layers with a larger product of input and output channels dominate computation, and channel pruning of these layers yields the most significant reduction in computational cost. Based on a sensitivity-oriented channel pruning criterion, channel pruning is prioritized in high-level residual modules. High-level residual modules are responsible for extracting abstract semantic features, and appropriately reducing their channel count has a relatively small impact on the final detection accuracy. By pruning layer by layer and evaluating the change in accuracy, the final channel configuration for each layer is determined, reducing the number of model parameters to approximately three percent of the baseline model while maintaining detection accuracy.
[0045] After the aforementioned structured compression, the final lightweight detection network is obtained. This network contains only a single detection head matched to the target scale, and the number of channels in each layer of the backbone network has been optimized and pruned, resulting in a significant reduction in the overall number of parameters. This lightweight network can run in real time on edge computing devices while maintaining high-precision positioning capabilities for distribution network equipment, providing accurate trimmed areas for subsequent defect identification.
[0046] The second stage employs a detection network integrating multiple core modules to accurately locate and classify defects on the cropped images of individual devices. This network comprises four collaborative core modules: The higher-order relation modeling module enhances the model's ability to distinguish between easily confused backgrounds and true defects. This module establishes cross-regional associations between feature points by constructing a hypergraph structure, encoding global contextual information into each local feature. Specifically, the module first unfolds the input feature map into a sequence of feature nodes, each node corresponding to a position on the feature map. Then, a hypergraph structure is constructed in the feature space based on the distance relationships between nodes. Nodes with close distances are grouped into the same hyperedge, forming an association matrix between nodes and hyperedges. Through a message passing mechanism from node to hyperedge and back to node, each node can aggregate information from other nodes within its hyperedge, thereby encoding global contextual information into each local feature. This higher-order relation modeling enables the model to distinguish regions with similar local textures but different contexts, effectively reducing false detections caused by complex backgrounds.
[0047] A lightweight hybrid aggregation module is used to reduce computational complexity. This module splits the input features into two parts along the channel dimension, applies spatial convolution to only one part to extract spatial features, and retains the original information of the other part. Then, pointwise convolution is used to achieve full-channel information interaction, and residual connections are combined to maintain feature expressiveness. Specifically, in this embodiment, the lightweight hybrid aggregation module is used to maintain feature expressiveness while reducing computational complexity. This module splits the input features into two parts along the channel dimension, with one part of the channels participating in spatial convolution to extract spatial features, and the other part of the channels retaining the original information. By applying convolution to only some channels, the computational cost of spatial convolution is significantly reduced. Subsequently, the two parts of features are re-concatenated along the channel dimension, and pointwise convolution is used to achieve full-channel information interaction and fusion. Finally, residual connections are introduced to add the module input to the processed features, which ensures smooth gradient propagation and enhances feature reusability. This design significantly reduces computational overhead while maintaining the model's feature extraction capability.
[0048] The multi-branch dynamic fusion module is used to achieve adaptive multi-scale semantic convergence. This module stacks multi-scale feature maps from different network depths after channel alignment, generates dynamic fusion weights for each branch using global information, and performs weighted summation of features at different scales. Specifically, this module receives multi-scale feature maps from different network depths as input. First, it unifies the number of channels in each branch feature to the same dimension through convolutional layers. Then, it stacks the aligned multi-branch features along the channel dimension to form a feature tensor containing multi-scale information. Global average pooling is performed on the stacked features to aggregate global spatial information and generate channel description vectors. These description vectors are processed by a multilayer perceptron to output the dynamic fusion weights of each branch on different channels. After normalization, these weights are weighted and summed on the original multi-branch features to achieve adaptive selection and fusion of features at different scales, enabling the network to dynamically adjust the contribution ratio of multi-scale features according to the content of the input image.
[0049] The denormalized self-attention module maintains long-range dependency modeling capabilities within linear complexity. This module replaces traditional layer normalization with dynamic activation functions, constructing suppressive attention weights by calculating the energy of each feature point, avoiding explicit query-key matrix multiplication. This reduces computational complexity while preserving the ability to perceive global information. Specifically, the module uses dynamic activation functions to perform learnable amplitude compression and affine transformations on the input features, effectively suppressing outliers and smoothing gradients, ensuring training stability after denormalization. In attention computation, the module constructs suppressive attention weights by statistically analyzing the energy distribution of each feature point, avoiding explicit query-key matrix multiplication. Specifically, feature points with higher energy receive lower attention weights, and feature points with lower energy receive higher attention weights, thus achieving interaction between features. This design reduces the computational complexity of attention from a square relationship with the sequence length to a linear relationship, significantly reducing computation while maintaining the ability to perceive global information.
[0050] S105. The augmented dataset and multimodal training data are organized into a meta-learning task set, and the two-stage defect identification model is trained using meta-learning. The joint loss function of fusion detection and segmentation is optimized, and finally an identification model for identifying distribution network defects is obtained.
[0051] Specifically, in this embodiment, the high-quality augmented dataset obtained in step S102 and the multimodal training data obtained in step S103 are uniformly organized to construct a meta-learning task set. First, the samples in the dataset are grouped according to two dimensions: defect category and imaging modality. Defect categories include types such as cracks, damage, burning, and corrosion; imaging modalities include three types: visible light images, infrared images, and point cloud contour images. Each defect category and each imaging modality combination constitutes an independent meta-task, such as a crack-visible light task, a crack-infrared task, a damage-visible light task, etc. Through this grouping method, each meta-task focuses on the identification task of a specific defect type under a specific imaging modality, enabling the model to learn general and rapid adaptability across different tasks.
[0052] For each constructed meta-task, its samples are divided into a support set and a query set. The support set consists of a subset of samples extracted from each task, used to simulate the limited number of labeled samples available to the model when facing a new task. The support set is typically set to a few to a dozen images per class and modality. The query set consists of the remaining samples from the task, used to evaluate the model's performance after rapid adaptation on the support set. The samples in the support set and query set do not overlap, ensuring the objectivity of the evaluation results. Through this partitioning method, each meta-task simulates a few-shot learning scenario, providing task-level training samples for meta-learning training.
[0053] A cross-task meta-training approach is employed to optimize the model. In each training iteration, a batch of tasks is randomly sampled from all meta-tasks. For each sampled task, the model is first rapidly updated with gradients using its support set, simulating the model's adaptation process when facing new tasks. Then, the loss function of the adapted model is calculated using the query set of that task, and the initial parameters of the model are updated based on this loss. In this way, the model is trained on a large number of different meta-tasks, gradually learning how to quickly learn the features of new tasks from a small number of support set samples, rather than simply overfitting to specific tasks. After sufficient cross-task meta-training, the model possesses excellent rapid adaptation capabilities.
[0054] After meta-training, the model's rapid adaptability is verified. When encountering novel defects not found in the training set in practical applications, only a small number of labeled samples of that defect type need to be collected to form a support set. This support set is then input into the trained meta-learning model, and a detection model adapted to the new defect category can be obtained through a few gradient updates. The adapted model is then used to identify defects in newly acquired images, achieving rapid and accurate detection of novel defects without retraining the entire network. This capability has significant practical value for addressing the constantly emerging new types of defects in power distribution network inspections.
[0055] Meta-learning training of the two-stage defect identification model includes: Each task comprises a support set and a query set. Specifically, in this embodiment, the composition of each meta-learning task is first clarified. Each task consists of two parts: a support set and a query set. The support set contains a small number of labeled samples to simulate a scenario where the model has only a small amount of training data when facing a new task. The query set contains the remaining labeled samples in the task and is used to evaluate the model's performance after rapid adaptation on the support set. The samples in the support set and query set come from a combination of the same defect category and imaging modality, but there is no overlap between them, ensuring the objectivity and validity of the evaluation results.
[0056] A model-independent meta-learning algorithm is employed to optimize model parameters through a double-loop training process. The inner loop uses the support set to perform rapid gradient updates on the model, simulating the adaptation process to new tasks. The outer loop calculates the joint loss function for fusion detection and segmentation based on the query set and optimizes the initial parameters of the model. Specifically, in this embodiment, a model-independent meta-learning algorithm is used to train a two-stage defect recognition model. The training process consists of two loops. In the inner loop, for each sampled meta-task, the support set of that task is used to perform rapid gradient updates on the model. Specifically, samples from the support set are input into the model, the joint loss function for fusion detection and segmentation is calculated, and then the model parameters are updated with several steps of gradient descent based on this loss, resulting in temporarily adjusted model parameters for the current task. This process simulates the model's rapid adaptation capability when facing new tasks, enabling the model to quickly learn the features of new tasks from a small number of labeled samples.
[0057] In the outer loop, the model updated by the inner loop is evaluated using the query set for each task. Samples from the query set are input into the temporarily adjusted model, and the joint loss function for fusion detection and segmentation is calculated. This loss reflects the model's generalization ability to unseen samples in the same task after adaptation to a small number of samples. Then, based on the query set loss for all sampled tasks, the model's initial parameters are optimized and updated. The optimization objective is to minimize the model's loss on the query set after rapid adaptation, thereby enabling the model to learn a good initial parameter state from which it can quickly adapt to new tasks with only a small number of gradient updates.
[0058] Specifically, the model-independent meta-learning algorithm is used to optimize the model parameters through inner and outer double-layer loops. The optimization process can be represented as follows: , , in, This is the task loss function for fusing detection and segmentation. These are the initial parameters for the meta-learning of the model. For the first The task-specific parameters are updated in the loop within each task. For the inner loop learning rate, The learning rate for the outer loop. For the first The support set for each task For the first A query set for each task. The number of tasks sampled in each meta-training batch. For parameters Find the gradient.
[0059] Internal circulation utilizes support set For model parameters Perform fast gradient updates to obtain task-specific adaptation parameters. Outer loop utilizes query set Calculate the gradient of the task loss for the meta-parameters. Cumulative updates are performed to improve the model's ability to adapt quickly.
[0060] After training, when a new type of defect appears, only a small number of labeled samples are needed as a support set for a few gradient updates to obtain a detection model adapted to the new category. Specifically, in this embodiment, after the above meta-learning training, the model obtains good initial parameters. When a new type of defect not covered in the training set appears during actual distribution network inspection, only a small number of labeled samples of this new type of defect need to be collected to form a support set. These support set samples are input into the meta-learned model, and starting from the model's initial parameters, a few gradient updates are performed on the support set to obtain detection model parameters adapted to the new type of defect. Then, this quickly updated model is used to detect defects in newly acquired images, without retraining the entire network or collecting a large number of new samples, to achieve accurate identification of new defects. This feature has significant practical value for the constantly emerging new defects in distribution network inspection.
[0061] During the meta-learning training of the two-stage defect identification model, a difficult negative example replay training strategy is also introduced, which includes: A preliminarily converged detection model is used to perform offline inference on the training set and unlabeled background images to collect high-confidence false detection candidate boxes. Specifically, in this embodiment, a preliminarily converged detection model is first used to perform offline inference on the training set images and a large number of unlabeled background images. During the inference process, the model outputs a large number of detection candidate boxes and their corresponding confidence scores. Among these candidate boxes, those with low overlap with any real target boxes but high confidence scores from the model are selected. These candidate boxes represent areas where the model is currently prone to false detections, i.e., areas where the model incorrectly identifies background textures or non-target objects as distribution network equipment or defects, and are highly deceptive.
[0062] When testing the input image, enhancement operations are performed to filter out stable false detection samples whose confidence changes before and after enhancement are less than a threshold. Specifically, in this embodiment, high-confidence false detection candidate boxes collected through offline inference are further screened for stability. Enhancement operations such as horizontal flipping are performed on each original image during testing. The enhanced image is then input into the model again for inference to obtain the confidence scores of candidate boxes at the same location before and after enhancement. The difference between these two confidence scores is calculated; the smaller the difference, the more stable the false detection sample is across different viewpoints. Candidate boxes whose confidence differences before and after enhancement are below a preset threshold are selected. These samples represent a false detection pattern that the model is prone to false detection not only at a single viewpoint but also maintains a stable false detection pattern across multiple viewpoints, exhibiting higher stubbornness and forming a core component of hard-to-handle samples.
[0063] These samples, along with background texture patches far from the real targets, are used to construct a hard-to-detect sample library. Specifically, in this embodiment, the image regions corresponding to the false detection candidate boxes selected through stability screening are cropped and saved, while background regions far from any real targets are collected as background texture patches. These false detection region images and background texture patches are then organized into an independent hard-to-detect sample library. Each sample in the hard-to-detect sample library does not contain a real detection target, but has visual features that easily induce model misjudgment, such as textures similar to defects or shapes similar to device outlines.
[0064] In the subsequent warm-start training phase, the original training data is mixed with the difficult-to-fail sample library for iterative training, enhancing the model's ability to suppress false targets. Specifically, in this embodiment, during the warm-start phase of meta-learning training, the original labeled training data is mixed with the constructed difficult-to-fail sample library in a certain proportion to form a new training batch. In each training iteration, the model simultaneously receives real target samples and difficult-to-fail samples, and calculates the loss function for both. For difficult-to-fail samples, the model needs to output a prediction result without a target; any misclassification of a difficult-to-fail sample as a defect will result in a significant loss. Through this mixed training method, the model gradually learns to suppress responses to difficult-to-fail samples, enhancing its ability to distinguish false targets.
[0065] Specifically, in this embodiment, during the warm-start training process, the difficult-to-detect sample library is dynamically updated every few iterations. The model of the current iteration is used to perform offline inference on the training set and background images again, collecting new high-confidence false detection candidate boxes and replacing those old samples that can already be correctly identified under the current model. Through this dynamic update mechanism, the difficult-to-detect sample library always reflects the false detection patterns that the model currently finds most difficult to distinguish, allowing the training process to continuously focus on the model's weak areas and constantly improve the model's discrimination ability. After multiple rounds of iterative training, the model's ability to suppress false targets is significantly enhanced, effectively reducing the false detection rate in practical applications.
[0066] Example 2
[0067] like Figure 2 As shown, this application provides an architecture diagram of a distribution network defect sample enhancement and small-sample high-precision identification system based on generative adversarial networks, applied to the distribution network defect sample enhancement and small-sample high-precision identification system based on generative adversarial networks as described in Embodiment 1, including: The data acquisition module 210 is used to acquire visible light images, infrared thermal images and point cloud data of distribution network equipment and label the defect categories to obtain a multimodal small sample dataset for providing a benchmark for real sample distribution and multimodal alignment.
[0068] The data augmentation module 220 is used to train a StarGANv2-based data augmentation module using the multimodal small sample dataset, and to generate synthetic defect samples using the normal visible light images in the dataset as the source domain, and obtain the augmented dataset after screening.
[0069] The style transfer module 230 is used to convert visible light images into infrared images and point cloud contour images based on geometric prior information, and simultaneously generate corresponding semantic segmentation labels to obtain multimodal training data.
[0070] The model building module 240 is used to build a two-stage defect identification model. In the first stage, a lightweight detection network is used to locate and trim distribution network equipment. In the second stage, a detection network that integrates high-order relationship modeling and multi-scale features is used for defect identification.
[0071] The training and output module 250 is used to organize the augmented dataset and multimodal training data into a meta-learning task set, perform meta-learning training on the two-stage defect identification model, optimize the joint loss function of fusion detection and segmentation, and finally obtain an identification model for identifying distribution network defects.
[0072] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.
[0073] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.
[0074] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.
[0075] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.
[0076] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.
[0077] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.
[0078] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.
[0079] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.
[0080] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.
Claims
1. A method for enhancing distribution network defect samples and achieving high-precision identification of small samples based on generative adversarial networks, characterized in that, Includes the following steps: Visible light images, infrared thermal images, and point cloud data of power distribution network equipment are collected and defect categories are labeled to obtain a multimodal small sample dataset that provides a benchmark for real sample distribution and multimodal alignment. Using the multimodal small sample dataset, a data augmentation module based on StarGANv2 was trained, and synthetic defect samples were generated using normal visible light images as the source domain. After screening, the augmented dataset was obtained. Based on geometric prior information, the visible light image is converted into an infrared image and a point cloud contour image through a style transfer module, and corresponding semantic segmentation labels are generated simultaneously to obtain multimodal training data. A two-stage defect identification model was built. In the first stage, a lightweight detection network was used to locate and trim distribution network equipment. In the second stage, a detection network that integrates high-order relationship modeling and multi-scale features was used for defect identification. The augmented dataset and multimodal training data are organized into a meta-learning task set, and the two-stage defect identification model is trained using meta-learning. The joint loss function of fusion detection and segmentation is optimized, and finally an identification model for identifying distribution network defects is obtained.
2. The method according to claim 1, characterized in that, The process of training the StarGANv2-based data augmentation module includes: Each defect type is defined as an independent visual domain. Random noise is converted into a style code of the target domain through a mapping network. The generator takes a normal visible light image and the style code as input. The content features of the source domain image and the style features of the target domain are fused through an adaptive instance normalization layer to generate a synthetic sample that retains the original image structure but presents the visual features of the target defect. The differences between samples generated by different style codes are constrained by diversity loss to ensure the morphological diversity of the generated defects.
3. The method according to claim 1, characterized in that, The screening process includes: The generated images are objectively evaluated using FID scores. The distribution distance between the generated images and the real training set in the feature space of the pre-trained network is calculated, and outlier images that deviate from the real distribution by more than a preset threshold are removed. Calculate the LPIPS perceptual similarity between the remaining images and the real training set, and remove images whose perceptual similarity is lower than a preset threshold. A pre-trained defect classification network is used to evaluate the defect category confidence of the generated images, and images with confidence scores below a preset threshold are removed. Based on the combined results of the FID score evaluation, the LPIPS perceptual similarity evaluation, and the defect category confidence evaluation, synthetic samples with clear defect features and visual realism are retained to construct a high-quality augmented dataset.
4. The method according to claim 1, characterized in that, The process of converting visible light images into infrared images and point cloud contour images based on geometric prior information and using a style transfer module includes: Prior information is extracted from the geometric model of the distribution network equipment to generate a mask image containing precise contours; A recurrent generative adversarial network is constructed, which contains two generators and two discriminators. The network achieves cross-modal transformation from visible light images to infrared images and point cloud contour images by constraining the bidirectional mapping between the mask image domain and the real infrared image domain, and between the mask image domain and the real point cloud contour image domain through the cycle consistency loss. During the conversion process, the segmentation head is trained using the intermediate layer features of the generator, and semantic segmentation labels that are pixel-aligned with the generated image are output simultaneously.
5. The method according to claim 1, characterized in that, The lightweight detection network used in the first stage is obtained by structurally compressing the baseline detection model through a heuristic model pruning strategy. The pruning strategy specifically includes: Based on the task-structure matching criterion, and considering the relatively uniform scale of distribution network equipment in wide field-of-view images, feature pyramid structures and irrelevant multi-scale detection heads are removed, and only detection heads that match the target scale are retained. Based on the sensitivity-oriented channel pruning criterion, and taking into account the sensitivity distribution of convolutional layer computation with respect to the number of channels, channel pruning is preferentially implemented in high-level residual modules with a large product of input and output channel numbers, thereby reducing the number of model parameters while maintaining detection accuracy.
6. The method according to claim 1, characterized in that, The detection network used in the second stage, which integrates high-order relation modeling and multi-scale features, includes the following core modules that work together: The high-order relation modeling module flattens the input feature map into a sequence of nodes, constructs a hypergraph adjacency matrix in the feature space with a distance threshold, and encodes global context information into each local feature through a message passing mechanism from node to hyperedge and then to node, thereby enhancing the model's ability to distinguish between easily confused backgrounds and real defects. The lightweight hybrid aggregation module divides the input features into two parts along the channel dimension. It applies spatial convolution only to the first part to extract spatial features, while maintaining the identity mapping in the second part. Then, it achieves full-channel information interaction through pointwise convolution and combines residual connections to reduce computational complexity while maintaining feature expressiveness. The multi-branch dynamic fusion module stacks multi-scale feature maps from different depths of the network after channel alignment, aggregates global information through global average pooling, generates dynamic fusion weights for each branch through a multilayer perceptron, and performs weighted summation of multi-branch features after normalization to achieve adaptive multi-scale semantic convergence. The denormalized self-attention module uses a dynamic activation function to perform learnable amplitude compression and affine transformation on the input features instead of layer normalization. It constructs suppressive attention weights by calculating the energy of each feature point, avoiding explicit query-key matrix multiplication, and reducing computational complexity while maintaining long-range dependency modeling capabilities.
7. The method according to claim 1, characterized in that, The process of organizing the augmented dataset and multimodal training data into a meta-learning task set includes: The samples in the augmented dataset and the multimodal training data are grouped according to defect category and imaging modality. Each combination of defect category and imaging modality constitutes a meta-task. In each task, a portion of the samples are extracted as a support set for rapid model adaptation, and the remaining samples are extracted as a query set for evaluating the performance of the adapted model. Through cross-task meta-training, the model learns to quickly learn features of new categories of defects from the support set, and the performance of the adapted model is evaluated on the query set.
8. The method according to claim 7, characterized in that, Meta-learning training of the two-stage defect identification model includes: Each task contains a support set and a query set; The model-independent meta-learning algorithm is adopted to optimize the model parameters through inner and outer double loops. The inner loop uses the support set to perform fast gradient updates on the model to simulate the adaptation process to new tasks, while the outer loop calculates the joint loss function of fusion detection and segmentation based on the query set and optimizes the initial parameters of the model. After training, when a new type of defect appears, only a small number of labeled samples are needed as a support set for a small gradient update to obtain a detection model that adapts to the new category.
9. The method according to claim 1, characterized in that, During the meta-learning training of the two-stage defect identification model, a difficult negative example replay training strategy is also introduced, which includes: Using the initially converged detection model, offline inference is performed on the training set and unlabeled background images to collect high-confidence false detection candidate boxes; When testing the input image, perform enhancement operations and filter out stable false detection samples whose confidence changes before and after enhancement are less than a threshold. These samples, along with background texture patches that are far removed from the real targets, are used to construct a hard-to-handle sample library. In the subsequent hot start training phase, the original training data is mixed with the hard negative sample library and iteratively trained to enhance the model's ability to suppress false targets. The stability screening includes: For the candidate box set generated by offline inference, samples that meet the following conditions are selected to construct a hard-to-fail sample library: the overlap between the candidate box and the real target box is less than a preset threshold, the confidence of the candidate box is higher than a preset threshold, and the difference in confidence before and after the enhancement during testing is less than a preset threshold. The above selection conditions ensure that the samples included in the hard-to-fail sample library are both highly deceptive and maintain a stable false detection pattern under different perspectives.
10. A distribution network defect sample enhancement and small-sample high-precision identification system based on generative adversarial networks, applied to the method described in any one of claims 1 to 9, characterized in that, The system includes: The data acquisition module is used to collect visible light images, infrared thermal images and point cloud data of power distribution network equipment and label the defect categories to obtain a multimodal small sample dataset that provides a benchmark for real sample distribution and multimodal alignment. The data augmentation module is used to train a StarGANv2-based data augmentation module using the multimodal small sample dataset, and to generate synthetic defect samples using the normal visible light images in the dataset as the source domain, and obtain the augmented dataset after screening. The style transfer module is used to convert visible light images into infrared images and point cloud contour images based on geometric prior information, and simultaneously generate corresponding semantic segmentation labels to obtain multimodal training data. The model building module is used to build a two-stage defect identification model. The first stage uses a lightweight detection network to locate and trim distribution network equipment, and the second stage uses a detection network that integrates high-order relationship modeling and multi-scale features for defect identification. The training and output module is used to organize the augmented dataset and multimodal training data into a meta-learning task set, perform meta-learning training on the two-stage defect identification model, optimize the joint loss function of fusion detection and segmentation, and finally obtain an identification model for identifying distribution network defects.