Insulator small sample defect detection method and system, electronic equipment and medium

Abstract

The application provides an insulator small sample defect detection method and system, electronic equipment and medium, and belongs to the field of image processing. The method comprises: acquiring a to-be-detected image and a support set; based on an insulator defect detection model and the support set, detecting insulator strings and defects in the to-be-detected image to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when defects exist. The insulator defect detection model comprises an insulator string detection network and a defect detection network. The insulator string detection network is used for detecting the positions of the insulator strings in the to-be-detected image and segmenting the insulator strings in the to-be-detected image into a plurality of query sub-block feature maps. The defect detection network is used for classifying and regressing each query sub-block feature map according to the support set. The application realizes high-precision detection of insulator defects in the case of extremely few defect samples.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, system, electronic device, and medium for detecting small-sample defects in insulators based on rotating target detection. Background Technology

[0002] Transmission lines, as the link in power transmission, are a vital functional component of the power supply system, directly affecting the safe transmission and quality of electrical energy. Insulators are widely used in power systems to provide tension support for transmission lines and isolate the transmission conductors from the grounding structure, thereby preventing potential dangers during power transmission and playing a crucial role in power transmission and transformation.

[0003] Insulators are constantly exposed to the natural environment, making them susceptible to various faults such as missing caps, breakage, and deformation, which seriously threaten the reliable operation of electrical equipment. Power grids typically have complex interconnected topologies; early detection of potential insulator faults can effectively prevent major accidents. Therefore, efficient and accurate insulator condition monitoring and timely detection of potential insulator faults are essential for the long-term safe operation of the power grid.

[0004] Existing insulator defect detection methods still rely on a certain amount of defect samples (more than 50 images). However, in practical applications, the sample size for some insulator defect categories is insufficient to meet the data requirements of these detection methods. For example, in some cases, the number of samples available for training on a certain type of insulator defect may not exceed 10 images. In such situations, conventional deep learning-based target detection network training cannot converge, the limited information cannot support the learning of large deep networks, and the detection accuracy of existing small-sample networks still needs improvement. Therefore, it is of great significance to deeply explore and utilize the feature information of small samples to improve the detection accuracy of small-sample networks. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, electronic device, and medium for detecting small sample defects in insulators, which can achieve high-precision detection of insulator defects with very few defect samples.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for detecting defects in small samples of insulators, comprising:

[0008] Acquire the image to be detected and the support set; the support set includes multiple detailed images of normal insulators and multiple detailed images of defective insulators;

[0009] Based on the insulator defect detection model and the support set, the insulator strings and defects in the image to be detected are detected to determine whether there are defects in the insulator strings in the image to be detected and the location of the defects when they exist.

[0010] The insulator defect detection model includes an insulator string detection network and a defect detection network; the insulator string detection network is used to detect the position of the insulator string in the image to be detected and to segment the insulator string in the image to be detected into multiple query sub-block feature maps; the defect detection network is used to perform classification regression on each query sub-block feature map according to the support set.

[0011] The insulator string detection network is obtained by training a first target detection network in advance using a normal insulator string dataset and a small sample insulator defect dataset; the defect detection network is obtained by training a second target detection network in advance using a small sample insulator defect dataset and the support set; the normal insulator string dataset includes multiple normal insulator string images; each normal insulator string image is labeled with an insulator string region bounding box; the small sample insulator defect dataset includes multiple defective insulator images; each defective insulator image is labeled with an insulator string region bounding box and a defect region bounding box.

[0012] Optionally, based on the insulator defect detection model and the support set, the insulator strings and defects in the image to be detected are detected to determine whether there are defects in the insulator strings in the image to be detected and the location of the defects when they exist, specifically including:

[0013] The position of the insulator string in the image to be detected is detected by an insulator string detection network to obtain an insulator string feature map;

[0014] Based on a pre-defined defect sub-block ratio, the feature map of the insulator string is segmented to obtain multiple insulator sub-blocks;

[0015] Multiple insulator sub-blocks are mapped onto the feature map of the insulator string, and a rotation region of interest pooling operation is used to convert all insulator sub-blocks into feature maps of the same size, resulting in multiple corresponding query sub-block feature maps.

[0016] Based on the support set, the feature maps of each query sub-block are classified and regressed through a defect detection network to determine whether there are defects in the insulator string in the image to be detected and the location of the defects when they exist.

[0017] Optionally, the defect detection network includes an embedding module, a few-sample classification module, and a bounding box regression module;

[0018] Based on the support set, a defect detection network is used to classify and regress the feature maps of each query sub-block to determine whether there are defects in the insulator string in the image to be detected, and the location of the defects when they exist. Specifically, this includes:

[0019] The normal class feature map is determined based on the detailed map of each normal insulator; the size of the normal class feature map is the same as the size of the feature map of each query sub-block.

[0020] Defect class feature maps are determined based on the detailed images of each defective insulator; the size of the defect class feature map is the same as the size of the feature map of each query sub-block.

[0021] The embedding module extracts features from the normal class feature map, the defect class feature map, and each query sub-block feature map to obtain deep features of normal insulators, deep features of defective insulators, and deep features of multiple query sub-blocks.

[0022] For any query sub-block deep feature, the distance between the query sub-block deep feature and the normal insulator deep feature is calculated using the small sample classification module to obtain a first distance; the distance between the query sub-block deep feature and the defective insulator deep feature is calculated to obtain a second distance; based on the first distance and the second distance, the category of the query sub-block deep feature is determined; the category of the query sub-block deep feature is either a normal insulator sub-block or a defective insulator sub-block;

[0023] The bounding box regression module regresses the deep features of the query sub-block to determine the position of the deep features of the query sub-block in the image to be detected.

[0024] Optionally, a normal class feature map is determined based on the detail map of each normal insulator, specifically including:

[0025] Feature extraction is performed on the detail images of each normal insulator to obtain the normal detail feature images corresponding to each normal insulator detail image;

[0026] After summing all normal detail feature maps element by element, divide by the number of normal detail feature maps to obtain the normal average feature map;

[0027] Max pooling is performed on the normal average feature map to obtain the normal class feature map.

[0028] Optionally, a defect class feature map is determined based on the detailed map of each defective insulator, specifically including:

[0029] Feature extraction is performed on the detail images of each defective insulator to obtain the defect detail feature images corresponding to each defective insulator detail image;

[0030] After summing the elements of all the defect detail feature maps, divide by the number of defect detail feature maps to obtain the average defect feature map.

[0031] Max pooling is performed on the average feature map of the defects to obtain the feature map of the defect class.

[0032] Optionally, the training process of the insulator defect detection model includes:

[0033] For any sample image, the position of the insulator string in the sample image is detected by the first target detection network to obtain the target bounding box of the insulator string in the sample image; the sample image can be any normal insulator string image or any defective insulator image;

[0034] For any defective insulator image, the target bounding box of the insulator string corresponding to the defective insulator image is segmented into multiple query sub-block sample feature maps;

[0035] Based on the support set, determine the feature maps of normal class samples and defective class samples;

[0036] Based on the feature maps of each query sub-block sample, the feature map of the normal class sample, and the feature map of the defective class sample, the defect target box in the defective insulator image is determined by the second target detection network.

[0037] The loss function is determined based on the target bounding box of the insulator string in each sample image, the marked region bounding box of the insulator string in each sample image, the target bounding box of the defect in each defective insulator image, and the marked region bounding box of the defective insulator image.

[0038] Based on the loss function, the first target detection network and the second target detection network are iteratively trained until the loss function converges, resulting in an insulator string detection network and a defect detection network.

[0039] Optionally, the loss function is:

[0040] L=α(L cls +L reg )+β(L fsl_cls +L fsl_reg );

[0041] Where L is the loss function value, α is the first weight balancing parameter, and L cls L is the classification loss of the first object detection network. reg L is the regression loss of the first object detection network, β is the second weight balancing parameter, and L is the regression loss of the first object detection network. fsl_cls L is the classification loss for the second object detection network. fsl_reg The regression loss is used for the second target detection network.

[0042] To achieve the above objectives, the present invention also provides the following solution:

[0043] A small sample defect detection system for insulators, comprising:

[0044] An image acquisition unit is used to acquire the image to be detected and a support set; the support set includes multiple detailed images of normal insulators and multiple detailed images of defective insulators;

[0045] The defect detection unit, connected to the image acquisition unit, is used to detect the insulator string and defects in the image to be detected based on the insulator defect detection model and the support set, so as to determine whether there are defects in the insulator string in the image to be detected and the location of the defects when they exist.

[0046] The insulator defect detection model includes an insulator string detection network and a defect detection network; the insulator string detection network is used to detect the position of the insulator string in the image to be detected and to segment the insulator string in the image to be detected into multiple query sub-block feature maps; the defect detection network is used to perform classification regression on each query sub-block feature map according to the support set.

[0047] The insulator string detection network is obtained by training a first target detection network in advance using a normal insulator string dataset and a small sample insulator defect dataset; the defect detection network is obtained by training a second target detection network in advance using a small sample insulator defect dataset and the support set; the normal insulator string dataset includes multiple normal insulator string images; each normal insulator string image is labeled with an insulator string region bounding box; the small sample insulator defect dataset includes multiple defective insulator images; each defective insulator image is labeled with an insulator string region bounding box and a defect region bounding box.

[0048] To achieve the above objectives, the present invention also provides the following solution:

[0049] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to enable the electronic device to perform the above-described method for detecting small sample defects in insulators.

[0050] To achieve the above objectives, the present invention also provides the following solution:

[0051] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for detecting small sample defects in insulators.

[0052] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0053] This invention trains an insulator string detection network using sufficient normal insulator string images and a defect detection network using a small sample insulator defect dataset. The insulator string detection network accurately locates the position of the insulator string. Then, the insulator string in the image to be detected is segmented into multiple query sub-block feature maps, providing the defect detection network with the location information of insulator sub-blocks that may contain defects, thereby narrowing the search range for insulator defects. Finally, the highly accurate defect detection network determines whether each query sub-block feature map belongs to a defect, thus improving the accuracy of insulator defect detection. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart of the insulator small sample defect detection method provided by the present invention;

[0056] Figure 2 This is a schematic diagram of the structure of an insulator defect detection model;

[0057] Figure 3 A schematic diagram for distance metric calculation;

[0058] Figure 4 This is a schematic diagram of the insulator small sample defect detection system provided by the present invention.

[0059] Symbol explanation:

[0060] 1-Image acquisition unit, 2-Defect detection unit. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] The purpose of this invention is to provide a method, system, electronic device, and medium for detecting small-sample defects in insulators. The invention employs a rotating target detection method to detect insulator strings, dividing the detected insulator strings into sub-blocks. Based on the local features of the insulator sub-blocks, prototype features are constructed using a small number of defective and normal samples. The insulator defect status is determined by measuring the distance between the local features and the prototype features. Ultimately, accurate detection of insulator defects is achieved using a small number of samples.

[0063] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] Example 1

[0065] like Figure 1 As shown, this embodiment provides a method for detecting small sample defects in insulators, including:

[0066] Step 100: Obtain the image to be detected and the support set. The support set includes multiple detailed images of normal insulators and multiple detailed images of defective insulators. The support set consists of a small number of detailed images of normal and defective insulators extracted from the inspection images, used to assist in model classification.

[0067] Step 200: Based on the insulator defect detection model and the support set, detect the insulator strings and defects in the image to be detected, so as to determine whether there are defects in the insulator strings in the image to be detected and the location of the defects when they exist.

[0068] like Figure 2 As shown, the insulator defect detection model includes an insulator string detection network and a defect detection network.

[0069] The insulator string detection network is used to detect the position of insulator strings in the image to be detected and to segment the insulator strings in the image to be detected into multiple query sub-block feature maps. The insulator string detection network is obtained by pre-training a first target detection network using a normal insulator string dataset and a small sample insulator defect dataset.

[0070] The defect detection network is used to perform classification and regression on the feature maps of each query sub-block based on the support set. The defect detection network is pre-trained on a second target detection network using a small-sample insulator defect dataset and the support set. The defect detection network includes an embedding module, a small-sample classification module, and a bounding box regression module. The embedding module consists of multiple convolutional blocks, each including a convolutional layer, a batch normalization layer, an activation function layer, and a pooling layer, used to further extract deeper features.

[0071] The normal insulator string dataset includes multiple images of normal insulator strings. Each normal insulator string image is labeled with an insulator string bounding box. The small sample insulator defect dataset includes multiple images of defective insulators. Each defective insulator image is labeled with both an insulator string bounding box and a defect bounding box. The insulator string bounding boxes are used to fine-tune the insulator string detection network, while the defect bounding boxes are used to train the subsequent defect detection network. The normal insulator string dataset contains a sufficient number of insulator string image samples, while the small sample insulator defect dataset contains a small number of defect images of different categories.

[0072] In this embodiment, the roLabelImg software is used to annotate the normal insulator string dataset and the small sample insulator defect dataset. Both the insulator string region box and the defect region box include the center point coordinates (x, y), width w, height h, and rotation angle θ. They can be converted to poly format as needed, and the horizontal and vertical coordinates (x1, y1, x2, y2, x3, y3, x4, y4) of the four corner points of the region box are directly stored. In addition, different categories are assigned to each defect region box.

[0073] Further, step 200 includes:

[0074] Step 210: Detect the position of the insulator string in the image to be detected using the insulator string detection network to obtain the insulator string feature map.

[0075] In the insulator string detection network, the backbone network uses network weights pre-trained on ImageNet or other datasets as initial weights, and trains a first target detection network using sufficient normal insulator string images to detect the location information of insulator strings in the images. The insulator string feature maps generated by the backbone network are retained for subsequent region segmentation, defect detection, and classification.

[0076] Step 220: Based on the pre-set defect sub-block ratio, the feature map of the insulator string is segmented to obtain multiple insulator sub-blocks.

[0077] Since defects in insulators are always located inside the insulator string, segmenting the detected feature map of the insulator string yields a series of location information for potentially defective insulator sub-blocks. The width of the clipping frame is the same as the width of the insulator string, and a series of width and height dimensions and frame intervals are set according to the length-to-width ratio of the defective sub-blocks. The insulator string is clipped from top to bottom to obtain the coordinate information of a series of insulator sub-blocks, i.e., candidate insulator defect frames.

[0078] The detection of insulator defects in this invention is less affected by the background. The candidate boxes for insulator defects are all cropped from the insulator string. Therefore, no matter how complex the background of the insulator inspection image is, it will not appear in the candidate box and will not affect the small sample classification module's judgment of defects.

[0079] Step 230: Map multiple insulator sub-blocks onto the feature map of the insulator string, and use a rotation region of interest pooling operation to convert all insulator sub-blocks into feature maps of the same size, thereby obtaining multiple corresponding query sub-block feature maps.

[0080] Specifically, the coordinate information of multiple insulator sub-blocks is mapped onto the insulator string feature map to obtain the corresponding sub-block feature map. Since the sizes of the insulator sub-blocks are different, the sizes of the obtained sub-block feature maps are also different. Therefore, all sub-block feature maps are input into a rotational region of interest pooling layer to uniformly adjust all sub-block feature maps to the same size, thereby obtaining the query sub-block feature map.

[0081] Steps 210 to 230 narrowed the search range for insulator defects and improved the accuracy of regression branches.

[0082] Step 240: Based on the support set, perform classification regression on the feature maps of each query sub-block through the defect detection network to determine whether there are defects in the insulator string in the image to be detected and the location of the defects when they exist.

[0083] Furthermore, step 240 includes:

[0084] Step 241: Determine the normal class feature map based on the detail map of each normal insulator. The size of the normal class feature map is the same as the size of the feature map of each query sub-block.

[0085] Specifically, features are extracted from the detail images of each normal insulator to obtain the corresponding normal detail feature images. The summation of all normal detail feature images is then performed element-wise, and divided by the number of normal detail feature images to obtain the normal average feature image. Finally, a max-pooling operation is performed on the normal average feature image to obtain the normal class feature image.

[0086] Step 242: Determine the defect class feature map based on the detailed map of each defective insulator. The size of the defect class feature map is the same as the size of the feature map of each query sub-block.

[0087] Specifically, feature extraction is performed on the detail images of each defective insulator to obtain the corresponding defect detail feature map. The summation of all defect detail feature maps element-wise is then divided by the number of defect detail feature maps to obtain the average defect feature map. Max pooling is then performed on the average defect feature map to obtain the defect class feature map. Both the normal class feature map and the defect class feature map are in vector form.

[0088] In this embodiment, the detailed images of each normal insulator and each defective insulator are fed into a backbone network that shares weights with the insulator string detection network, resulting in multiple corresponding support sample feature maps (including normal detail feature maps and defect detail feature maps). The normal average feature map encompasses the characteristics of normal images. The defect average feature map encompasses the characteristics of defective images.

[0089] Step 243: The embedding module extracts features from the normal class feature map, the defect class feature map, and each query sub-block feature map to obtain the deep features of normal insulators, the deep features of defective insulators, and the deep features of multiple query sub-blocks.

[0090] Specifically, the query sub-block feature map, the normal class feature map, and the defect class feature map are fed into the embedding module, thereby embedding the three types of features into a new feature space, resulting in the deep feature Ф1 of normal insulators, the deep feature Ф2 of defective insulators, and a series of deep features U of query sub-blocks. i Among them, the deep features Ф1 of normal insulators and the deep features Ф2 of defective insulators are two types of prototype features, which provide discrimination criteria for classification in the subsequent small sample classification module.

[0091] By feeding all feature maps into the same convolutional block, we can obtain deeper feature maps. At this point, all the newly generated deep feature maps can be considered to be in a new feature space (because they have gone through the same convolutional module).

[0092] Step 244: For any query sub-block deep feature, calculate the distance between the query sub-block deep feature and the normal insulator deep feature using the small sample classification module to obtain a first distance. Calculate the distance between the query sub-block deep feature and the defective insulator deep feature to obtain a second distance. Determine the category of the query sub-block deep feature based on the first distance and the second distance. Specifically, if the first distance is greater than the second distance, the query sub-block deep feature is a normal insulator sub-block; otherwise, the query sub-block deep feature is a defective insulator sub-block.

[0093] In the few-sample classification module, such as Figure 3 As shown, the deep features Ф1 of normal insulators and Ф2 of defective insulators are prototype features. The deep features U of each query sub-block are calculated in the embedding space. i Distance metrics between two types of prototype features. The first and second distances can be Euclidean distance, cosine similarity, etc.

[0094] Furthermore, during network training, a softmax operation is performed on the first distance d1 and the second distance d2 to convert the distances into confidence scores s1 and s2, as shown in the following formula:

[0095]

[0096] Step 245: Regress the deep features of the query sub-block using the bounding box regression module to determine the position of the deep features of the query sub-block in the image to be detected.

[0097] Specifically, the deep features of the query sub-block are input into the fully connected layer to obtain the bounding box regression parameters for each insulator sub-block. These bounding box regression parameters are used to adjust the position of the query sub-block's bounding box. Since the query sub-block is a rotated bounding box, two regression methods can be used. Method 1 regresses five parameters (x, y, w, h, θ) for each bounding box, corresponding to the midpoint coordinates, width, height, and angular offset of the rotated box, respectively. Method 2 regresses eight parameters (x1, y1, x2, y2, x3, y3, x4, y4) for each bounding box, corresponding to the x and y coordinates of the four corner points of the bounding box, respectively. During bounding box regression, the fully connected layer is used to predict the changes in each parameter, i.e., (Δx, Δy, Δw, Δh, Δθ) or (Δx1, Δy1, Δx2, Δy2, Δx3, Δy3, Δx4, Δy4).

[0098] The query sub-block bounding box is the specific coordinate information of the insulator sub-block in the image to be detected. Each set of coordinate information can be used to draw a box in the image to be detected.

[0099] Step 240 utilizes the high classification accuracy of small sample networks to improve the accuracy of classification branches.

[0100] To better understand the technical solution of this invention, the training process of the insulator defect detection model includes:

[0101] (1) For any sample image, the position of the insulator string in the sample image is detected by a first target detection network to obtain the target bounding box of the insulator string in the sample image. The sample image can be any normal insulator string image or any defective insulator image.

[0102] During training, the normal insulator string dataset and the small sample insulator defect dataset are divided into their respective training and test sets. The training set is used to train the model, and the test set is used to test the model.

[0103] (2) For any defective insulator image, the target box of the insulator string corresponding to the defective insulator image is divided into multiple query sub-block sample feature maps.

[0104] (3) Determine the feature map of normal class samples and the feature map of defective class samples based on the support set.

[0105] (4) Based on the feature maps of each query sub-block sample, the feature map of the normal class sample, and the feature map of the defective class sample, the defect target box in the defective insulator image is determined by the second target detection network.

[0106] (5) Determine the loss function based on the target bounding box of the insulator string in each sample image, the marked region bounding box of the insulator string in each sample image, the target bounding box of the defect in each defective insulator image, and the marked region bounding box of the defective insulator image.

[0107] (6) Based on the loss function, the first target detection network and the second target detection network are iteratively trained until the loss function converges, so as to obtain the insulator string detection network and the defect detection network.

[0108] During the training phase, the predicted values ​​(confidence scores and regression parameters of bounding boxes) obtained from forward propagation are substituted into the loss function along with the defect region boxes to calculate the loss of the current model. Backpropagation calculates the gradient of each layer from the output to the input in the reverse direction according to the loss function, and updates the weight parameters of each layer until training ends.

[0109] This invention employs an end-to-end multi-task loss method for optimization training. The network loss function consists of two parts: the classification loss L of the first object detection network. cls and regression loss L reg And the few-sample classification loss L of the first object detection network fsl_cls and regression loss L fsl_cls The loss function is:

[0110] L=α(L cls +L reg )+β(L fsl_cls +L fsl_reg );

[0111] Where L is the loss function value, α is the first weight balancing parameter, and L cls L is the classification loss of the first object detection network. reg L is the regression loss of the first object detection network, β is the second weight balancing parameter, and L is the regression loss of the first object detection network. fsl_cls L is the classification loss for the second object detection network. fsl_reg This is the regression loss for the second object detection network. α and β are used to adjust the proportion of each loss component at different stages of training to improve training performance. The classification loss is calculated using both positive and negative samples, while the regression loss is calculated using only positive samples.

[0112] When training the insulator string detection network, β is set to a relatively small value, while α is set to a value much larger than β, thus ensuring the accuracy of the insulator string detection box position information. When the insulator string detection network reaches loss convergence, a defect detection network is added to classify and regress the target defect boxes. At this point, β is increased and α is decreased, making β greater than α, thereby increasing the classification and regression loss of the defect detection network while decreasing the classification and regression loss of the insulator string detection network. This shifts the training focus to defect detection, improving defect detection accuracy.

[0113] This invention uses a sufficient dataset of normal insulator strings to pre-train an insulator string detection network to detect the position information of insulator strings in an image. After the loss converges, the insulator string detection network is fine-tuned using a small dataset of insulator defects, and a defect detection network is trained.

[0114] During the verification or testing phase, the confidence scores of the sub-block bounding boxes are combined with the bounding box regression parameters to identify bounding boxes with defect class scores higher than normal class scores. These bounding boxes are then adjusted using the bounding box regression parameters to obtain more accurate bounding box location information. After obtaining the fine-tuned bounding boxes and their corresponding confidence scores, Rotated Non-Maximum Suppression (RotatedNMS) is performed. Finally, the detection boxes and scores are mapped onto the image to be detected, the detection results are output, and the Average Precision (AP) is calculated.

[0115] In insulator inspection, there is an abundant sample size of normal insulator strings, but a scarce sample size of insulator defects. Therefore, this invention first trains an insulator string detection network using sufficient images of normal insulator strings to accurately locate their positions. Then, the insulator strings are segmented to provide the defect detection network with location information of potentially defective sub-blocks, aiming to narrow the search range for insulator defects and improve the accuracy of the regression branch. Subsequently, a distance metric is used to calculate the similarity between the query insulator sub-block and the support set features. Leveraging the high classification accuracy of small-sample networks, the network determines whether an insulator sub-block is a defect, thereby improving the accuracy of the classification branch.

[0116] Example 2

[0117] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a small sample defect detection system for insulators is provided below.

[0118] like Figure 4 As shown, the insulator small sample defect detection system provided in this embodiment includes: an image acquisition unit 1 and a defect detection unit 2.

[0119] The image acquisition unit 1 is used to acquire the image to be detected and the support set. The support set includes multiple detailed images of normal insulators and multiple detailed images of defective insulators.

[0120] The defect detection unit 2 is connected to the image acquisition unit 1. The defect detection unit 2 is used to detect the insulator string and defects in the image to be detected based on the insulator defect detection model and the support set, so as to determine whether there are defects in the insulator string in the image to be detected and the location of the defects when there are defects.

[0121] The insulator defect detection model includes an insulator string detection network and a defect detection network. The insulator string detection network is used to detect the position of the insulator string in the image to be detected and to segment the insulator string in the image to be detected into multiple query sub-block feature maps. The defect detection network is used to perform classification regression on each query sub-block feature map based on the support set.

[0122] The insulator string detection network is trained on a first target detection network using a normal insulator string dataset and a small sample insulator defect dataset. The defect detection network is trained on a second target detection network using a small sample insulator defect dataset and the support set. The normal insulator string dataset includes multiple images of normal insulator strings, each with an insulator string region bounding box. The small sample insulator defect dataset includes multiple images of defective insulators, each with an insulator string region bounding box and a defect region bounding box.

[0123] This invention combines large-sample insulator string identification with small-sample defect detection. First, a rotating target detection algorithm is used to extract the insulator string region. The detected insulator string is then segmented into sub-blocks, and local features are constructed for each sub-block. Simultaneously, a small number of defective and normal samples are used to construct prototype features. Finally, the distance between the local features and the prototype features is used to determine the insulator defect status and achieve accurate defect detection. Therefore, this invention combines the precise localization of large samples with the classification accuracy of small samples, enabling the detection of small-sample defects that conventional large-sample networks cannot detect. Compared to other small-sample networks, it can achieve higher detection accuracy with fewer samples.

[0124] Example 3

[0125] This embodiment provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the insulator small sample defect detection method of Embodiment 1.

[0126] Alternatively, the aforementioned electronic device may be a server.

[0127] In addition, this embodiment of the invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the insulator small sample defect detection method of Embodiment 1.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0129] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting defects in small samples of insulators, characterized in that, The insulator small sample defect detection method comprises: acquiring a to-be-detected image and a support set; the support set comprises multiple normal insulator detail images and multiple defective insulator detail images; based on an insulator defect detection model and the support set, detecting insulator strings and defects in the to-be-detected image to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when defects exist; the insulator defect detection model comprises an insulator string detection network and a defect detection network; the insulator string detection network is used to detect the positions of insulator strings in the to-be-detected image and segment the insulator strings in the to-be-detected image into multiple query sub-block feature maps; the defect detection network is used to classify and regress each query sub-block feature map according to the support set; the insulator string detection network is obtained by pre-training a first target detection network using a normal insulator string data set and a small sample insulator defect data set; the defect detection network is obtained by pre-training a second target detection network using the small sample insulator defect data set and the support set; the normal insulator string data set comprises multiple normal insulator string images; each normal insulator string image is labeled with an insulator string region frame; the small sample insulator defect data set comprises multiple defective insulator images; each defective insulator image is labeled with an insulator string region frame and a defect region frame.

2. The method of claim 1, wherein the step of determining the defect of the small sample of the insulator is performed by using a machine learning algorithm. based on an insulator defect detection model and the support set, detecting insulator strings and defects in the to-be-detected image to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when defects exist, specifically comprising: detecting the positions of insulator strings in the to-be-detected image through an insulator string detection network to obtain an insulator string feature map; segmenting the insulator string feature map based on a pre-set defect sub-block proportion to obtain multiple insulator sub-blocks; mapping the multiple insulator sub-blocks to the insulator string feature map and converting all insulator sub-blocks into feature maps of the same size through a rotated region of interest pooling operation to obtain corresponding multiple query sub-block feature maps; classifying and regressing each query sub-block feature map through a defect detection network according to the support set to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when defects exist.

3. The method of claim 2, wherein the step of determining the defect is performed by using a neural network. the defect detection network comprises an embedding module, a small sample classification module and a bounding box regression module; classifying and regressing each query sub-block feature map through a defect detection network according to the support set to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when defects exist, specifically comprising: determining a normal class feature map according to each normal insulator detail image; the size of the normal class feature map is the same as that of each query sub-block feature map; determining a defect class feature map according to each defective insulator detail image; the size of the defect class feature map is the same as that of each query sub-block feature map; The embedding module is used for feature extraction on the normal class feature map, the defect class feature map and each query sub-block feature map respectively, so as to obtain normal insulator deep features, defect insulator deep features and a plurality of query sub-block deep features; For any query sub-block deep feature, the small sample classification module is used to calculate the distance between the query sub-block deep feature and the normal insulator deep feature, so as to obtain a first distance; and the distance between the query sub-block deep feature and the defect insulator deep feature is calculated, so as to obtain a second distance; and the category of the query sub-block deep feature is determined according to the first distance and the second distance; the category of the query sub-block deep feature is a normal insulator sub-block or a defect insulator sub-block; The bounding box regression module is used for regression on the query sub-block deep feature, so as to determine the position of the query sub-block deep feature in the image to be detected.

4. The method of claim 3, wherein the step of determining the defect is performed by using a neural network. The normal class feature map is determined according to each normal insulator detail map, specifically including: The feature extraction is performed on each normal insulator detail map respectively, so as to obtain a normal detail feature map corresponding to each normal insulator detail map; After element-wise summation of all normal detail feature maps, the number of normal detail feature maps is divided, so as to obtain a normal average feature map; The maximum pooling operation is performed on the normal average feature map, so as to obtain the normal class feature map.

5. The method of claim 3, wherein the step of determining the defect is performed by using a neural network. The defect class feature map is determined according to each defect insulator detail map, specifically including: The feature extraction is performed on each defect insulator detail map respectively, so as to obtain a defect detail feature map corresponding to each defect insulator detail map; After element-wise summation of all defect detail feature maps, the number of defect detail feature maps is divided, so as to obtain a defect average feature map; The maximum pooling operation is performed on the defect average feature map, so as to obtain the defect class feature map.

6. The method of claim 1, wherein, The training process of the insulator defect detection model includes: For any sample image, the first target detection network is used to detect the position of the insulator string in the sample image, so as to obtain the insulator string target box of the sample image; the sample image is any normal insulator string image or any defect insulator image; For any defect insulator image, the insulator string target box corresponding to the defect insulator image is segmented into a plurality of query sub-block sample feature maps; The normal class sample feature map and the defect class sample feature map are determined according to the support set; The second target detection network is used to determine the defect target box in the defect insulator image according to each query sub-block sample feature map, the normal class sample feature map and the defect class sample feature map; The loss function is determined according to the insulator string target box of each sample image, the insulator string region box labeled in each sample image, the defect target box in each defect insulator image and the defect region box labeled in each defect insulator image; Based on the loss function, the first target detection network and the second target detection network are iteratively trained until the loss function converges, so as to obtain the insulator string detection network and the defect detection network.

7. The method of claim 6, wherein the step of determining the presence of a defect in the small sample of the insulator further comprises: The loss function is: L = a(L cls + L reg ) + b(L fsl_cls + L fsl_reg ); Wherein, L is a loss function value, a is a first weight balance parameter, L cls is a classification loss of the first target detection network, L reg is a regression loss of the first target detection network, β is a second weight balance parameter, L fsl_cls is a classification loss of the second target detection network, L fsl_reg is a regression loss of the second target detection network.

8. An insulator mini-sample defect detection system, applied to the insulator mini-sample defect detection method of any one of claims 1 to 7, characterized in that, The insulator small sample defect detection system includes: An image acquisition unit is configured to acquire a to-be-detected image and a support set; the support set includes multiple normal insulator detail images and multiple defective insulator detail images; A defect detection unit is connected with the image acquisition unit and is configured to detect insulator strings and defects in the to-be-detected image based on an insulator defect detection model and the support set, to determine whether the insulator strings in the to-be-detected image have defects and the defect positions when the insulator strings have defects. The insulator defect detection model includes an insulator string detection network and a defect detection network; the insulator string detection network is configured to detect the positions of the insulator strings in the to-be-detected image and segment the insulator strings in the to-be-detected image into multiple query sub-block feature maps; and the defect detection network is configured to classify and regress each query sub-block feature map according to the support set. The insulator string detection network is obtained by pre-training a first target detection network using a normal insulator string data set and a small-sample insulator defect data set; the defect detection network is obtained by pre-training a second target detection network using a small-sample insulator defect data set and the support set; the normal insulator string data set includes multiple normal insulator string images; each normal insulator string image is labeled with an insulator string region frame; and the small-sample insulator defect data set includes multiple defective insulator images; each defective insulator image is labeled with an insulator string region frame and a defect region frame.

9. An electronic device, comprising: An electronic device includes a memory and a processor; the memory is configured to store a computer program; and the processor is configured to run the computer program to enable the electronic device to perform the insulator small-sample defect detection method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is executed by the processor to implement the insulator small-sample defect detection method in any one of claims 1 to 7.

Images