Insulator defect detection model construction method and detection method

Abstract

The invention relates to the technical field of insulator defect detection, and provides an insulator defect detection model construction method and a detection method. The method comprises the following steps: inputting an insulator image sample set into a trunk feature extraction network of a parameter to-be-adjusted model to obtain a first feature map; inputting the first feature map into a mixed domain convolution attention module to obtain a second feature map; inputting the second feature map into a meta-feature extractor module, and learning meta-features of multiple scales from the second feature map; inputting the element features into a weighting module, and matching corresponding weighting vectors for the element features of multiple scales; obtaining a third feature map according to the element features of the multiple scales and the weighted vectors; according to the third feature map, obtaining a detection result of the insulator image sample set; and according to the difference between the detection result and the real defect result, performing parameter adjustment on the to-be-adjusted parameter model to obtain an insulator defect detection model. By adopting the method, the accuracy of insulator defect detection can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of insulator defect detection, and in particular to an insulator defect detection model construction method, an insulator defect detection model construction device, an insulator defect image detection method, an insulator defect image detection device, a computer device, a computer-readable storage medium and a computer program product. Background Art

[0002] Insulators are an important part of power transmission lines. Due to long-term exposure to the external environment, insulators on power transmission lines will inevitably be affected by factors such as bird pecking, wind erosion, ice pressure, and high temperature, resulting in defects such as breakage, chipping, and fracture, which bring great safety hazards to the stable operation of power transmission lines. Therefore, defect detection of insulators has become one of the important tasks of power transmission line inspection.

[0003] Currently, insulator defect detection can be performed based on the detection model of the YOLO (You Only Look Once) target detection algorithm. However, in actual situations, the incidence of insulator defects is relatively low. At the same time, weather and environmental changes have a great impact on the accuracy of insulator detection results. The insulator defect image dataset that can be used for detection model training is seriously insufficient, resulting in overfitting of the detection model due to too few training samples, low accuracy, high power consumption of the detection model, insufficient performance, and difficulty in deployment on unmanned aerial vehicle edge computing terminals. Summary of the invention

[0004] Based on this, it is necessary to provide an insulator defect detection model construction method, an insulator defect detection model construction device, an insulator defect image detection method, an insulator defect image detection device, a computer device, a computer-readable storage medium and a computer program product to address the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for constructing an insulator defect detection model, comprising:

[0006] Inputting the insulator image sample set into the backbone feature extraction network of the model to be adjusted in parameters, obtaining a first feature map of the insulator image sample set;

[0007] Inputting the first feature map into a mixed domain convolutional attention module of a model to be adjusted, extracting features of the first feature map in a channel dimension and a spatial dimension, and obtaining a second feature map of the insulator image sample set;

[0008] Inputting the second feature map into a meta-feature extractor module of a model to be adjusted, learning meta-features of multiple scales from the second feature map; inputting the meta-features into a weighting module of a model to be adjusted, matching corresponding weighting vectors for the meta-features of the multiple scales; obtaining a third feature map of the insulator image sample set according to the meta-features of the multiple scales and the weighting vectors;

[0009] Obtaining a detection result of the insulator image sample set according to the third feature map;

[0010] According to the difference between the detection result and the actual defect result, the parameters of the model to be adjusted are adjusted to obtain an insulator defect detection model.

[0011] In one embodiment, before inputting the insulator image sample set into a backbone feature extraction network of an insulator defect detection model, the method further includes:

[0012] Obtaining an initial sample set of insulator images;

[0013] The initial sample set of the insulator image is input into the image resizing model, so that the image resizing model performs the following processing steps: the input initial sample set of the insulator image is divided into two branches, the first branch passes through the convolution layer, the leaky rectified linear unit, the standard convolution layer, the leaky rectified linear unit and the batch normalization layer in sequence, and then enters the bilinear adjuster, the output result of the bilinear adjuster is divided into two small branches, the first small branch passes through multiple residual blocks and the convolution layer, the batch normalization layer, and then is merged with the second small branch and passes through the standard convolution layer; the second branch passes through the bilinear adjuster and then is merged with the first branch to obtain the resized initial sample set of the insulator image;

[0014] The resized initial sample set of insulator images is used as the insulator image sample set.

[0015] In one embodiment, the step of obtaining an initial sample set of insulator images includes:

[0016] According to the collected first-category insulator images and the generative adversarial network, insulator pseudo-defect image samples are obtained;

[0017] Partially or incompletely annotate the collected insulator images to obtain the second type of insulator annotated image samples;

[0018] An initial sample set of insulator images is obtained according to the insulator pseudo-defect image samples and the second type of insulator labeled image samples.

[0019] In one embodiment, the mixed domain convolutional attention module includes a channel attention module and a spatial attention module, and the mixed domain convolutional attention module inputs the first feature map into the parameter-to-be-adjusted model, extracts features of the first feature map in the channel dimension and the spatial dimension, and obtains a second feature map of the insulator image sample set, including:

[0020] Performing feature processing on the first feature map through the channel attention module to obtain a first channel feature map;

[0021] By means of the spatial attention module, based on the width and height of the first channel feature map, performing maximum pooling and average pooling operations on the first channel feature map to obtain a second channel feature map;

[0022] Obtaining a second weight coefficient according to the second channel feature map;

[0023] A second feature map of the insulator image sample set is obtained according to the first channel feature map and the second weight coefficient.

[0024] In one embodiment, performing feature processing on the first feature map by the channel attention module to obtain the first channel feature map includes:

[0025] Performing maximum pooling and average pooling operations on the first feature map through the channel attention module to obtain the pooled first feature map;

[0026] Inputting the first feature map after pooling into a fully connected layer and an activation function to obtain a first weight coefficient;

[0027] A first channel feature map is obtained according to the first feature map and the first weight coefficient.

[0028] In one embodiment, obtaining a second weight coefficient according to the second channel feature map includes:

[0029] Performing a convolution operation on the second channel feature map to obtain the second channel feature map after the convolution operation;

[0030] Performing feature dimensionality reduction processing on the second channel feature map after the convolution operation to obtain the second channel feature map after the feature dimension is reduced;

[0031] A convolution operation is performed on the second channel feature map after reducing the feature dimension, and an S-shaped growth curve function is input to obtain a second weight coefficient.

[0032] In a second aspect, the present application also provides a method for detecting an insulator defect image, comprising:

[0033] Acquire an image of an insulator to be inspected;

[0034] The insulator image is input into an insulator defect detection model to obtain a defect detection result of the insulator image; the insulator defect detection model is obtained according to the above-mentioned insulator defect detection model construction method embodiment.

[0035] In a third aspect, the present application also provides an insulator defect detection model construction device, comprising:

[0036] A first feature map acquisition module is used to input the insulator image sample set into the backbone feature extraction network of the model to be adjusted in parameters, so as to obtain a first feature map of the insulator image sample set;

[0037] A second feature map acquisition module is used to input the first feature map into a mixed domain convolution attention module of a model to be adjusted, extract features of the first feature map in a channel dimension and a spatial dimension, and obtain a second feature map of the insulator image;

[0038] a third feature map acquisition module, configured to input the second feature map into a meta-feature extractor module of a model to be adjusted, learn meta-features of multiple scales from the second feature map; input the meta-features into a weighting module of a model to be adjusted, match corresponding weighting vectors for the meta-features of the multiple scales; and obtain a third feature map of the insulator image sample set according to the meta-features of the multiple scales and the weighting vectors;

[0039] A detection result acquisition module, used to obtain the detection result of the insulator image sample set according to the third feature map;

[0040] The model acquisition module is used to adjust the parameters of the model to be adjusted according to the difference between the detection result and the actual defect result, so as to obtain the insulator defect detection model.

[0041] In a fourth aspect, the present application also provides a device for detecting an insulator defect image, comprising:

[0042] An insulator image acquisition module, used for acquiring an insulator image to be detected;

[0043] The defect detection result acquisition module is used to input the insulator image into the insulator defect detection model to obtain the defect detection result of the insulator image; the insulator defect detection model is obtained by the above-mentioned insulator defect detection model construction method embodiment.

[0044] In a fifth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0045] In a sixth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the above method.

[0046] In a seventh aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to execute the above method.

[0047] The present application inputs an insulator image sample set into a backbone feature extraction network of a model whose parameters are to be adjusted, and obtains a first feature map of the insulator image sample set; inputs the first feature map into a mixed domain convolutional attention module of the model whose parameters are to be adjusted, extracts features of the first feature map in channel dimension and spatial dimension, and obtains a second feature map of the insulator image; inputs the second feature map into a meta-feature extractor module of the model whose parameters are to be adjusted, and learns meta-features of multiple scales from the second feature map; inputs the meta-features into a weighting module of the model whose parameters are to be adjusted, and matches corresponding weighting vectors for meta-features of multiple scales; obtains a third feature map of the insulator image sample set based on the meta-features of multiple scales and the weighting vectors; obtains a detection result of the insulator image sample set based on the third feature map; and adjusts the parameters of the model whose parameters are to be adjusted based on the difference between the detection result and the actual defect result, and obtains an insulator defect detection model. This application adds a mixed domain convolutional attention module to the insulator defect detection model to improve the feature extraction capability of the model. It also introduces a meta-feature learner and a weighting module into the insulator defect detection model, which can avoid the problems of being unable to train and overfitting when the number of insulator defect samples is insufficient, effectively improve the accuracy of insulator defect detection, and improve the performance of the insulator defect detection model, reduce power consumption, and can be deployed on unmanned aerial vehicle edge computing terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 An application environment diagram of an insulator defect detection model building method in one embodiment;

[0050] Figure 2 A schematic diagram of a flow chart of a method for constructing an insulator defect detection model in one embodiment;

[0051] Figure 3 A schematic diagram of the structure of an insulator defect detection model in one embodiment;

[0052] Figure 4 is a schematic diagram of obtaining a third characteristic graph in one embodiment;

[0053] Figure 5 is a structural schematic diagram of an image size adjustment model in one embodiment;

[0054] Figure 6 A diagram showing an application environment of a method for detecting an insulator defect image in one embodiment;

[0055] Figure 7 is a schematic flow chart of a method for detecting an insulator defect image in one embodiment;

[0056] Figure 8 A structural block diagram of an insulator defect detection model building device in one embodiment;

[0057] Fig. 9 is a structural block diagram of a device for detecting insulator defect images in one embodiment;

[0058] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The present application embodiment provides a method for constructing an insulator defect detection model. The present embodiment can be executed by a computer device, such as Figure 1 As shown, the computer device can obtain an insulator image sample set, and then obtain an insulator defect detection model. It can be understood that the computer device can be implemented by a server, a terminal, or an interactive system between a terminal and a server. In this embodiment, the method includes Figure 2 The steps shown are:

[0061] Step S201: input the insulator image sample set into the backbone feature extraction network of the model to be adjusted to obtain a first feature map of the insulator image sample set.

[0062] The insulator defect detection model to be trained is called a parameter-adjusted model.

[0063] In actual situations, due to the low incidence of insulator failures, the insulator image data that can be used for model training is seriously insufficient. In the case of insufficient number of insulator defect samples (in this case, the insulator defect samples can be called insulator defect small samples), this application improves the network structure of YOLOv7 on the basis of the existing YOLOv7 target detection algorithm, and obtains the following Figure 3 The insulator defect detection model shown in the figure can detect whether the insulator in the insulator image has defects (faults). The UPSample module is an upsampling module, and the upsampling method used is the nearest neighbor interpolation; the SPPCSP (Spatial Pyramid Pooling, Cross Stage Partial) module combines the characteristics of spatial pyramid pooling and cross-stage partial connection, which can better handle targets of different scales and improve the expression ability of features; the CAT module (Cross Stage Aggregation) is a feature-level fusion method; the REP module is divided into two parts, one is the training part (train) and the other is the reasoning part (deploy); the CBM module is used to improve the performance of the network in the target detection task, and is composed of the C3 module (Cross Stage Partial) and the SPP module (Spatial Pyramid Pooling).

[0064] The structure of the backbone feature extraction network is as follows Figure 3 As shown in the figure, the CBS module is composed of a convolutional layer (Conv), a batch normalization layer (BN) and an S-shaped growth curve function, where the S-shaped growth curve function can also be called a sigmoid activation function (SiLU); the ELAN (Effective Long-Range Aggregation Network) module is an efficient network structure that controls the shortest and longest gradient paths so that the network can learn more features and has stronger robustness; the MP layer is mainly divided into a maximum pooling layer (Maxpool) and a CBS module, where MP-1 and MP-2 are mainly changes in the ratio of the number of channels; the ELAN-W module is very similar to the ELAN module, except that the ELAN-W module selects a different number of outputs in the second branch.

[0065] Through the backbone feature extraction network (backbone), features of the insulator image sample set can be extracted to obtain the first feature map of the insulator image sample set.

[0066] Step S202: input the first feature map into the mixed domain convolutional attention module of the model to be adjusted, extract features of the first feature map in the channel dimension and the spatial dimension, and obtain a second feature map of the insulator image sample set.

[0067] The mixed domain convolutional attention module (CBAM) is an attention mechanism that aims to enhance the convolutional neural network (CNN) to learn the importance of different feature maps and introduce attention mechanisms between different layers to improve the ability of feature representation.

[0068] Through the mixed domain convolutional attention module, the features of the first feature map are extracted in the channel dimension and the spatial dimension to obtain the second feature map of the insulator image sample set, enriching the extracted high-level features of the insulator in the channel dimension and the spatial dimension.

[0069] Step S203, input the second feature map into the meta-feature extractor module of the model whose parameters are to be adjusted, learn meta-features of multiple scales from the second feature map; input the meta-features into the weighting module of the model whose parameters are to be adjusted, match the corresponding weighting vectors for the meta-features of multiple scales; obtain the third feature map of the insulator image sample set according to the meta-features of multiple scales and the weighting vectors. A meta-feature extractor generally refers to a module that can automatically learn and extract feature representations. The main function of a meta-feature extractor is to extract features with high information content and expression ability from raw data for subsequent tasks such as classification, detection, and recognition.

[0070] like Figure 4 As shown, through the meta-feature extractor module, meta-features of three different scales are learned from the second feature map, and the entire learning process can be learned and evaluated to obtain learning experience; N labeled insulator image samples are input into the weighting module, one for each category, and three groups of N weighted vectors are output, one for each scale; according to the weighted vectors, the meta-features of the three scales are recalibrated through channel multiplication to obtain the third feature map of the insulator image sample set.

[0071] The weighting module extracts meta-information from the insulator image samples and uses it to amplify meta-features that are informative for new objects in the insulator image samples. The weighting module (reweighting module) is pre-trained with the COCO (Common Objects in COntext) large public dataset and the ImageNet large public dataset as base class datasets, so that the weighting module can adaptively match different weights according to different features.

[0072] Step S204: obtaining a detection result of the insulator image sample set according to the third feature map.

[0073] The detection result of the insulator image sample set can be obtained according to the third feature map of the insulator image sample set.

[0074] Step S205, adjusting the parameters of the model to be adjusted according to the difference between the detection result and the actual defect result, to obtain an insulator defect detection model.

[0075] When the insulator image sample set is obtained, the defect result of each insulator image sample in the insulator image sample set can be manually determined as a true defect result.

[0076] According to the difference between the detection result and the actual defect result, a difference value can be obtained. According to the difference value, the parameters of the model to be adjusted are adjusted until the difference value is less than the difference threshold. The model to be adjusted at this time is used as the insulator defect detection model.

[0077] In the above-mentioned insulator defect detection model construction method, a mixed domain convolutional attention module is added to the insulator defect detection model to improve the feature extraction ability of the model. A meta-feature learner and a weighting module are also introduced into the insulator defect detection model, which can avoid the problems of untrainable and overfitting when the number of insulator defect samples is insufficient, effectively improve the accuracy of insulator defect detection, and improve the performance of the insulator defect detection model, reduce power consumption, and can be deployed on unmanned aerial vehicle edge computing terminals.

[0078] In one of the embodiments, before the insulator image is input into a pre-built insulator defect detection model, the method provided by the present application also includes: obtaining an initial sample set of the insulator image; inputting the initial sample set of the insulator image into an image resizing model, so that the image resizing model performs the following processing steps: dividing the input initial sample set of the insulator image into two branches, the first branch sequentially passes through a convolution layer, a leaky rectified linear unit, a standard convolution layer, a leaky rectified linear unit and a batch normalization layer, and then enters a bilinear adjuster, dividing the output result of the bilinear adjuster into two small branches, the first small branch passes through multiple residual blocks and convolution layers, batch normalization layers, and then merges with the second small branch, and then passes through a standard convolution layer; the second branch passes through the bilinear adjuster and is merged with the first branch to obtain a resized initial sample set of the insulator image; the resized initial sample set of the insulator image is used as the insulator image sample set.

[0079] In actual situations, due to the low incidence of insulator defects, the insulator image sample set (insulator defect image dataset) that can be used for model training is seriously insufficient. Therefore, the insulator image sample set can be expanded through data enhancement. Specifically, the data enhancement of insulator images can be performed by randomly transforming, flipping and rotating the image color to expand the initial sample set of insulator images.

[0080] After expanding the initial sample set of insulator images, the image size can be adjusted. In the image resizing stage (resize), a new image resizing model is introduced to replace the traditional image resizer. The image resizing model can achieve image resizing (resize) stage through joint learning with the detection model (detection model), so that the detection model can better adapt to various images and further improve the performance of the detection model.

[0081] The structure of the image resizing model is as follows Figure 5 As shown in the figure, the input insulator image initial sample set is divided into two branches. The first branch passes through the convolution layer, leaky rectified linear unit (LeakyReLu function), standard convolution layer, leaky rectified linear unit and batch normalization (BN) layer in sequence, and then enters the bilinear resizer. The output result of the bilinear resizer is divided into two small branches. The first small branch passes through multiple (r) residual blocks (res blocks) and convolution layers, batch normalization layers, and then merges with the second small branch, and then passes through the standard convolution layer; the second branch passes through the bilinear resizer and merges with the first branch to obtain the resized second insulator image sample set and output it. The middle convolution layer of the image resizing model has n (n=16) kernels of size 3×3. The convolution kernel size of the first and last layers is 7×7. Compared with the parameters of the traditional image resizer, the parameters introduced by the image resizing model are very small, which can make the insulator defect detection model better adapt to various images and further improve the performance of the insulator defect detection model.

[0082] Before the insulator defect detection model is trained, an initial sample set of insulator images can be obtained; the initial sample set of insulator images is input into the image resizing model so that the image resizing model performs the following processing steps: the input initial sample set of insulator images is divided into two branches, the first branch passes through a convolution layer, a leaky rectified linear unit, a standard convolution layer, a leaky rectified linear unit and a batch normalization layer in sequence, and then enters a bilinear adjuster, the output result of the bilinear adjuster is divided into two small branches, the first small branch passes through multiple residual blocks and convolution layers, batch normalization layers, and then is merged with the second small branch, and then passes through a standard convolution layer; the second branch passes through the bilinear adjuster and is merged with the first branch to obtain a resized initial sample set of insulator images; the resized initial sample set of insulator images is used as the insulator image sample set.

[0083] In this embodiment, the initial sample set of insulator images is input into the image resizing model to obtain the resized initial sample set of insulator images; the resized initial sample set of insulator images is used as the insulator image sample set; and the model is trained according to the insulator image sample set to obtain the insulator defect detection model, so that the insulator defect detection model can better adapt to various types of images and further improve the performance of the insulator defect detection model.

[0084] In one of the embodiments, the obtaining of the initial sample set of insulator images comprises the following specific steps: obtaining insulator pseudo-defect image samples based on the collected first-category insulator images and a generative adversarial network; partially or incompletely annotating the collected insulator images to obtain second-category insulator annotated image samples; and obtaining the initial sample set of insulator images based on the insulator pseudo-defect image samples and the second-category insulator annotated image samples.

[0085] The collected first-category insulator images can be input into the generative adversarial network to obtain insulator pseudo-defect image samples, which can enhance the diversity and richness of the initial sample set of insulator images and reduce the dependence on insulator image annotation data. In addition, self-supervised learning methods can be used to learn useful insulator feature representations from a large number of unlabeled first-category insulator images.

[0086] The collected insulator images are partially or incompletely labeled to obtain the second type of insulator labeled image samples, which can maintain a high detection performance of the insulator defect detection model when the collected insulator images are scarce.

[0087] According to the insulator pseudo-defect image samples and the second-category insulator labeled image samples, an initial sample set of insulator images is obtained.

[0088] In this embodiment, an initial sample set of insulator images is obtained based on insulator pseudo-defect image samples and second-category insulator annotated image samples, thereby enhancing the diversity and richness of the initial sample set of insulator images, and maintaining a high detection performance of the insulator defect detection model when the collected insulator images are scarce.

[0089] In one of the embodiments, model training is performed based on an insulator image sample set to obtain an insulator defect detection model, and the specific steps are as follows: obtaining the insulator artificial marking boxes of the insulator image sample set; clustering the insulator artificial marking boxes according to the marking box size to obtain clustering results; determining a pre-selected box based on the clustering results; and performing model training based on the insulator image sample set and the pre-selected box to obtain an insulator defect detection model.

[0090] The pre-selected box size provided by the insulator defect detection model may not be a good fit for the insulator size in the insulator image. If the size of the pre-selected box is significantly different from the size of the insulator marking box, the detection effect of the insulator defect detection model may be affected. In order to obtain the size of the insulator marking box that matches the insulator image sample set, the size of the insulator marking box of the insulator image sample set can be clustered according to the k-means algorithm.

[0091] Insulators in the insulator image sample set can be marked to obtain the insulator artificial marking boxes of the insulator image sample set; the insulator artificial marking boxes are clustered according to the marking box size to obtain the clustering result; according to the clustering result, the pre-selection box is determined, for example, if the clustering result is 9 pre-selection boxes of different sizes, the 9 pre-selection boxes of different sizes are determined as the pre-selection boxes of the insulator defect detection model; the model is trained according to the insulator image sample set and the pre-selection box to obtain the insulator defect detection model.

[0092] In this embodiment, the insulator artificial marking frames are clustered according to the marking frame size, the insulator marking frame size of the insulator image sample set is matched, and the model is trained according to the insulator image sample set and the pre-selected frame to obtain the insulator defect detection model, thereby improving the detection effect of the insulator defect detection model and improving the accuracy of insulator defect detection.

[0093] In one of the embodiments, the mixed domain convolutional attention module includes a channel attention module and a spatial attention module. The first feature map is input into the mixed domain convolutional attention module of the model whose parameters are to be adjusted, and the features of the first feature map are extracted in the channel dimension and the spatial dimension to obtain the second feature map of the insulator image sample set. The specific steps are as follows: the first feature map is feature processed by the channel attention module to obtain the first channel feature map; the first channel feature map is maximum pooled and average pooled based on the width and height of the first channel feature map by the spatial attention module to obtain the second channel feature map; the second weight coefficient is obtained according to the second channel feature map; the second feature map of the insulator image sample set is obtained according to the first channel feature map and the second weight coefficient.

[0094] The mixed domain convolutional attention module consists of two submodules: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM). The Channel Attention Module is used to model the relationship between different channels of the feature map, so that the insulator defect detection model can adaptively adjust the importance between channels; the Spatial Attention Module is used to model the relationship between different spatial positions of the feature map, thereby improving the expression and utilization of spatial information.

[0095] By combining the channel attention module and the spatial attention module, the mixed domain convolutional attention module can introduce the attention mechanism in both the channel dimension and the spatial dimension, so that the insulator defect detection model can automatically learn and adjust the information weights in the feature map according to different task requirements, thereby improving the performance of the insulator defect detection model on various visual tasks.

[0096] Specifically, the first feature map is input into the channel attention module, the insulator features of the first feature map are found, and the first channel feature map is obtained; the first channel feature map is input into the spatial attention module to find the position information of the insulator. Specifically, based on the width and height of the first channel feature map, the first channel feature map is subjected to maximum pooling and average pooling operations, and the feature dimension is limited to 1×1 to obtain the second channel feature map; according to the second channel feature map, the second weight coefficient is obtained; according to the first channel feature map and the second weight coefficient, the second feature map of the insulator image sample set is obtained, and the recalibration in channel and space is completed.

[0097] In this embodiment, high-level features of the insulator are extracted in the channel dimension and the spatial dimension, and are weighted to the initial features after obtaining the weights of the space and the channel respectively, thereby completing the dual attention adjustment of the features, focusing more on the important features of the insulator from both the channel and space aspects, and enhancing the receptive field of the insulator defect detection model, so that the insulator defect detection model can better understand the insulator image, thereby improving the accuracy of insulator defect detection.

[0098] In one of the embodiments, feature processing is performed on the first feature map through a channel attention module to obtain a first channel feature map, and the specific steps are as follows: maximum pooling and average pooling operations are performed on the first feature map through the channel attention module to obtain a pooled first feature map; the pooled first feature map is input into a fully connected layer and an activation function to obtain a first weight coefficient; and the first channel feature map is obtained based on the first feature map and the first weight coefficient.

[0099] Through the channel attention module, the first feature map is subjected to maximum pooling and average pooling operations to obtain the pooled first feature map, which can enrich the extracted high-level features of the insulator in the channel dimension; the pooled first feature map is input into the fully connected layer and the activation function to obtain the first weight coefficient; the inner product operation is performed on the first feature map and the first weight coefficient to obtain the first channel feature map.

[0100] In this embodiment, a first channel feature map is obtained based on the first feature map and the first weight coefficient, and the weight of the channel is weighted to the initial feature, i.e., the first feature map, to complete the feature attention adjustment, so that the insulator defect detection model can better understand the insulator image, thereby improving the accuracy of insulator defect detection.

[0101] In one embodiment, a second weight coefficient is obtained according to the second channel feature map, and the specific steps are as follows: performing a convolution operation on the second channel feature map to obtain a second channel feature map after the convolution operation; performing feature dimension reduction processing on the second channel feature map after the convolution operation to obtain a second channel feature map after reducing the feature dimension; performing a convolution operation on the second channel feature map after reducing the feature dimension, and inputting an S-shaped growth curve function to obtain a second weight coefficient.

[0102] A convolution operation is performed on the second channel feature map, which may be a single convolution operation, to obtain a second channel feature map after the convolution operation; the second channel feature map after the convolution operation may be input into an activation function to perform feature dimension reduction processing to obtain a second channel feature map after reducing the feature dimension; a convolution operation is performed on the second channel feature map after reducing the feature dimension, which may be a single convolution operation, to increase the feature dimension to the original dimension, and the second channel feature map increased to the original dimension is input into an S-shaped growth curve function to obtain a second weight coefficient.

[0103] In this embodiment, a convolution operation and a feature dimension reduction process are performed on the second channel feature map, and finally an S-shaped growth curve function is input to obtain a more accurate second weight coefficient.

[0104] The present application also provides a method for detecting defective images of insulators. ... method can be executed by a computer device. Figure 6 As shown, the computer device can obtain the insulator image to be detected, and then obtain the defect detection result of the insulator image. It can be understood that the computer device can be implemented by a server, a terminal, or an interactive system between a terminal and a server. In this embodiment, the method includes Figure 7 The steps shown are:

[0105] Step S701, obtaining an image of an insulator to be inspected.

[0106] Based on the drone inspection technology, the insulator image taken by the drone can be obtained as the insulator image to be inspected.

[0107] Step S702, inputting the insulator image into an insulator defect detection model to obtain a defect detection result of the insulator image; the insulator defect detection model is obtained according to the above-mentioned insulator defect detection model construction method embodiment.

[0108] The insulator defect detection model is obtained according to the insulator defect detection model construction method of the present application. The insulator defect detection model can detect the insulator image and output the corresponding defect detection result.

[0109] In order to better understand the above method, an application example of the insulator defect detection model construction method of the present application is described in detail below.

[0110] According to the collected first-category insulator images and the generative adversarial network, insulator pseudo-defect image samples are obtained; the collected insulator images are partially or incompletely labeled to obtain second-category insulator labeled image samples; according to the insulator pseudo-defect image samples and the second-category insulator labeled image samples, an initial sample set of insulator images is obtained.

[0111] The initial sample set of the insulator image is input into the image resizing model, so that the image resizing model performs the following processing steps: the input initial sample set of the insulator image is divided into two branches, the first branch passes through the convolution layer, the leaky rectified linear unit, the standard convolution layer, the leaky rectified linear unit and the batch normalization layer in sequence, and then enters the bilinear adjuster, the output result of the bilinear adjuster is divided into two small branches, the first small branch passes through multiple residual blocks and the convolution layer, the batch normalization layer, and then is merged with the second small branch and passes through the standard convolution layer; the second branch passes through the bilinear adjuster and is merged with the first branch to obtain the resized initial sample set of the insulator image; the resized initial sample set of the insulator image is used as the insulator image sample set.

[0112] The insulator image sample set is input into the backbone feature extraction network of the model to be adjusted, and the first feature map of the insulator image sample set is obtained; the first feature map is input into the mixed domain convolution attention module of the model to be adjusted, and the first feature map is feature processed through the channel attention module to obtain a first channel feature map; the first channel feature map is subjected to maximum pooling and average pooling operations based on the width and height of the first channel feature map through the spatial attention module to obtain a second channel feature map; according to the second channel feature map, a second weight coefficient is obtained; according to the first channel feature map and the second weight coefficient, a second feature map of the insulator image is obtained.

[0113] The second feature map is input into the meta-feature extractor module of the model whose parameters are to be adjusted, and meta-features of multiple scales are learned from the second feature map; the meta-features are input into the weighting module of the model whose parameters are to be adjusted, and corresponding weighting vectors are matched for the meta-features of multiple scales; a third feature map of the insulator image sample set is obtained according to the meta-features of multiple scales and the weighting vectors; a detection result of the insulator image sample set is obtained according to the third feature map; and according to the difference between the detection result and the actual defect result, the parameters of the model whose parameters are to be adjusted are adjusted to obtain an insulator defect detection model.

[0114] In addition, the feature pyramid network structure in the insulator defect detection model can be optimized to achieve more sophisticated multi-scale detection. By performing feature extraction and defect detection at different scales, the defect characteristics of the insulator at different perspectives can be better captured, improving the robustness of detection.

[0115] After obtaining the insulator defect detection model, we can combine real-time image acquisition technology and defect tracking algorithm to build a real-time tracking and early warning system for insulator defects based on drones. The system can immediately issue an early warning when an insulator defect is detected, providing strong support for timely maintenance and ensuring the safe operation of transmission lines.

[0116] After obtaining the insulator defect detection model, the insulator defect detection model can also be optimized by introducing domain adaptation technology or transfer learning algorithm to enhance the cross-domain transfer learning capability of the insulator defect detection model, so that the trained insulator defect detection model can quickly adapt to insulator defect detection tasks in different scenarios.

[0117] After obtaining the insulator defect detection model, a visual interpretation tool for the insulator defect detection model can also be developed to provide intuitive visual display and interpretation of the detection results. At the same time, combined with the fault diagnosis knowledge base, detailed fault diagnosis suggestions and maintenance guidance can be provided to maintenance personnel.

[0118] After obtaining the insulator defect detection model, it is also possible to combine multiple sensor data (such as infrared thermal imaging, laser radar, etc.) to obtain insulator images of multiple modal types, and perform multimodal fusion detection based on the insulator images of multiple modal types to obtain the detection results of the insulator images of each modal type. Based on the detection results of the insulator images of each modal type, the combined detection results of the insulator images are obtained. The detection results of different modalities can provide complementary information, thereby improving the accuracy and robustness of insulator defect detection.

[0119] In view of the changes of transmission line insulators in different seasons and weather conditions, the adaptability of the insulator defect detection model can be enhanced. Specifically, seasonal characteristics and weather condition characteristics are introduced into the insulator defect detection model, so that the insulator defect detection model can better cope with detection tasks in different environments.

[0120] In this embodiment, an adaptive feature fusion mechanism, namely a weighted module, is introduced to automatically adjust the fusion weights of features of different scales according to the specific features of the insulator image, so as to more accurately capture the key features of insulator defects and improve the accuracy of detection. This embodiment also optimizes the structure of the insulator defect detection model, designs a lighter network architecture, reduces the amount of calculation and parameters, improves the detection speed, and maintains a high detection accuracy. This is especially important in edge devices or application scenarios with high real-time requirements.

[0121] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0122] Based on the same inventive concept, the embodiment of the present application also provides an insulator defect detection model construction device for implementing the above-mentioned insulator defect detection model construction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more insulator defect detection model construction device embodiments provided below can refer to the limitations of the insulator defect detection model construction method above, and will not be repeated here.

[0123] In an exemplary embodiment, Figure 8 As shown, a device for building an insulator defect detection model is provided, wherein:

[0124] A first feature map acquisition module 801 is used to input the insulator image sample set into the backbone feature extraction network of the model to be adjusted in parameters, and obtain a first feature map of the insulator image sample set;

[0125] A second feature map acquisition module 802 is used to input the first feature map into a mixed domain convolution attention module of a model to be adjusted, extract features of the first feature map in a channel dimension and a spatial dimension, and obtain a second feature map of the insulator image;

[0126] The third feature map acquisition module 803 is used to input the second feature map into the meta-feature extractor module of the parameter to be adjusted model, learn meta-features of multiple scales from the second feature map; input the meta-features into the weighting module of the parameter to be adjusted model, match the meta-features of the multiple scales with corresponding weighting vectors; obtain the third feature map of the insulator image sample set according to the meta-features of the multiple scales and the weighting vectors;

[0127] A detection result acquisition module 804 is used to obtain the detection result of the insulator image sample set according to the third feature map;

[0128] The model acquisition module 805 is used to adjust the parameters of the model to be adjusted according to the difference between the detection result and the actual defect result, so as to obtain the insulator defect detection model.

[0129] In one embodiment, the device also includes a sample set acquisition module, which is used to: acquire an initial sample set of an insulator image; input the initial sample set of the insulator image into an image resizing model, so that the image resizing model performs the following processing steps: divide the input initial sample set of the insulator image into two branches, the first branch passes through a convolution layer, a leaky rectified linear unit, a standard convolution layer, a leaky rectified linear unit and a batch normalization layer in sequence, and then enters a bilinear adjuster, and the output result of the bilinear adjuster is divided into two small branches, the first small branch passes through multiple residual blocks and convolution layers, batch normalization layers, and then is merged with the second small branch and passes through a standard convolution layer; the second branch passes through the bilinear adjuster and is merged with the first branch to obtain a resized initial sample set of the insulator image; the resized initial sample set of the insulator image is used as the insulator image sample set.

[0130] In one of the embodiments, the sample set acquisition module is further used to: obtain insulator pseudo-defect image samples based on the collected first-category insulator images and the generative adversarial network; partially or incompletely annotate the collected insulator images to obtain second-category insulator annotated image samples; and obtain an initial sample set of insulator images based on the insulator pseudo-defect image samples and the second-category insulator annotated image samples.

[0131] In one embodiment, the second feature map acquisition module 802 is further used to: perform feature processing on the first feature map through the channel attention module to obtain a first channel feature map; perform maximum pooling and average pooling operations on the first channel feature map based on the width and height of the first channel feature map through the spatial attention module to obtain a second channel feature map; obtain a second weight coefficient based on the second channel feature map; and obtain a second feature map of the insulator image based on the first channel feature map and the second weight coefficient.

[0132] In one embodiment, the second feature map acquisition module 802 is further used to: perform maximum pooling and average pooling operations on the first feature map through the channel attention module to obtain the first feature map after pooling; input the first feature map after pooling into the fully connected layer and the activation function to obtain a first weight coefficient; and obtain a first channel feature map based on the first feature map and the first weight coefficient.

[0133] In one embodiment, the second feature map acquisition module 802 is further used to: perform a convolution operation on the second channel feature map to obtain the second channel feature map after the convolution operation; perform feature dimension reduction processing on the second channel feature map after the convolution operation to obtain the second channel feature map after the feature dimension is reduced; perform a convolution operation on the second channel feature map after the feature dimension is reduced, and input an S-shaped growth curve function to obtain a second weight coefficient.

[0134] Each module in the above-mentioned insulator defect detection model construction device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0135] Based on the same inventive concept, the embodiment of the present application also provides an insulator defect detection model construction device for implementing the above-mentioned insulator defect detection model construction method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more insulator defect detection model construction device embodiments provided below can refer to the limitations of the insulator defect detection model construction method above, and will not be repeated here.

[0136] In an exemplary embodiment, Fig. 9 As shown, a device for detecting an insulator defect image is provided, wherein:

[0137] An insulator image acquisition module 901 is used to acquire an insulator image to be detected;

[0138] The defect detection result acquisition module 902 is used to input the insulator image into the insulator defect detection model to obtain the defect detection result of the insulator image; the insulator defect detection model is obtained according to the method according to any one of claims 1 to 6. In an exemplary embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as insulator images and detection results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for constructing an insulator defect detection model and a method for detecting an insulator defect image are implemented.

[0139] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing an insulator defect detection model, characterized in that: The method comprises: Inputting the insulator image sample set into the backbone feature extraction network of the model to be adjusted in parameters, obtaining a first feature map of the insulator image sample set; Inputting the first feature map into a mixed domain convolutional attention module of a model to be adjusted, extracting features of the first feature map in a channel dimension and a spatial dimension, and obtaining a second feature map of the insulator image sample set; Inputting the second feature map into a meta-feature extractor module of a model to be adjusted, learning meta-features of multiple scales from the second feature map; inputting the meta-features into a weighting module of a model to be adjusted, matching corresponding weighting vectors for the meta-features of the multiple scales; obtaining a third feature map of the insulator image sample set according to the meta-features of the multiple scales and the weighting vectors; Obtaining a detection result of the insulator image sample set according to the third feature map; According to the difference between the detection result and the actual defect result, the parameters of the model to be adjusted are adjusted to obtain an insulator defect detection model.

2. The method according to claim 1, characterized in that Before inputting the insulator image sample set into a backbone feature extraction network of an insulator defect detection model, the method further includes: Obtaining an initial sample set of insulator images; The initial sample set of the insulator image is input into the image resizing model, so that the image resizing model performs the following processing steps: the input initial sample set of the insulator image is divided into two branches, the first branch passes through the convolution layer, the leaky rectified linear unit, the standard convolution layer, the leaky rectified linear unit and the batch normalization layer in sequence, and then enters the bilinear adjuster, the output result of the bilinear adjuster is divided into two small branches, the first small branch passes through multiple residual blocks and the convolution layer, the batch normalization layer, and then is merged with the second small branch and passes through the standard convolution layer; the second branch passes through the bilinear adjuster and then is merged with the first branch to obtain the resized initial sample set of the insulator image; The resized initial sample set of insulator images is used as the insulator image sample set.

3. The method according to claim 2, characterized in that The step of obtaining an initial sample set of insulator images comprises: According to the collected first-category insulator images and the generative adversarial network, insulator pseudo-defect image samples are obtained; Partially or incompletely annotate the collected insulator images to obtain the second type of insulator annotated image samples; An initial sample set of insulator images is obtained according to the insulator pseudo-defect image samples and the second type of insulator labeled image samples.

4. The method according to claim 1, characterized in that The mixed domain convolution attention module includes a channel attention module and a spatial attention module. The mixed domain convolution attention module inputs the first feature map into the parameter-to-be-adjusted model, extracts features of the first feature map in the channel dimension and the spatial dimension, and obtains a second feature map of the insulator image sample set, including: Performing feature processing on the first feature map through the channel attention module to obtain a first channel feature map; By means of the spatial attention module, based on the width and height of the first channel feature map, performing maximum pooling and average pooling operations on the first channel feature map to obtain a second channel feature map; Obtaining a second weight coefficient according to the second channel feature map; A second feature map of the insulator image sample set is obtained according to the first channel feature map and the second weight coefficient.

5. The method according to claim 4, characterized in that The step of performing feature processing on the first feature map by the channel attention module to obtain a first channel feature map includes: Performing maximum pooling and average pooling operations on the first feature map through the channel attention module to obtain the pooled first feature map; Inputting the first feature map after pooling into a fully connected layer and an activation function to obtain a first weight coefficient; A first channel feature map is obtained according to the first feature map and the first weight coefficient.

6. The method according to claim 4, characterized in that The obtaining a second weight coefficient according to the second channel feature map includes: Performing a convolution operation on the second channel feature map to obtain the second channel feature map after the convolution operation; Performing feature dimensionality reduction processing on the second channel feature map after the convolution operation to obtain the second channel feature map after the feature dimension is reduced; A convolution operation is performed on the second channel feature map after reducing the feature dimension, and an S-shaped growth curve function is input to obtain a second weight coefficient.

7. A method for detecting an insulator defect image, characterized in that: The method comprises: Acquire an image of an insulator to be inspected; The insulator image is input into an insulator defect detection model to obtain a defect detection result of the insulator image; the insulator defect detection model is obtained according to the method according to any one of claims 1 to 6.

8. An insulator defect detection model building device, characterized in that: The device comprises: A first feature map acquisition module is used to input the insulator image sample set into the backbone feature extraction network of the model to be adjusted in parameters, so as to obtain a first feature map of the insulator image sample set; A second feature map acquisition module is used to input the first feature map into a mixed domain convolution attention module of a model to be adjusted, extract features of the first feature map in a channel dimension and a spatial dimension, and obtain a second feature map of the insulator image sample set; a third feature map acquisition module, configured to input the second feature map into a meta-feature extractor module of a model to be adjusted, learn meta-features of multiple scales from the second feature map; input the meta-features into a weighting module of a model to be adjusted, match corresponding weighting vectors for the meta-features of the multiple scales; and obtain a third feature map of the insulator image sample set according to the meta-features of the multiple scales and the weighting vectors; A detection result acquisition module, used to obtain the detection result of the insulator image sample set according to the third feature map; The model acquisition module is used to adjust the parameters of the model to be adjusted according to the difference between the detection result and the actual defect result, so as to obtain the insulator defect detection model.

9. An insulator defect image detection device, characterized in that: The device comprises: An insulator image acquisition module, used for acquiring an insulator image to be detected; A defect detection result acquisition module is used to input the insulator image into an insulator defect detection model to obtain a defect detection result of the insulator image; the insulator defect detection model is obtained according to the method according to any one of claims 1 to 6.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images