Small sample insulator defect detection method based on cross-domain feature fusion migration

Abstract

The invention discloses a small sample insulator defect detection method based on cross-domain feature fusion migration. The method comprises the following steps: pre-training a ConvNeXt network through an ImageNet image data set; freezing the weight of a trunk convolutional layer of the ConvNeXt network, and constructing dynamic feature modulation modules based on an attention mechanism in a shallow network and a deep network of the ConvNeXt network for extracting local features and global features of the insulator; and fusing the extracted local and global features, and realizing multi-dimensional interaction of local detail features and global semantic features by adopting a top-down feature information stacking transmission strategy and a bottom-up progressive fusion strategy. The local-global interaction mechanism and the transfer learning method are organically combined, a cross-domain feature transfer channel is established, robust mapping from low-level visual features to high-level semantic features is effectively achieved, and the detection precision of the insulator image under the condition that samples are rare is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target detection, and particularly relates to a small sample insulator defect detection method based on cross-domain feature fusion transfer. BACKGROUND

[0002] As a key part of the power transmission line, the insulator plays an important role in supporting the conductor and isolating the current to prevent short circuit, etc., ensuring the safety and stability of the line. However, the insulator is easily affected by various factors such as environmental factors, mechanical stress and electrical stress during long-term use, and various defects such as breakage, flashover and string drop occur. If these defects are not discovered and repaired in time, they may cause equipment failure and even large-scale power accidents, so it is very important to regularly check the running state of the insulator.

[0003] With the development of artificial intelligence technology, especially the rise of deep learning and big data technology, the defect detection technology has entered a new stage. Due to the powerful processing capability of visual images, deep convolutional neural network models are often used for defect monitoring. However, although the current mainstream target detection algorithms such as YOLO have certain advancement in the field of target detection, such deep convolutional neural networks usually need a large number of image data as a data set to obtain higher precision. However, the number of defect samples is usually small, such as the insulator defects of the power transmission line targeted by the present application, so it is difficult to perform deep learning through direct model training.

[0004] There are two solutions to the small sample defect detection problem: one is data augmentation, that is, expanding the small sample data set to achieve sufficient sample size for deep learning; the second is transfer learning, which retains part of the model parameters pre-trained on other similar domain data sets, and trains the remaining model with the target domain data set. However, the commonly used transfer learning method improves the classification performance through the basic fully connected layer (FC) improvement method, which often leads to insufficient model generalization ability and low robustness. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the problem of insufficient model generalization ability caused by the existing transfer learning method. The present application provides a small sample insulator defect detection method based on cross-domain feature fusion transfer, which can improve the precision of defect detection and obtain better detection effect under the condition of insufficient samples.

[0006] The technical scheme for achieving the object of the present application is as follows: a small sample insulator defect detection method based on cross-domain feature fusion transfer, comprising the following steps:

[0007] Step 1, using an insulator data set as a detection data set and pre-processing the data set;

[0008] Step 2, build a ConvNeXt network and use an ImageNet image dataset as a training set to perform transfer learning training on the ConvNeXt network;

[0009] Step 3, construct a local feature aggregation module in the shallow network of the ConvNeXt network and a global feature aggregation module in the deep network;

[0010] Step 4, introduce a feature fusion module into the ConvNeXt network to construct a cross-level bidirectional feature interaction path and realize multi-dimensional interaction of local detail features and global semantic features;

[0011] Step 5, freeze the weights of the convolutional layers of the ConvNeXt network, train the ConvNeXt network using a defect insulator dataset, and obtain a defect detection result.

[0012] Further, in step 3, the local feature aggregation module constructed in the shallow network is an improved discrete convolution module, specifically, the convolution kernel in the sub-network Block of the ConvNeXt network is replaced, and the original fixed-size convolution kernel is adjusted to a variable convolution kernel, i.e., a convolution kernel that can adjust the size of the dynamic receptive field in real time according to the image size, while performing channel attention weighting on the convolution result to highlight important local features.

[0013] Further, the local feature aggregation module includes a learnable discrete convolution module, which decomposes continuous convolution kernels into a set of learnable discrete components by discretizing the convolution kernel parameter space, and combines the components with dynamically generated weight coefficients α k to form the final convolution kernel W ds , satisfying the following expression:

[0014]

[0015] where N represents the total number of discrete base convolution kernels, α k is a dynamically generated weight coefficient, indicates that the discrete base convolution kernel is dynamically generated according to the input image features to enhance the response capability of different image regions.

[0016] Further, in step 3, the global feature aggregation module constructed in the deep network is a multi-head attention mechanism added to the ConvNeXt backbone network, specifically: an attention mechanism is added between the convolutional layers and the fully connected layers of the ConvNeXt network, which gives an input feature map F∈R H×W×Cwherein H, W, C represent the height, width, and channel number of the feature map respectively, the query matrix Q, the key matrix K, and the value matrix V are generated by linear projection, multiple heads are calculated in parallel, and self-attention calculation and weighting are performed through the decomposed matrix, and the formula based on the scaled dot-product attention mechanism is as follows:

[0017]

[0018] wherein Attention(Q, K, V) refers to the calculation process of the attention mechanism; d k refers to the dimension of the key matrix K, that is, the dimension length of each query matrix Q and the key matrix K; Softmax refers to normalizing the scaled similarity matrix into a probability distribution for use as attention weights. After obtaining the attention output, residual connection is performed with the original input feature to retain the original information and enhance the global semantic modeling capability.

[0019] Further, in step 4, the introduced feature fusion module has the following steps:

[0020] Step 4.1, the high-resolution local feature of 56*56 in the shallow layer (Stage 1-2) is up-sampled through bilinear interpolation, and the low-resolution global feature of 7*7 in the deep layer (Stage 3-4) is down-sampled through a pooling operation;

[0021] Step 4.2, the up-sampled local feature and the down-sampled global feature are spliced in the channel dimension to form multi-scale fusion features;

[0022] Step 4.3, the deep semantic feature is fused with the shallow detail feature level by level through up-sampling, and the shallow detail feature is fused with the deep semantic feature through attention weighting to generate a unified multi-scale feature pyramid;

[0023] Step 4.4, the fused multi-scale features are input into the global average pooling and the fully connected layer.

[0024] Further, in step 5, a hierarchical freezing method is adopted, that is, during the model transfer training process, the convolution layer parameters are fixed, and only the pre-training parameters of the ConvNeXt Block and the fully connected layer are adjusted.

[0025] A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0026] A computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the steps of the above method.

[0027] A computer program product comprising a computer program which, when executed by a processor, implements the steps of the above method.

[0028] The small sample insulator defect detection method based on cross-domain feature fusion migration provided by the application, in view of the insulator defect sample scarce scene, combines the transfer learning method and the feature fusion technology, and improves the defect detection precision;

[0029] Compared with the prior art, the application has the following beneficial effects and advantages:

[0030] 1) The ImageNet-22K dataset is used for pre-training, and a better initial model weight is obtained;

[0031] 2) The transfer learning method is used to solve the problem of poor training fitting effect caused by insufficient defect samples;

[0032] 3) The feature fusion technology is used to interact the local feature and the global feature, and the problem of insufficient model generalization ability in the traditional transfer learning method is optimized;

[0033] 4) The application has strong generalization and can be used in any sample scarce detection problem. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a network structure diagram of the ConvNeXt model.

[0035] Figure 2 It is a whole flow chart of the transfer training of the application.

[0036] Figure 3 It is a multi-level feature extraction structure diagram of the application.

[0037] Figure 4 It is a local feature aggregation and global feature aggregation step diagram of the application.

[0038] Figure 5 It is a local-global interaction mechanism information exchange diagram of the application.

[0039] Figure 6 It is a comparison diagram of the insulator detection accuracy of the training set using the feature aggregation technology and not using the feature aggregation technology. DETAILED DESCRIPTION

[0040] The application provides a small sample insulator defect detection method based on cross-domain feature fusion migration, comprising: pre-training a ConvNeXt network through an ImageNet image dataset to obtain a group of initial pre-training weights; freezing the main convolutional layer weights of the ConvNeXt network, and constructing a dynamic feature modulation module based on an attention mechanism in the shallow and deep layers of the ConvNeXt network to extract local and global features of the insulator; fusing the extracted local and global features with each other, adopting a top-down feature information stacking transmission and a bottom-up progressive fusion strategy to realize multidimensional interaction of local detailed features and global semantic features. The above local-global interaction mechanism is organically combined with a transfer learning method to establish a cross-domain feature migration channel, effectively realizing robust mapping of low-level visual features to high-level semantic features, and improving the detection accuracy of insulator images in the case of few samples.

[0041] The technical solutions of the application will be described in further detail below in combination with the drawings and embodiments.

[0042] A small sample insulator defect detection method based on cross-domain feature fusion migration has the following specific steps:

[0043] Step 1, selecting an insulator dataset as a detection dataset and further performing dataset preprocessing;

[0044] All insulator image datasets, i.e., a Chinese power transmission line insulator dataset (CPLID), are obtained, and photos taken by a drone are supplemented;

[0045] The images are preprocessed, the obtained images are first screened to remove blurred and poor-quality images, then the small defect parts are cropped and adjusted to a standardized size, i.e., the size of the dataset pictures is set to 224x224x3, and finally 1774 defect instances are obtained;

[0046] The insulator defect data are classified into three categories, i.e., damage, flashover and string drop, and the training set and the test set are divided according to a 4:1 ratio.

[0047] Step 2, as shown in Figure 2 , a ConvNeXt network is built, and the ImageNet image dataset is used as a training set to pre-train the ConvNeXt network through transfer learning;

[0048] The pre-training dataset of the transfer learning source domain selects an image classification dataset ImageNet-22K covering 22000 categories;

[0049] The ConvNeXt network is built, and its structure is as shown in Figure 1As shown, ConvNeXt is an efficient pure convolutional network that combines the advantages of traditional convolutional neural networks and Transformers. The key parts of building this network include ConvNeXt Block and down-sampling layers.

[0050] ConvNeXt Block contains depthwise convolution (Depthwise Conv), LayerNorm, and two layers of MLP, which is the key to extracting insulator defect features.

[0051] Down-sampling layers are mainly used to reduce image resolution.

[0052] Select the model pre-training parameters and optimizers. The pre-training parameters include learning rate, weight decay coefficient, momentum, iteration number, and batch size to get the initial model weight.

[0053] Use the dataset created in step 1 as the target domain dataset for transfer learning.

[0054] Freeze the weights of the convolutional layers of the ConvNeXt backbone network, change the structure of the fully connected layer, and train the remaining part of the ConvNeXt network using a small sample insulator dataset.

[0055] For the ConvNeXt Block module, local feature aggregation is achieved through bilinear interpolation.

[0056] For the down-sampling layer, global feature aggregation is achieved through maximum pooling.

[0057] Step 3: Construct local feature aggregation modules in the shallow network (Stage 1-2) of the ConvNeXt model and global feature aggregation modules in the deep network (Stage 3-4). The process can be shown by Figure 3 、 Figure 4 As shown:

[0058] Deploy the improved discrete convolution module in the shallow network part. Adjust the parameters of the discrete convolution kernel (DSConv).

[0059] According to the spatial distribution of input features, dynamically adjust the sampling position of the convolution kernel (offset learning) to make the convolution kernel adaptive to the defect shape.

[0060] Perform channel attention weighting on the convolution result to highlight important local features.

[0061] Construct a global feature extractor based on multi-head self-attention in the deep network.

[0062] Input feature map F ∈ R H×W×C, the query matrix Q, key matrix K and value matrix V are generated by linear projection, and self-attention calculation is performed through the decomposed matrix;

[0063] The attention output is connected in residual connection with the original input feature.

[0064] The local feature aggregation module decomposes the traditional continuous convolution kernel into a set of learnable discrete components by discretizing the convolution kernel parameter space, and the discrete set is mathematically expressed as:

[0065]

[0066] Where N represents the total number of discrete base convolution kernels, represents the discrete base convolution kernel, and α k is a dynamically generated weight coefficient. The variable convolution mechanism dynamically adjusts the sampling network offset Δp n , so that the convolution kernel can adapt to the irregular geometric features of the surface defects of the insulator.

[0067] To further enhance the feature representation capability, a channel attention unit is introduced, and the local feature output F' local The calculation process can be represented as:

[0068] F′ local =σ(MLP(AvgPool(F local )))*F local

[0069] Where F local is the local feature map; AvgPool is the global average pooling; MLP is a multilayer perceptron for modeling the dependency between channels; and σ is a sigmoid activation function.

[0070] The global feature needs to introduce an attention mechanism, given the input feature map F∈R H×W×C , the query matrix Q, key matrix K and value matrix V are generated by linear projection, and self-attention calculation is performed through the decomposed matrix, and the attention weight calculation is:

[0071]

[0072] Where Attention(Q,K,V) refers to the calculation process of the attention mechanism; d k refers to the dimension of the key matrix K, that is, the dimension length of each query matrix Q and key matrix K; and Softmax refers to normalizing the scaled similarity matrix to a probability distribution, which is used as the attention weight.

[0073] To preserve spatial detail information, the attention output is connected with the original input feature in residual connection, preserving the original information and enhancing long-range dependencies. A residual enhanced attention module is designed, whose global feature output F ′ global The calculation process can be represented as:

[0074] F′ global = LayerNorm(F + Dropout(MSA(F)))

[0075] Where F is the input feature map; MSA is the multi-head attention module; Dropout is the regularization operation to prevent overfitting; LayerNorm is the layer normalization process.

[0076] Step 4, introduce a feature fusion module in the ConvNeXt network to build a cross-level bidirectional feature interaction channel, realize multi-dimensional interaction of local detail features and global semantic features, and the process can be shown by Figure 5

[0077] The shallow high-resolution local features are upsampled to the same resolution as the deep features through bilinear interpolation;

[0078] The deep low-resolution global features are downsampled to match the resolution of the shallow features through pooling operation;

[0079] The upsampled local features and downsampled global features are concatenated in the channel dimension to form multi-scale fusion features;

[0080] The deep semantic features are gradually fused with the shallow detail features through upsampling, and the shallow detail features are modulated through attention weighted modulation of the deep semantic features to generate a unified multi-scale feature pyramid.

[0081] The fused multi-scale features are input into global average pooling (GAP) and fully connected layer (FC) to output classification results;

[0082] The local enhanced feature F fusion after bottom-up fusion is represented as:

[0083] F fusion = Conv 1×1 ([Up(F low ), Down(F high )])

[0084] Where F low , F high represent the input features from the shallow and deep layers respectively; Up and Down represent up and down sampling operations respectively; Conv 1×1 is the convolution operation for fusion; ​

[0085] Top-down global semantic feature F′ low The guidance is expressed as:

[0086] F′ low =A high ×Up(F low )

[0087] Among them, A high The attention guidance matrix or weight map generated from deep (stage 3-4) features;

[0088] Finally, through the hierarchical fusion architecture of the feature pyramid network, a multi-scale feature pyramid with unified semantic features is generated, which contains four stages of output feature maps {P2, P3, P4, P5}, whose resolutions are {1 / 4, 1 / 8, 1 / 16, 1 / 32} of the input image.

[0089] Step 5: Freeze the weights of the convolutional layer of the ConvNeXt network and train the ConvNeXt network using the defective insulator dataset to obtain defect detection results.

[0090] The improved ConvNeXt model was trained and tested, and compared with the model before improvement. The training parameters selected by the present invention are as follows: the initial learning rate (LR) is set to 0.00005, the transfer learning rate is 0.0005, the weight decay coefficient is set to 0.005, the momentum is set to 0.9, the optimizer is SGD, the batch_size is set to 16, and the number of epochs is set to 400. The detection accuracy of the model before improvement is compared with that of the original model. Figure 6 As shown in the figure, it can be seen that the improved local-global feature aggregation mechanism has improved the accuracy of insulator defect detection compared with using only local features, global features or the original model.

[0091] The above description is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change, or equivalent structural change made to the above dataset composition and training parameters based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A small sample insulator defect detection method based on cross-domain feature fusion migration, characterized in that: The method comprises the following steps: Step 1: Use the insulator dataset as the detection dataset and preprocess the dataset; Step 2: Build a ConvNeXt network and use the ImageNet image dataset as a training set to perform transfer learning training on the ConvNeXt network. Step 3: Build a local feature aggregation module in the shallow network of the ConvNeXt network and a global feature aggregation module in the deep network; Step 4: Introduce a feature fusion module into the ConvNeXt network to build a cross-level bidirectional feature interaction path to achieve multi-dimensional interaction between local detail features and global semantic features; Step 5: Freeze the weights of the convolutional layer of the ConvNeXt network and train the ConvNeXt network using the defective insulator dataset to obtain defect detection results.

2. The small sample insulator defect detection method based on cross-domain feature fusion migration according to claim 1 is characterized in that: In step 3, the local feature aggregation module constructed in the shallow network is an improved discrete convolution module. Specifically, the convolution kernel in the sub-network Block of the ConvNeXt network is replaced by a variable convolution kernel, which can adjust the size of the dynamic receptive field in real time according to the image size, and perform channel attention weighting on the convolution result.

3. The small sample insulator defect detection method based on cross-domain feature fusion migration according to claim 2 is characterized in that: The local feature aggregation module includes a learnable discrete convolution module, which decomposes the continuous convolution kernel into a set of learnable discrete components by discretizing the convolution kernel parameter space and using dynamically generated weight coefficients α k Weighted combination to form the final convolution kernel W ds , satisfying the following expression: Among them, N represents the total number of discrete basis convolution kernels, α k is the dynamically generated weight coefficient, It indicates that the discrete basis convolution kernel is dynamically generated according to the input image features to achieve enhanced responsiveness to different image regions.

4. The small sample insulator defect detection method based on cross-domain feature fusion migration according to claim 1 is characterized in that: In step 3, a global feature aggregation module is constructed in the deep network. It is necessary to add a multi-head attention mechanism to the ConvNeXt backbone network. Specifically, an attention mechanism is added between the convolutional layer and the fully connected layer of the ConvNeXt network. The attention mechanism is based on the given input feature map F∈R H×W×C , where H, W, and C represent the height, width, and number of channels of the feature map, respectively. The query matrix Q, key matrix K, and value matrix V are generated through linear projection. Multiple heads are calculated in parallel, and then the self-attention calculation and weighting are performed through the decomposed matrix. The formula based on the scaled dot product attention mechanism is as follows: Among them, Attention(Q,K,V) refers to the calculation process of the attention mechanism; d k Refers to the dimension of the key matrix K, that is, the dimension length of each query matrix Q and key matrix K; Softmax refers to the scaled similarity matrix Normalized to probability distribution and used as attention weight; After obtaining the attention output, a residual connection is performed with the original input features to retain the original information and enhance the global semantic modeling capability.

5. The small sample insulator defect detection method based on cross-domain feature fusion migration according to claim 1 is characterized in that: In step 4, the introduced feature fusion module has the following steps: Step 4.1: Upsample the shallow local features through bilinear interpolation, and downsample the deep global features through pooling. Step 4.2: Concatenate the upsampled local features and the downsampled global features according to the channel dimension to form a multi-scale fusion feature; Step 4.3: Deep semantic features are gradually fused with shallow detail features through upsampling. Shallow detail features are then fused with deep semantic features through attention weighting to generate a unified multi-scale feature pyramid. Step 4.4: The fused multi-scale features are input into the global average pooling and fully connected layers.

6. The small sample insulator defect detection method based on cross-domain feature fusion migration according to claim 1 is characterized in that: In step 5, a layered freezing method is adopted, that is, during the model transfer training process, the convolution layer parameters are fixed, and only the pre-training parameters of the ConvNeXt Block and the fully connected layer are adjusted.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images