Insulator defect detection method based on feature extraction and cross-scale fusion

Abstract

The invention discloses an insulator defect detection method based on feature extraction and cross-scale fusion, and belongs to the technical field of insulator defect detection methods. According to the method, a defect detection model is obtained through training to find defects existing in a detected insulator image, and the method comprises the following steps: S1, inputting an image data set and a corresponding label data set; s2, designing a lightweight fast multi-scale extraction module to complete feature extraction; s3, designing an efficient feature fusion module to complete feature fusion; s4, selecting image features from the output of the cross-weighted bidirectional feature pyramid network through intersection-to-parallel ratio perception query selection; and finally, obtaining a confidence coefficient score and a bounding box of the target through a solution terminal, and further finishing decoding. The method effectively solves the problems that an existing detection method is insufficient in feature extraction capacity in the feature extraction stage and large in model parameter in the feature fusion stage.

Description

Technical Field

[0001] The present invention relates to the technical field of insulator defect detection methods, and specifically to an insulator defect detection method based on feature extraction and cross-scale fusion. Background Art

[0002] There are two main structures for insulator defect detection methods: convolutional neural network-based and transformer-based. The detection method based on convolutional neural network has developed from a two-stage method based on region-based convolutional neural network to a more concise and efficient one-stage method. The workflow of the one-stage algorithm usually includes four main steps: preprocessing, feature extraction, classification and positioning, and post-processing. First, the input image is subjected to necessary preprocessing operations, such as resizing and normalization, to meet the requirements of network input. Subsequently, the powerful feature extraction capability of deep convolutional neural network is used to abstract high-level feature representations from the image. On the feature map, the algorithm presets a series of anchor points, which serve as reference boxes to cover objects that may exist in the image. Each anchor point undergoes processing by two parallel branches: the classification branch is responsible for predicting whether the anchor point contains objects of a specific category, while the bounding box regression branch is responsible for adjusting the position and size of the anchor point to more accurately match the actual boundary of the object. Since the one-stage algorithm directly predicts on the entire feature map, a large number of candidate bounding boxes will be generated. In order to remove redundant and overlapping bounding boxes, the algorithm usually uses post-processing techniques such as non-maximum suppression to retain the bounding box with the highest score as the final detection result. Compared with the one-stage algorithm, the two-stage object detection algorithm goes further in the pursuit of detection accuracy. The workflow of the two-stage algorithm includes four steps: candidate region generation, feature extraction, classification and positioning, and post-processing. First, the region proposal method (such as selective search, region proposal network) is used to extract candidate regions that may contain objects from the image. These candidate regions are used as input for subsequent processing, greatly reducing the range of image regions that need to be processed. Next, feature extraction is performed on each candidate region. Similar to the one-stage algorithm, a convolutional neural network is also used here to extract feature representations of the candidate regions. Then, classifiers and bounding box regressors are applied to the extracted features to determine the category and precise location of the candidate regions, respectively. Finally, overlapping bounding boxes are removed through post-processing techniques such as non-maximum suppression to obtain the final detection results.

[0003] Unlike the object detection method based on convolutional neural network, the object detection method based on transformer cancels the non-maximum suppression processing and further develops into a real-time end-to-end object detection model. The object detection method based on transformer cleverly combines the advantages of convolutional neural network and transformer, providing a new perspective for the object detection task. First, the convolutional neural network acts as a feature extractor, which is responsible for extracting high-level feature representations from the input image. These feature maps not only contain rich spatial information, but also contain the underlying features of the image such as texture and color. Subsequently, these feature maps are fed into the encoder part of the transformer, which further mines the global dependencies between different positions in the feature map through the self-attention mechanism to enhance the model's overall understanding of the image content. In the decoder stage, the transformer-based object detector adopts a similar structure to the transformer in natural language processing, but the goal is to generate object detection results. The input of the decoder includes a series of learnable object queries, which can be understood as "questions" about potential objects in the image. The decoder combines the feature maps output by the encoder with these object queries through the cross-attention mechanism to gradually refine the category prediction and bounding box regression of each object. Finally, through one forward propagation, the transformer-based object detection method can directly output the detection results of all objects in the image without relying on complex post-processing steps like traditional methods.

[0004] However, the detection method based on convolutional neural network directly uses lightweight convolution and lightweight backbone network in the feature extraction stage and the feature fusion stage, which will reduce the detection accuracy of the model for insulator defects. The detection method based on transformer directly uses lightweight backbone network in the feature extraction stage and cannot meet the detection requirements of high accuracy. At the same time, the use of attention mechanism in the feature fusion stage will increase the parameters of the model and do not meet the lightweight requirements. Therefore, the detection method based on transformer requires a lightweight and efficient feature extraction network and a feature fusion solution that maintains a balance between performance and parameters. In view of this, the present invention proposes an insulator defect detection method based on feature extraction and cross-scale fusion to solve the above problems. Summary of the invention

[0005] 1. Technical Problems to be Solved by the Present Invention

[0006] The purpose of the present invention is to propose an insulator defect detection method based on feature extraction and cross-scale fusion to solve the problem raised in the background technology. The present invention designs a more efficient and lightweight feature extraction structure (fast multi-scale extraction module) to solve the problem of insufficient feature extraction capability of the detection method in the feature extraction stage; at the same time, a lighter fusion module (efficient feature fusion module) and a more efficient fusion network (cross-weighted bidirectional feature pyramid network) are designed to solve the problem of large model parameters in the feature fusion stage of the existing detection method.

[0007] 2. Technical solution

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] An insulator defect detection method based on feature extraction and cross-scale fusion is provided, wherein a defect detection model is obtained by training to find defects existing in the detected insulator image. The method comprises the following steps:

[0010] S1. Input image dataset and corresponding label dataset: given an insulator image dataset, the dataset includes original images and label files corresponding to the original images; splicing the original images in the dataset, and then scaling the spliced ​​images to adjust them to a uniform size; training and evaluating supervised learning tasks, using the annotation information in the existing label files, training a neural network model for identifying image defect types; using a set of preset anchor frames to predict targets of different sizes and aspect ratios, and in the model training process, adaptively adjusting the size of the anchor frames according to the actual size and aspect ratio of the target;

[0011] S2. Feature extraction: Design a lightweight and fast multi-scale extraction module to perform convolution and pooling operations on the input image, convert the original input image into a multi-layer feature map, extract feature representations with rich semantic information, and use the final output feature map to represent the location and category of insulator defects;

[0012] S3. Feature fusion: Design an efficient feature fusion module to combine feature maps at different levels and generate feature maps with multi-scale information to improve the accuracy of target detection. Specifically, it includes the following contents:

[0013] S3.1, the feature fusion process is divided into two parts: top-down and bottom-up;

[0014] S3.2, feature maps at different levels are fused together through upsampling and downsampling operations to generate a cross-weighted bidirectional feature pyramid network;

[0015] S3.3, fusing the feature maps of the top-down part and the bottom-up part to obtain a final feature map for insulator defect detection;

[0016] S4. Decoding: Image features are selected from the output of the cross-weighted bidirectional feature pyramid network through intersection-over-union-aware query selection; finally, the confidence score and bounding box of the target are obtained through the decoding head.

[0017] Preferably, S2 specifically includes the following contents:

[0018] S2.1, the fast multi-scale feature extraction module performs an initial lightweight convolution operation on the input image features through partial convolution. The partial convolution selects a subset of channels for convolution operation, and the remaining channels remain unchanged; the function of the output of the partial convolution is expressed as:

[0019] P out =G PConv (F x ,c p ,n)

[0020] Among them, F x represents the input feature; P out Represents the output after partial convolution; G PConv (·) represents a partial convolution operation; c p Indicates the channel where the convolution operation is actually performed, and the value of n is c p / C;

[0021] S2.2. After the partial convolution is completed, a multi-layer perceptron is used to learn more complex feature representations, and the generalization ability of the neural network model is improved through regularization technology;

[0022] S2.3. The ability to extract complex features of the neural network model is enhanced by a lightweight attention module. The function of the final output of the fast multi-scale extraction module is expressed as:

[0023] F out =G EMA (G DropPath (G MLP (P out )))

[0024] Among them, F out represents the features enhanced by the fast multi-scale extraction module; G MLP (·) represents a multilayer perceptron; G DropPath (·) represents the regularization technique; G EMA (·) denotes an efficient multi-scale attention module.

[0025] Preferably, the efficient feature fusion module includes an efficient semantic and detail fusion module, which is responsible for the fusion of features of more than two layers. The specific implementation process is as follows:

[0026] (1) Assuming that there are k input features of different sizes, the function of the enhanced fusion process of the efficient semantic and detail fusion module is expressed as:

[0027]

[0028] in, represents the kth input of the efficient semantic and detail fusion module; Multiply(·)

[0029] is element-by-element multiplication; G GhostConv (·) is a lightweight ghost convolution;

[0030] The enhanced fusion process is specifically expressed as follows: the efficient semantic and detail fusion module first unifies the size of k inputs to the feature size of the first input through the Resize(·) operation; for each feature of the same size after processing, the efficient semantic and detail fusion module performs a lightweight ghost convolution G GhostConv (·) Enhance the flow and fusion of semantic information; Finally, all features are fused through efficient element-by-element multiplication (·) to retain important spatial information and detail features;

[0031] (2) Let the output of the efficient semantic and detail fusion module be I out , the convolution layer completes the secondary feature enhancement and obtains the output result C out , the function of this process is expressed as:

[0032] C out =G SiLU (G BN (Conv1(I out )))

[0033] Among them, Conv1(·) represents the convolution operation with a convolution kernel size of 1×1; G BN (·) and G SiLU (·) denotes the normalization layer and SiLU activation function, respectively;

[0034] (3) Use a multi-branch convolution layer to complete the reparameterized convolution operation in the training phase; when performing reparameterized convolution in the inference phase, reparameterize the branch parameters to the main branch to form an equivalent single convolution layer to reduce memory consumption; C obtained in step (2) out After three re-parameterized convolution operations to continuously enhance key features, the output R is obtained. out , the function of this process is expressed as:

[0035] E out =G SiLU (G BN (Conv1(Add(C out , R out ))))

[0036] Among them, E out Indicates the enhanced feature output after the efficient feature fusion module fuses multiple layers of feature input; Add(F x , F x ) represents the element-wise addition of two features.

[0037] Preferably, the cross-weighted bidirectional feature pyramid network expands the top-down and bottom-up bidirectional paths into three-way paths, adds a path before the top-down path to preprocess the input, enhances the useful features of the input, connects the newly added path nodes to the nodes of the bottom-up path of the same layer, and strengthens the information transfer between adjacent paths; and adds a simple layer between two adjacent layers under the same path to reduce the information loss caused by cross-level fusion.

[0038] 3. Beneficial effects

[0039] (1) This paper proposes a new fast multi-scale extraction module, which combines the advantages of partial convolution and efficient multi-scale attention modules, reduces redundant convolution calculations in the backbone network, and greatly improves the feature extraction capability of the network.

[0040] (2) The present invention proposes an efficient feature fusion module to effectively fuse feature information at different levels. The efficient feature fusion module enhances the feature information of each input layer through a new efficient semantic and detail fusion module, effectively restoring the key detail features in the feature map. At the same time, the model reparameterization technology is used to reduce memory consumption and improve feature fusion efficiency.

[0041] (3) The present invention designs a new cross-weighted bidirectional feature pyramid network, which integrates more features through multiple weighted fusions, further improving the detection accuracy of the model.

[0042] (4) The present invention designs an efficient and lightweight insulator defect detection model, which realizes insulator defect detection in complex environments through a variety of efficient and lightweight designs. The detection model of the present invention has shown good insulator defect detection capabilities in a large number of experiments on open source synthetic fog insulator datasets and self-made insulator datasets.

[0043] (5) In order to solve the problem that the existing insulator datasets are of low quality and inconsistent with the real environment, the present invention collects a large number of insulator defect images in the field. After screening and labeling, the present invention constructs a dataset called "self-made insulator dataset" in the embodiment, which contains 2312 insulator images and annotates defect types such as breakage. The insulator images in the self-made insulator dataset have complex backgrounds, including trees, grass, farmland and towers. In addition, the sizes of insulator defects include large, medium and small, which makes the self-made insulator dataset highly diverse and representative. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is an overall framework diagram of the insulator defect detection method based on feature extraction and cross-scale fusion proposed in Example 1 of the present invention;

[0045] Figure 2This is a structural diagram of a fast multi-scale extraction module proposed in Example 2 of the present invention;

[0046] Figure 3 The cross-weighted bidirectional feature pyramid network proposed in Example 2 of the present invention is compared with the feature pyramid network and the path aggregation network, wherein (a) represents the feature pyramid network; (b) represents the path aggregation network; (c) represents the cross-weighted bidirectional feature pyramid network;

[0047] Figure 4 This is a structural diagram of an efficient feature fusion module proposed in Example 2 of the present invention;

[0048] Figure 5 These are graphs of the detection results on the open source synthetic fog insulator dataset and the self-made insulator dataset proposed in Example 2 of the present invention, wherein the pictures in the first and second rows are from the synthetic fog insulator dataset, and the pictures in the third and fourth rows are from the self-made insulator dataset. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0052] Embodiment 1:

[0053] The present invention proposes an insulator defect detection method based on feature extraction and cross-scale fusion. The model is obtained through training to find defects in the detected insulator image. The specific process is as follows:

[0054] (1) Input image dataset and corresponding label dataset: The given dataset contains the original images and the label files corresponding to them. First, in order to allow the model to process images of different sizes and channels, we will scale the spliced ​​images and adjust them to a uniform size of 640×640×3. Then, in order to predict objects of different scales and aspect ratios, a set of pre-set anchor boxes are used. During the training process, the model will adaptively adjust the size of the anchor box according to the actual size and aspect ratio of the target. Through the training and evaluation of supervised learning tasks, we can use the existing annotation information to train a neural network model that can accurately identify image defect categories.

[0055] (2) Feature extraction process: The feature extraction process is the core part of extracting image features. Its function is to convert the original input image into a multi-layer feature map for subsequent target detection tasks. The feature extraction process is mainly responsible for performing a series of convolution and pooling operations on the input image to extract feature representations with rich semantic information. For a 640×640×3 input image, after several convolution layers, normalization layers, and activation functions, a 32×32×512 feature map is obtained. We use the final output feature map to characterize information such as the location and category of insulator defects.

[0056] (3) Feature fusion process: Feature fusion mainly combines feature maps of different levels to generate feature maps with multi-scale information to improve the accuracy of target detection. The feature fusion process is divided into two parts: top-down and bottom-up. Feature maps of different levels are fused together through upsampling and downsampling operations to generate a cross-weighted bidirectional feature pyramid network. Finally, the feature maps of the top-down part and the bottom-up part are fused to obtain the final feature map for insulator defect detection.

[0057] (4) Decoding process: Image features are selected from the output of the cross-weighted bidirectional feature pyramid network through IoU-aware query selection. Finally, the confidence score and bounding box of the object are obtained through the decoding head.

[0058] The overall framework of the insulator defect detection method based on feature extraction and cross-scale fusion proposed in this invention is as follows: Figure 1As shown in the figure, the insulator image is first processed by the backbone network to extract its features. The features of the last four layers of the backbone network (layer 2, layer 3, layer 4 and layer 5) are input to the encoder. The two-dimensional features of layer 5 are converted into vectors and input into the attention-based intra-scale feature interaction module. The output of the attention-based intra-scale feature interaction module is then adjusted back to two dimensions and recorded as layer 6. The features of layers 2, 3, 4 and 6 are input into the cross-weighted bidirectional feature pyramid network for cross-scale fusion. Subsequently, image features are selected from the output of the cross-weighted bidirectional feature pyramid network through intersection-over-union-aware query selection. Finally, the confidence score and bounding box of the target are obtained through the decoding head. In the above process, the key structure includes three parts: (1) fast multi-scale extraction module; (2) cross-weighted bidirectional feature pyramid network; (3) efficient feature fusion module.

[0059] (1) Fast multi-scale extraction module

[0060] The fast multi-scale extraction module is a key component for efficient feature extraction in the backbone network, which is responsible for efficient extraction of complex features.

[0061] (2) Cross-weighted bidirectional feature pyramid network

[0062] The cross-weighted bidirectional feature pyramid network is the key structure in the encoder, which is responsible for processing and fusing the output of the backbone network.

[0063] (3) Efficient feature fusion module

[0064] The efficient feature fusion module is a key component of the cross-weighted bidirectional feature pyramid network, which is responsible for fusing more than two feature layers in the cross-weighted bidirectional feature pyramid network.

[0065] Embodiment 2:

[0066] Based on Example 1, but different in that, the insulator defect detection method based on feature extraction and cross-scale fusion proposed by the present invention is described below in conjunction with specific examples and drawings, and the specific contents are as follows.

[0067] Dataset

[0068] The present invention selects an open-source synthetic fog insulator dataset and a self-made insulator dataset for experiments. The synthetic fog insulator dataset contains 13718 insulator images. The self-made insulator dataset contains 2312 insulator images. The synthetic fog insulator dataset and the self-made insulator dataset are divided into a training set, a test set, and a validation set.

[0069] Training parameters

[0070] Hyperparameters are parameters that need to be set in advance during model training and inference, which affect the performance, speed, and stability of the model. By adjusting these parameters, the training and inference performance of the model can be optimized for different applications. The GPU used in the experiment is an NVIDIA A40, and the acceleration environment used is CUDA 12.2. During the training phase, the AdamW optimizer was used for training. In each set of experiments, the detection model was trained 300 times and the batch size was set to 4. At the same time, the input size of the model was set to 640×640. When training with the synthetic fog insulator dataset and the homemade insulator dataset, the initial learning rates were 0.0001 and 0.0005, respectively.

[0071] Evaluation indicators

[0072] In order to fully verify the performance of the present invention, six evaluation indicators are introduced in the present invention: accuracy, recall rate, mean average precision, parameter quantity, and floating point operation number.

[0073] There are two types of insulator images: normal images and images containing defects, which can be regarded as positive samples and negative samples respectively. Among them, true positive examples refer to the number of samples correctly classified as positive samples, false positive examples refer to the number of samples incorrectly classified as positive samples, and false negative examples refer to the number of samples incorrectly classified as negative samples.

[0074] Precision: represents how many of the results detected by the model as positive samples are truly positive samples. The higher the precision, the less likely the model is to mistakenly judge negative samples as positive samples.

[0075] Recall rate: represents the ratio of the number of positive samples correctly detected by the model to all true positive samples. The higher the recall rate, the better the model can capture the true target.

[0076] Average Average Precision: It combines the precisions at different confidence thresholds to get a comprehensive performance evaluation. The calculation of average average precision is based on the precision-recall curve, which is obtained by averaging the precision-recall curves of all categories.

[0077] Parameter quantity: indicates the number of trainable parameters in the model, including weights and biases. The more parameters there are, the more complex the model is and the stronger its representation ability is, but it may also increase the risk of overfitting.

[0078] Floating-point operations: indicates the number of floating-point operations performed by the model during a forward propagation process, which can reflect the computational complexity of the model.

[0079] Specific structure

[0080] (1) Fast multi-scale extraction module

[0081] like Figure 2As shown, in order to enhance feature extraction, the present invention designs a lightweight and efficient module, called the fast multi-scale extraction module. x ∈ H×W×C ,The fast multi-scale extraction module performs initial lightweight convolution operations through partial convolution, Figure 2 The specific operation process is shown in Figure 1. Partial convolution selects a subset of channels (such as 1 / 4 of the total number of channels) for convolution operation, while the remaining channels remain unchanged. The output of partial convolution is expressed mathematically as:

[0082] P out =G PConv (F x ,c p ,n)

[0083] Among them, let P out Represents the output after partial convolution, G PConv (·) indicates a partial convolution operation. p Indicates the channel where the convolution operation is actually performed, and the value of n is c p / C. The above operation allows partial convolution to reduce the amount of calculation to 1 / n of the original 2 Specifically, n is set to 4 in this example.

[0084] Then, the fast multi-scale extraction module uses a multi-layer perceptron after partial convolution to learn more complex feature representations. Subsequently, the generalization ability of the model is improved through regularization techniques. Finally, the fast multi-scale extraction module enhances the extraction ability of complex features through a lightweight attention module. The final output of the fast multi-scale extraction module can be described as:

[0085] F out =G EMA (G DropPath (G MLP (P out )))

[0086] Among them, F out represents the features enhanced by the fast multi-scale extraction module, G MLP (·) represents a multi-layer perceptron, G DropPath (·) is a regularization technique. G EMA (·) represents an efficient multi-scale attention module. For an input F x ∈ H×W×C, the efficient multi-scale attention module will divide it into g groups of feature maps of size H×W×(C / g). The input features of the efficient multi-scale attention module are divided into 8 groups, and then the spatial attention mechanism is applied within each group, so that the model can better focus on important spatial locations. Thanks to the grouping mechanism of the efficient multi-scale attention module, it can retain the information on each channel and reduce the computational overhead.

[0087] (2) Cross-weighted bidirectional feature pyramid network

[0088] In multi-scale feature fusion, the traditional feature pyramid network fuses features from top to bottom, resulting in significant loss of shallow feature information during propagation. The path aggregation network integrates bottom-up paths to enhance the feature pyramid network and promote the effective transmission of lower-level information. However, it does not effectively exploit the correlation between feature maps from all levels of the feature pyramid. To this end, a cross-weighted bidirectional feature pyramid network is designed to obtain richer feature information between different layers.

[0089] like Figure 3 As shown in the figure, unlike the path aggregation network, the cross-weighted bidirectional feature pyramid network expands the bidirectional path (top-down and bottom-up) into a similar three-way path. A path is added before the top-down path to preprocess the input and enhance the useful features of the input. Inspired by the residual structure, the present invention connects the newly added path nodes to the nodes of the bottom-up path of the same layer. This cross-connection method strengthens the information transfer between adjacent paths. The present invention not only considers the feature fusion between the same layer under different paths, but also adds a simple layer between two adjacent layers under the same path to reduce the information loss caused by cross-level fusion. Through the above design, the cross-weighted bidirectional feature pyramid network can effectively improve the overall detection accuracy of the model.

[0090] (3) Efficient feature fusion module

[0091] Each efficient feature fusion module is responsible for the fusion of more than two layers of features. All inputs of the efficient feature fusion module must first be enhanced and fused through the efficient semantic and detail fusion modules. Their specific structures are as follows: Figure 4 As shown in Figure 2, for k input features of different sizes, the enhanced fusion process of the efficient semantic and detail fusion module can be expressed as:

[0092] G ESDF (·)=Multiply(G GhostConv (Resize(F x i , F x i+1 , ..., F x i+n)))

[0093] Among them, F x i+k is the kth input of the efficient semantic and detail fusion module. The efficient semantic and detail fusion module first unifies the size of the k inputs to the feature size of the first input through the Resize(·) operation. For each feature of the same size after processing, the efficient semantic and detail fusion module performs a lightweight ghost convolution G GhostConv (·) enhances the flow and fusion of semantic information, ensuring that no additional parameters and calculations are added. Finally, the efficient element-by-element multiplication Multiply(·) is used to fuse all features and retain important spatial information and detail features.

[0094] Let the output of the efficient semantic and detail fusion module be I out Then, the convolutional layer completes the secondary feature enhancement and obtains the output result C out , described by the formula:

[0095] C out =G SiLU (G BN (Conv1(I out )))

[0096] Where Conv1(·) represents a convolution operation with a kernel size of 1×1. BN (·) and G SiLU (·) denote the normalization layer and SiLU activation function, respectively.

[0097] Another key strategy for efficient feature fusion is reparameterized convolution. Reparameterized convolution uses multi-branch convolution layers in the training phase, and reparameterizes the parameters of these branches to the main branch in the inference phase to form an equivalent single convolution layer, thereby reducing memory consumption. Reparameterized convolution can significantly improve the inference speed of the model while maintaining or improving the performance of the model. out After three re-parameterized convolution operations, the key features are continuously enhanced, and the output R is obtained. out , which is expressed by the formula:

[0098] R out =RepConv(RepConv(RepConv(C out )))

[0099] Among them, RepConv(·) represents the reparameterized convolution operation.

[0100] In the final stage of the efficient feature fusion module, feature information C at different levels is fused out and R out, thereby enhancing the model's ability to understand and represent the input image. The output E of the efficient feature fusion module out The formula is:

[0101] E out =G SiLU (G BN (Conv1(Add(C out , R out ))))

[0102] Among them, E out It is the enhanced feature output after the efficient feature fusion module fuses multiple layers of feature input. Add(F x , F x ) represents the element-by-element addition of two features. Through the efficient feature fusion module, the cross-weighted bidirectional feature pyramid network strengthens the representation of key features and ultimately achieves more detailed and reliable cross-level fusion.

[0103] Experimental Results

[0104] In order to verify the effectiveness of the present invention, experiments are conducted on synthetic fog insulator datasets and self-made insulator datasets. Figure 5 The specific detection results are shown. The present invention selects the insulator image with complex background in the synthetic fog insulator dataset ( Figure 5 The first and second rows in the figure) and the insulator images containing complex defects in the homemade insulator dataset ( Figure 5 The third and fourth rows in the figure). It can be seen from the detection results that, compared with the current mainstream technology, the present invention has higher detection accuracy for insulator images with complex backgrounds and complex defects. At the same time, the present invention has fewer parameters and is more conducive to deployment in resource-constrained drone inspection equipment.

[0105] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and improved concepts of the present invention within the technical scope disclosed by the present invention, and they should be covered by the protection scope of the present invention.

Claims

1. An insulator defect detection method based on feature extraction and cross-scale fusion, characterized in that: A defect detection model is obtained by training to find defects in the detected insulator image. The method comprises the following steps: S1. Input image dataset and corresponding label dataset: given an insulator image dataset, the dataset includes original images and label files corresponding to the original images; splicing the original images in the dataset, and then scaling the spliced ​​images to adjust them to a uniform size; training and evaluating supervised learning tasks, using the annotation information in the existing label files, training a neural network model for identifying image defect types; using a set of preset anchor frames to predict targets of different sizes and aspect ratios, and in the model training process, adaptively adjusting the size of the anchor frames according to the actual size and aspect ratio of the target; S2. Feature extraction: Design a lightweight and fast multi-scale extraction module to perform convolution and pooling operations on the input image, convert the original input image into a multi-layer feature map, extract feature representations with rich semantic information, and use the final output feature map to represent the location and category of insulator defects; S3. Feature fusion: Design an efficient feature fusion module to combine feature maps at different levels and generate feature maps with multi-scale information to improve the accuracy of target detection. Specifically, it includes the following contents: S3.1, the feature fusion process is divided into two parts: top-down and bottom-up; S3.2, feature maps at different levels are fused together through upsampling and downsampling operations to generate a cross-weighted bidirectional feature pyramid network; S3.3, fusing the feature maps of the top-down part and the bottom-up part to obtain a final feature map for insulator defect detection; S4. Decoding: Image features are selected from the output of the cross-weighted bidirectional feature pyramid network through intersection-over-union-aware query selection; finally, the confidence score and bounding box of the target are obtained through the decoding head.

2. The insulator defect detection method based on feature extraction and cross-scale fusion according to claim 1 is characterized in that: The S2 specifically includes the following contents: S2.1, the fast multi-scale feature extraction module performs an initial lightweight convolution operation on the input image features through partial convolution. The partial convolution selects a subset of channels for convolution operation, and the remaining channels remain unchanged; the function of the output of the partial convolution is expressed as: P out =G PConv (F x ,c p ,n) Among them, F x represents the input feature; P out Represents the output after partial convolution; G PConv (·) represents a partial convolution operation; c p Indicates the channel where the convolution operation is actually performed, and the value of n is c p / C; S2.

2. After the partial convolution is completed, a multi-layer perceptron is used to learn more complex feature representations, and the generalization ability of the neural network model is improved through regularization technology; S2.

3. The ability to extract complex features of the neural network model is enhanced by a lightweight attention module. The function of the final output of the fast multi-scale extraction module is expressed as: F out =G EMA (G DropPath (G MLP (P out ))) Among them, F out represents the features enhanced by the fast multi-scale extraction module; G MLP (·) represents a multilayer perceptron; G DropPath (·) represents the regularization technique; G EMA (·) denotes an efficient multi-scale attention module.

3. The insulator defect detection method based on feature extraction and cross-scale fusion according to claim 1 is characterized in that: The efficient feature fusion module includes an efficient semantic and detail fusion module, which is responsible for the fusion of features at more than two layers. The specific implementation process is as follows: (1) Assuming that there are k input features of different sizes, the function of the enhanced fusion process of the efficient semantic and detail fusion module is expressed as: Among them, F x i+k represents the kth input of the efficient semantic and detail fusion module; Multiply(·) is element-by-element multiplication; GGhostConv(·) is a lightweight ghost convolution; The enhanced fusion process is specifically expressed as follows: the efficient semantic and detail fusion module first unifies the size of k inputs to the feature size of the first input through the Resize(·) operation; for each feature of the same size after processing, the efficient semantic and detail fusion module performs a lightweight ghost convolution G GhostConv (·) Enhance the flow and fusion of semantic information; Finally, all features are fused through efficient element-by-element multiplication (·) to retain important spatial information and detail features; (2) Let the output of the efficient semantic and detail fusion module be I out , the convolution layer completes the secondary feature enhancement and obtains the output result C out , the function of this process is expressed as: C out =G SiLU (G BN (Conv1(I out ))) Among them, Conv1(·) represents the convolution operation with a convolution kernel size of 1×1; G BN (·) and G SiLU (·) denotes the normalization layer and SiLU activation function, respectively; (3) Use a multi-branch convolution layer to complete the reparameterized convolution operation in the training phase; when performing reparameterized convolution in the inference phase, reparameterize the branch parameters to the main branch to form an equivalent single convolution layer to reduce memory consumption; C obtained in step (2) out After three re-parameterized convolution operations to continuously enhance key features, the output R is obtained. out , the function of this process is expressed as: E out =G SiLU (G BN (Conv1(Add(C out ,R out )))) Among them, E out Indicates the enhanced feature output after the efficient feature fusion module fuses multiple layers of feature input; Add(F x , F x ) represents the element-wise addition of two features.

4. The insulator defect detection method based on feature extraction and cross-scale fusion according to claim 3 is characterized in that: The cross-weighted bidirectional feature pyramid network expands the top-down and bottom-up bidirectional paths into three-way paths, adds a path before the top-down path to pre-process the input, enhances the useful features of the input, connects the newly added path nodes to the nodes of the bottom-up path at the same layer, and strengthens the information transmission between adjacent paths; A simple layer is added between two adjacent layers in the same path to reduce the information loss caused by cross-level fusion.

Images