Power transmission line insulator multi-mode multi-type defect detection method based on MEWL-YOLO model

Abstract

The invention discloses a power transmission line insulator multi-modal multi-type defect detection method based on an MEWL-YOLO model. The method comprises the following steps: acquiring a multi-modal power transmission line data set; constructing an insulator multi-type defect detection model of the MEWL-YOLO model; the method specifically comprises the steps that on the basis of a YOLOv10n network framework, a multi-head attention mechanism and a gated convolution feedforward network are combined, a multi-head gated convolution feedforward network MGDB module is designed, and a C2K layer is optimized; a multi-scale efficient attention mechanism (EMA) is inserted into the Neck part of the YOLOv10n network; an Inner-Wise-MPDIOU loss function is adopted to replace a traditional loss function; a detection head in a YOLOv10n network is replaced by an LSCD lightweight detection head, and defect detection is carried out on the power transmission line insulator based on an insulator multi-type defect detection model of a constructed MEWL-YOLO model. According to the method, the MEWL-YOLO model is provided to realize defect detection of the multi-modal power transmission line insulator image, the detection effect of multi-modal and multi-type defects is improved, and the method has good practical application value.

Description

Technical Field

[0001] The present invention relates to the technical field of transmission line insulator detection, and in particular to a multi-modal and multi-type defect detection method for transmission line insulators based on a MEWL-YOLO model. Background Art

[0002] Transmission lines are a critical component of the power system, and their stability is crucial for the reliable transmission and supply of electricity. As transmission lines continue to operate for longer periods of time, insulators are susceptible to environmental factors, leading to damage, contamination, and even flashovers. These issues not only threaten the overall safety of transmission lines but can also lead to major power accidents. However, traditional insulator inspection methods, such as manual inspection and infrared camera inspection, suffer from a series of issues, including missed detections. Both manual and infrared camera inspections are commonly associated with missed detections during field target monitoring. Manual inspection relies on operator experience and attention, and is subject to factors such as fatigue and limited field of view, resulting in subjectivity, low efficiency, and poor consistency. While infrared cameras offer automated monitoring capabilities, their performance is susceptible to environmental impact. For example, they can miss distant, small, or stationary targets in high-temperature or low-contrast conditions. Furthermore, they are often limited by viewing angles, obstructions, and thermal imaging sensitivity, leading to detection failures.

[0003] YOLOv10 is currently the most advanced and widely used model, surpassing current mainstream object detectors in both detection speed and accuracy. Compared to the YOLOv8 algorithm, YOLOv10 introduces a training and inference method that does not require non-maximum suppression (NMS). It also employs a consistent dual allocation strategy, ensuring consistent training and inference processes, reducing computational overhead and latency. Furthermore, compared to YOLOv8, YOLOv10 features model architecture optimizations. First, by simplifying the design of the classification head, computational redundancy is reduced while maintaining high accuracy. Second, spatial reduction is decoupled from channel addition, reducing computational cost while retaining more information. Large kernel convolutions are introduced in deep networks to enhance the receptive field, combined with a partial self-attention (PSA) module to extract global features, improving detection capabilities without significantly increasing computational complexity. Finally, intrinsic rank analysis is used to identify redundant structures and replace them with more efficient modules (such as depthwise separable convolutions), optimizing the balance between model efficiency and accuracy.

[0004] While the YOLO family performs well when detecting large objects or key targets in a scene, it can suffer from inaccurate detection of small objects. This is particularly evident in the following aspects: Due to their small size and complex backgrounds, small objects typically occupy only a few pixels in an image, making them prone to missed detection and inaccurate detection using existing detection techniques. Traditional object detection algorithms, such as YOLO and Faster R-CNN, employ multiple downsampling techniques during feature extraction, which severely compresses key information about small objects, weakening their feature representation. Furthermore, the highly variable backgrounds of power transmission environments, such as the sky, mountains, forests, and buildings, can easily confuse insulators with the background, further complicating detection. Furthermore, existing models' anchor box matching strategies, loss function designs, and feature pyramid structures lack sufficient attention to small objects, further limiting detection performance. Some research has proposed enhancing small object recognition capabilities by improving feature fusion structures, introducing attention mechanisms, or adding shallow detection heads. For example, the paper "Improved YOLOv5 for Insulator Defect Detection in Complex Backgrounds" improves small object detection accuracy through the use of an SE module and deformable convolutions. Although there have been some improvements, challenges still exist in terms of model perception capabilities, data annotation quality and sample imbalance. There is an urgent need to further optimize the detection structure and build a dedicated dataset to improve the practicality and accuracy of small insulator target detection.

[0005] Existing insulator detection technologies still face problems such as missed detections, false detections, and low detection efficiency, especially when dealing with small targets. Due to the small size, complex features, and changing background environments of insulators, existing models often struggle to effectively extract the key features of small targets, resulting in frequent missed detections. Furthermore, insulators are similar in shape to wires and other objects, and the complex background makes false detections more likely, increasing the difficulty of detection. Although existing models such as YOLO have made some progress, they are computationally intensive and slow inference when processing high-resolution images and large-scale data, making them difficult to meet the needs of real-time monitoring. To this end, researchers have proposed methods such as multi-scale detection, feature fusion, and attention mechanisms to improve the accuracy of small target detection, while also introducing lightweight models to enhance detection efficiency. Despite this, further optimization of the model's performance in complex backgrounds and improvements in detection accuracy and real-time performance are still needed. Summary of the Invention

[0006] In order to solve the problems of insulator detection in the existing technology, such as missed detection, false detection, low detection efficiency, complex small target features, and small scale, the present invention proposes a multi-modal and multi-type defect detection method and system for transmission line insulators based on the MEWL-YOLO model. The method proposes the MEWL-YOLO model to realize defect detection on multi-modal transmission line insulator images, which not only improves the detection effect of multi-modal and multi-type defects, but also has good practical application value.

[0007] The technical solution adopted by the present invention is:

[0008] The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model includes the following steps:

[0009] Step 1: Obtain a multimodal transmission line dataset;

[0010] Step 2: Construct a MEWL-YOLO model for detecting multiple types of insulator defects. This includes:

[0011] 2.1: Based on the YOLOv10n network framework, this model combines the multi-head attention mechanism and the gated convolutional feedforward network to design the multi-head gated convolutional feedforward network MGDB module and optimize the C2K layer;

[0012] In order to filter effective features through the gating mechanism and combine deep convolution to strengthen local modeling, the GDFN module is referenced.

[0013] 2.2: Insert the multi-scale efficient attention mechanism EMA into the Neck part of the YOLOv10n network;

[0014] 2.3: Use Inner-Wise-MPDIOU loss function to replace the traditional loss function;

[0015] 2.4: Finally, the LSCD lightweight detection head is used to replace the built-in detection head in the YOLOv10n network.

[0016] Step 3: Based on the insulator multi-type defect detection model of the MEWL-YOLO model constructed in step 2, defects of transmission line insulators are detected.

[0017] In step 1, the multimodal transmission line dataset consists of multiple high-definition drone inspection images, infrared images, and ultraviolet discharge images, labeled with flashover, defect, and insulator. The labeled dataset is randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0018] In 2.1, the MDTA module is introduced, as follows:

[0019] Let the input be Among them, X is input; H is height; W is width; C is channel; Represents a three-dimensional tensor;

[0020] First, perform layer normalization on the input:

[0021] X norm =LayerNorm(X) (1);

[0022] In formula (1), X norm Represents the result after normalization of X; LayerNorm(X) represents the normalization of X; normalization is performed along the channel dimension:

[0023]

[0024] In formula (2), μ c represents the mean of channel c; X i,j,c Represents the result after normalization and scaling offset; i represents the horizontal coordinate in the feature map; j represents the vertical coordinate in the feature map.

[0025]

[0026] In formula (3), σ c represents the variance of channel c.

[0027]

[0028] In formula (4), X norm,i,j,c represents the output of the i, j, c position after normalization, scaling and offset; γ c represents the scaling parameter; β c represents the offset parameter; ∈ is a small constant.

[0029] The number of input channels is expanded to C′ through 1×1 convolution:

[0030]

[0031] In formula (5), X proj Represents the result after the input feature undergoes 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution; X norm Represents the normalized input features; It means that the convolution kernel projects the number of channels from C to C′.

[0032] Extract local spatial features through 3×3 depth-wise separable convolution (Dconv):

[0033]

[0034] In formula (6), Q represents Query; DWConv 3×3 (X proj ) represents a 3×3 depth-wise separable convolution operation; Represents the spatial structure of the feature map;

[0035]

[0036] In formula (7), K represents Key;

[0037]

[0038] In formula (8), V represents Value.

[0039] Reshape Q, K into The shape of a tensor represents R samples, each sample has C′ channels, each channel has a size of H×W, and then the spatial dimension is flattened to one dimension;

[0040] Q reshape =Reshape(Q),K reshape =Reshape(K)(9); in formula (9), Q reshape Reshape(Q) represents the reshaping result of Q; Reshape(Q) represents the reshaping of Q; K reshape Reshape(K) represents the reshaping result of K.

[0041] Calculate the attention weight of the channel dimension and apply the attention weight to the value V:

[0042]

[0043] In formula (10), A represents the attention matrix of the channel dimension; Indicates that the convolution kernel projects the number of channels from C to C′; represents the transposed query matrix; Indicates that the channel attention weight is calculated by scaling the dot product.

[0044]

[0045] In formula (11), V reshape Reshape(V) represents the reshaping result of V. Represents a two-bit tensor; Output attn Represents the weighted output.

[0046] Reshape the output to the original spatial dimensions:

[0047]

[0048] In formula (12), Output reshape Represents the reshaped tensor, the dimension is the same as the input image; Reshape(Output attn ) indicates adjusting the attention output to the spatial dimension of the original image.

[0049] Finally, the number of channels is restored and added to the original input, retaining the low-frequency information to obtain the output:

[0050]

[0051] X out =X+Output final (14);

[0052] In the above formula, Output final Represents the output of the reshaped tensor after 1×1 convolution; W 1×1′ Represents the weight matrix of the 1×1 convolution kernel; X out Represents the features after adding attention adjustment output.

[0053] Reference the GDFN module as follows:

[0054] Let the input be First, perform layer normalization on the input:

[0055] X norm =LayerNorm(X);

[0056] The input is then passed through a 1×1 convolution, mapping the input from C channels to C′ channels:

[0057] X conv1 =W 1×1 *X norm (15);

[0058] In formula (15), X conv1 represents the output of Xnorm after 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution, Represents a three-dimensional tensor; * represents a convolution operation;

[0059] Extract local spatial features through a 3×3 depth-wise separable convolution to reduce parameters:

[0060] X dconv,c =W 3×3,c *X conv1,c ,c=1,2,...,C′ (16);

[0061] In formula (16), Xdconv,c Represents X through a 3×3 convolution kernel conv1,c The result of the convolution operation; W 3×3,c Represents a 3×3 depth-wise separable convolution; The convolution kernel for each channel; X conv1,c represents the result of Xconv1 of c channels; c represents the index of different channels; C′ represents the feature dimension.

[0062] Through 1×1 convolution again, the feature dimension is restored from C′ to C, which is consistent with the input dimension;

[0063] X conv2 =W 1×1′ *X dconv (17);

[0064] In formula (17), X conv2 Represents X dconv The output result after 1×1 convolution; W 1×1′ represents the weight matrix of 1×1 convolution; X dconv Represents the result of convolution operation on Xconv1 through a 3×3 convolution kernel.

[0065] Add the module output to the original input, preserving low-frequency information:

[0066] X′=X+X conv2 (18);

[0067] In formula (18), X′ represents the information retained after the module output is added to the original input.

[0068] The multi-head gated convolutional feedforward network MGDB module is obtained by fusing the MDTA module and the GDFN module;

[0069] Let the input be First, perform layer normalization on the input to obtain X1:

[0070] X1=LayerNorm1(X)(19);

[0071] In formula (19), X1 represents the result of normalization processing on X; LayerNorm1(X) represents the operation of normalization processing on X.

[0072] Then, the output X1 passes through the MDTA module and is added to the input X, and the output result is layer normalized to obtain X2:

[0073] X2=LayerNorm2(X+MDTA(X1))(20);

[0074] In formula (20), X2 represents the output result; MDTA(X1) represents the output result after X1 passes through the MDTA module; LayerNorm2(X+MDTA(X1)) represents the normalization processing of the output result.

[0075] After the output X2 passes through the GDFN module, it is added to X and X1 to obtain the final output

[0076] X out =X+X1+GDFN(X2)(21);

[0077] In formula (21), X out Represents the output result; GDFN(X2) represents the output result of X2 through the GDFN module.

[0078] In 2.2, let the input be B represents the batch size; Represents a 4D tensor. First, the input is grouped according to the channel dimension and pooled vertically and horizontally.

[0079] X group =Reshape(X,(B·G,C / G,H,W))(22);

[0080] In formula (22), X group Indicates grouping by channel dimension and pooling results in vertical and horizontal directions;

[0081] (B·G,C / G,H,W) represents the reshaped input data X; G represents the number of groups; C represents the number of channels; C / G represents the number of channels in each group; Reshape(X,(B·G,C / G,H,W)) groups the input X by the channel dimension and reshapes it into (B,G,C / G,H,W).

[0082]

[0083] In formula (23), X h Indicates the result of the average pooling operation on Xgroup by height; AvgPool height (X group ) represents the average pooling operation of Xgroup by height; Represents a 4D tensor with width 1.

[0084]

[0085] In formula (24), X w Indicates the result of the average pooling operation on Xgroup by height; AvgPool width (Xgroup ) indicates that the average pooling operation is performed on Xgroup by height.

[0086] Splice the features from two directions and pass the features through a 1×1 convolution to obtain the spatial attention weight A spatial :

[0087]

[0088] In formula (25), H represents the result of the generated fusion features; It means that features of different dimensions are combined and 1x1 convolution is used to generate fused features; Indicates that two features are spliced along the channel direction; X h Represents the feature projection of the image in high dimensions; Represents the transposed features of the image in wide dimensions; Represents a 4D tensor.

[0089]

[0090] In formula (26), Y represents the output result of restoring the spatial dimension of the input feature after weighting the feature map by group; Reshape(X group ⊙σ(W)) indicates that the weighted result is transformed into a standard four-dimensional tensor; ⊙ indicates element-by-element multiplication; σ(W) indicates the weight matrix after the activation function (such as Siqmoid); Represents a 4D tensor.

[0091] X′ h ,X′ w =Split(H,[H,W],dim=2) (27);

[0092] In formula (27), X h ′ represents the attention feature along the H direction; X′ w Represents the attention feature along the w direction; Split(H,[H,W],dim=2) means dividing H into two parts; [H,W] represents the spatial dimension of the feature map; dim means operating on a certain dimension.

[0093] A spatial =σ(GroupNorm(X group ⊙X′ h ⊙(X′ w ) T )) (28);

[0094] In formula (28), A spatial represents the spatial attention map; σ(GroupNorm(X group ⊙X h′⊙(X′ w ) T )) means that each group of features is normalized and then activated by the function; GroupNorm(X group ⊙X h ′⊙(X′ w ) T ) means normalizing each set of features; (X′ w ) T Represents the transposed spatial attention features.

[0095] By matrix multiplication, we establish global-local feature associations, generate channel attention weights, apply the attention weights to the group features, and restore the output dimension to obtain Y:

[0096]

[0097] In formula (29), Q1 represents the channel attention vector; Flatten(AvgPool(X1)) represents the flattened vector after global average pooling of the feature map; AvgPool(X1) represents global average pooling of the feature map; X1 represents the input feature map; Represents a three-dimensional tensor.

[0098]

[0099] In formula (30), K1 represents the flattened local feature map; Flatten(X1) represents flattening (straightening) the tensor X1 into a two-dimensional tensor; Represents a three-dimensional tensor.

[0100]

[0101] In formula (31), Q2 represents the query vector Query of the second set of attention; Flatten(AvgPool(X2)) represents flattening into a two-dimensional tensor; AvgPool(X2) represents global average pooling of the spatial dimension (HxW) of X2; X2 represents the second set of input features.

[0102]

[0103] In formula (32), K2 represents the key of the second group of attention; Flatten(X2) means flattening the X2 spatial dimension;

[0104]

[0105] In formula (33), W represents the final spatial attention weight map, which is used to weight the input feature map; Indicates normalizing the similarity of the first feature map to a probability distribution; Transpose the key vector representing the first special graph; Indicates normalizing the similarity of the second feature map to a probability distribution; Transpose the key vector representing the second special graph; Represents a 4D tensor.

[0106]

[0107] In formula (34), Y represents the final spatial attention feature map, which has weighted features to enhance spatial sensitivity; Reshape(X group ⊙σ(W)) means restoring the weighted result to the standard image tensor shape; σ(W) means performing the activation function operation on the tensor W.

[0108] In 2.3, Inner-Wise-MPDIOU is introduced to replace CIOU. The calculation formula of Inner-Wise-MPDIOU loss function is as follows

[0109]

[0110] In formula (35), L Wise-Inner-MPDIoU represents the Inner-Wise-MPDIOU loss function; IoU is the Intersection over Union; d inner is the Euclidean distance between the feature center point inside the target box and the feature center point inside the prediction box; d center is the Euclidean distance between the center of the target box and the center of the prediction box; w union , h union The width and height of the bounding rectangle of the target box and the predicted box, respectively. λ1 and λ2 are hyperparameters used to adjust the weight of the loss term.

[0111] In 2.4, after the LSCD lightweight detection head extracts the feature map, it first compresses the channel through a 1×1 convolution; then, it extracts features through two 3×3 convolutions with shared parameters to enhance spatial perception; then, it decouples the classification CLS and positioning REG tasks to improve detection accuracy; finally, it performs dynamic scale adjustment to adaptively adjust the output weights of feature maps of different scales to optimize multi-scale detection effects.

[0112] The present invention provides a multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model, and the technical effects are as follows:

[0113] 1) The present invention effectively enhances the model's ability to detect multiple types of defects (such as cracks, flashovers, contamination, etc.) on transmission line insulators;

[0114] 2) This invention simultaneously improves its practicality and deployment efficiency in complex scenarios. The final improved model achieved a P of 91.27% and a mAP@0.5 of 75.87%. This represents a 4.6% improvement over the original model, with fewer parameters and a lower computational load. This model outperforms YOLOv10n and YOLOv8n in terms of computational load, parameter count, and model size, while also achieving high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] The present invention will be further described below with reference to the accompanying drawings and examples:

[0116] Figure 1 This is a flow chart of insulator detection according to the present invention.

[0117] Figure 2 This is the structural diagram of the MDTA (Multi-Dconv Head Transposed Attention) module introduced in the present invention.

[0118] Figure 3 The present invention uses the GDFN (Gated-Dconv Feed-Forward Network) module structure diagram to screen effective features through a gating mechanism.

[0119] Figure 4 This is the network structure diagram of the MEWL-YOLO model of the present invention.

[0120] Figure 5 The network structure diagram of the multi-scale efficient attention mechanism (EMA) inserted in the Neck part of the present invention.

[0121] Figure 6 This is the network structure diagram of the LSCD lightweight detection head of the present invention.

[0122] Figure 7 This is a partial image after data enhancement is implemented in the present invention. DETAILED DESCRIPTION

[0123] A multi-type defect detection method for insulators based on the MEWL-YOLO model. This method, based on the YOLOv10n framework, introduces a multi-head gated convolutional feedforward network (MGDB) module, combining a multi-head attention mechanism with a gated convolutional feedforward network to optimize the C2K layer. The MGDB module can simultaneously capture global features and process local features, enhancing feature extraction capabilities. A multi-scale efficient attention mechanism (EMA) is introduced in the neck layer to improve the detection capabilities of transmission line insulator defects under complex backgrounds. Furthermore, the Inner-Wise-MPDIOU loss function replaces the traditional loss function to more accurately measure the match between the predicted box and the target box, optimizing training results and detection accuracy. Finally, the LSCD lightweight detection head replaces the built-in YOLOv10n detection head, reducing computational effort and parameter count while maintaining detection performance. This method not only improves the detection performance of multimodal and multi-type defects but also demonstrates promising practical application value.

[0124] The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model includes the following steps:

[0125] Step 1: Obtain a multimodal transmission line dataset;

[0126] Step 2: Construct a MEWL-YOLO model for detecting multiple types of insulator defects. This includes:

[0127] 2.1: Improve the C2f network module to capture global context and fuse local features while reducing computational complexity. The MDTA (Multi-Dconv Head Transposed Attention) module is introduced:

[0128] Based on the YOLOv10n network framework, this model combines a multi-head attention mechanism with a gated convolutional feedforward network to design a multi-head gated convolutional feedforward network (MGDB) module and optimizes the C2K layer. The MGDB module captures global features through a multi-head attention mechanism and uses a gated convolutional feedforward network to process local features, thereby enhancing the model's feature extraction capabilities.

[0129] In order to filter effective features through the gating mechanism and combine deep convolution to enhance local modeling, the GDFN (Gated-Dconv Feed-Forward Network) module is referenced.

[0130] 2.2: Adding the EMA attention mechanism. When collecting multimodal transmission line insulator defect images, the acquired images are complex due to the influence of external conditions such as weather and lighting. Furthermore, the detection of small transmission line insulator defects is challenging for the detection model used. Therefore, a multi-scale efficient attention mechanism (EMA) is inserted into the Neck portion of the YOLOv10n network to enhance the positional features of transmission line insulators and their defects from the feature extraction network. This reduces the interference of irrelevant information in complex environments and improves the detection capability of transmission line insulator defects in complex backgrounds.

[0131] 2.3: Improve the detection model's regression accuracy for bounding box position and shape by using the Inner-Wise-MPDIOU loss function instead of the traditional loss function. This loss function is designed specifically for optimizing bounding box regression in target detection. It combines the traditional IOU with the distance constraint of bounding box key points to optimize the regression loss of the target detection box. It can more comprehensively measure the matching degree and distance relationship between the predicted box and the target box, thereby improving the model's training effect and detection accuracy.

[0132] 2.4: Finally, the LSCD lightweight detection head is used to replace the built-in detection head in the YOLOv10n network. This reduces the amount of model calculation and parameters without reducing the model detection accuracy, thus achieving model lightweighting.

[0133] Step 3: Based on the insulator multi-type defect detection model of the MEWL-YOLO model constructed in step 2, defects of transmission line insulators are detected.

[0134] In step 1, the multimodal transmission line dataset consists of 1,000 high-definition drone inspection images, 1,000 infrared images, and 1,000 ultraviolet discharge images, labeled as pollution flashover, broken, and insulator. The labeled dataset is randomly divided into training, validation, and test sets in an 8:1:1 ratio, with the training set consisting of 2,400 images and the validation set consisting of 300 images.

[0135] In step 1, the insulator image is input into the insulator multi-type defect detection model of the pre-trained MEWL-YOLO model and the detection results are output.

[0136] In 2.1, the MDTA (Multi-Dconv Head Transposed Attention) module is introduced as follows: Assume that the input is Among them, X is input; H is height; W is width; C is channel; Represents a three-dimensional tensor;

[0137] First, perform layer normalization on the input:

[0138] X norm =LayerNorm(X) (1);

[0139] In formula (1), X norm Represents the result after normalization of X; LayerNorm(X) represents the normalization of X; normalization is performed along the channel dimension:

[0140]

[0141] In formula (2), μ c represents the mean of channel c; X i,j,c Represents the result after normalization and scaling offset; i represents the horizontal coordinate in the feature map; j represents the vertical coordinate in the feature map.

[0142]

[0143] In formula (3), σ c represents the variance of channel c.

[0144]

[0145] In formula (4), X norm,i,j,c represents the output of the i, j, c position after normalization, scaling and offset; γ c represents the scaling parameter; β c represents the offset parameter; ∈ is a small constant.

[0146] The number of input channels is expanded to C′ through 1×1 convolution:

[0147]

[0148] In formula (5), X proj Represents the result after the input feature undergoes 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution; X norm Represents the normalized input features; It means that the convolution kernel projects the number of channels from C to C′.

[0149] Extract local spatial features through 3×3 depth-wise separable convolution (Dconv):

[0150]

[0151] In formula (6), Q represents Query; DWConv 3×3 (Xproj ) represents a 3×3 depth-wise separable convolution operation; Represents the spatial structure of the feature map;

[0152]

[0153] In formula (7), K represents Key;

[0154]

[0155] In formula (8), V represents Value.

[0156] Reshape Q, K into The shape of a tensor represents R samples, each sample has C′ channels, each channel has a size of H×W, and then the spatial dimension is flattened to one dimension;

[0157] Q reshape =Reshape(Q),K reshape =Reshape(K)(9); in formula (9), Q reshape Reshape(Q) represents the reshaping result of Q; Reshape(Q) represents the reshaping of Q; K reshape Reshape(K) represents the reshaping result of K.

[0158] Calculate the attention weight of the channel dimension and apply the attention weight to the value V:

[0159]

[0160] In formula (10), A represents the attention matrix of the channel dimension; Indicates that the convolution kernel projects the number of channels from C to C′; represents the transposed query matrix; Indicates that the channel attention weight is calculated by scaling the dot product.

[0161]

[0162] In formula (11), V reshape Reshape(V) represents the reshaping result of V. Represents a two-bit tensor; Output attn Represents the weighted output.

[0163] Reshape the output to the original spatial dimensions:

[0164]

[0165] In formula (12), Output reshapeRepresents the reshaped tensor, the dimension is the same as the input image; Reshape(Output attn ) indicates adjusting the attention output to the spatial dimension of the original image.

[0166] Finally, the number of channels is restored and added to the original input, retaining the low-frequency information to obtain the output:

[0167]

[0168] X out =X+Output final (14);

[0169] In the above formula, Output final Represents the output of the reshaped tensor after 1×1 convolution; W 1×1′ Represents the weight matrix of the 1×1 convolution kernel; X out Represents the features after adding attention adjustment output.

[0170] Reference the GDFN (Gated-Dconv Feed-Forward Network) module, as follows:

[0171] Let the input be First, perform layer normalization on the input:

[0172] X norm =LayerNorm(X);

[0173] The input is then passed through a 1×1 convolution, mapping the input from C channels to C′ channels:

[0174] X conv1 =W 1×1 *X norm (15);

[0175] In formula (15), X conv1 represents the output of Xnorm after 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution, Represents a three-dimensional tensor; * represents a convolution operation;

[0176] Extract local spatial features through a 3×3 depth-wise separable convolution (Dconv) to reduce parameters:

[0177] X dconv,c =W 3×3,c *X conv1,c ,c=1,2,...,C′ (16);

[0178] In formula (16), X dconv,cRepresents X through a 3×3 convolution kernel conv1,c The result of the convolution operation; W 3×3,c Represents a 3×3 depth-wise separable convolution; The convolution kernel for each channel; X conv1,c represents the result of Xconv1 of c channels; c represents the index of different channels; C′ represents the feature dimension.

[0179] Through 1×1 convolution again, the feature dimension is restored from C′ to C, which is consistent with the input dimension;

[0180] X conv2 =W 1×1′ *X dconv (17);

[0181] In formula (17), X conv2 Represents X dconv The output result after 1×1 convolution; W 1×1′ represents the weight matrix of 1×1 convolution; X dconv Represents the result of convolution operation on Xconv1 through a 3×3 convolution kernel.

[0182] Add the module output to the original input, preserving low-frequency information:

[0183] X′=X+X conv2 (18);

[0184] In formula (18), X′ represents the information retained after the module output is added to the original input.

[0185] The GDFN module design achieves the goals of lightweighting, enhancing multi-scale feature fusion and training stability by implementing the above steps. The present invention obtains a multi-head gated convolutional feedforward network (MGDB) module by fusing the MDTA module and the GDFN module;

[0186] Let the input be First, perform layer normalization on the input to obtain X1:

[0187] X1=LayerNorm1(X)(19);

[0188] In formula (19), X1 represents the result of normalization processing on X; LayerNorm1(X) represents the operation of normalization processing on X.

[0189] Then, the output X1 passes through the MDTA module and is added to the input X, and the output result is layer normalized to obtain X2:

[0190] X2=LayerNorm2(X+MDTA(X1))(20);

[0191] In formula (20), X2 represents the output result; MDTA(X1) represents the output result after X1 passes through the MDTA module; LayerNorm2(X+MDTA(X1)) represents the normalization processing of the output result.

[0192] After the output X2 passes through the GDFN module, it is added to X and X1 to obtain the final output

[0193] X out =X+X1+GDFN(X2)(21);

[0194] In formula (21), X out Represents the output result; GDFN(X2) represents the output result of X2 through the GDFN module.

[0195] In 2.2, the GDFN (Gated-Dconv Feed-Forward Network) module is referenced as follows:

[0196] Let the input be B represents the batch size; Represents a 4D tensor.

[0197] First, the input is grouped according to the channel dimension and pooled vertically and horizontally;

[0198] X group =Reshape(X,(B·G,C / G,H,W))(22);

[0199] In formula (22), X group Indicates grouping by channel dimension and pooling results in vertical and horizontal directions;

[0200] (B·G,C / G,H,W) represents the reshaped input data X; G represents the number of groups; C represents the number of channels; C / G represents the number of channels in each group; Reshape(X,(B·G,C / G,H,W)) groups the input X by the channel dimension and reshapes it into (B,G,C / G,H,W).

[0201]

[0202] In formula (23), X h Indicates the result of the average pooling operation on Xgroup by height; AvgPool height (X group ) represents the average pooling operation of Xgroup by height; Represents a 4D tensor with width 1.

[0203]

[0204] In formula (24), X w Indicates the result of the average pooling operation on Xgroup by height; AvgPool width (X group ) indicates that the average pooling operation is performed on Xgroup by height.

[0205] Splice the features from two directions and pass the features through a 1×1 convolution to obtain the spatial attention weight A spatial :

[0206]

[0207] In formula (25), H represents the result of the generated fusion features; It means that features of different dimensions are combined and 1x1 convolution is used to generate fused features; Indicates that two features are spliced along the channel direction; X h Represents the feature projection of the image in high dimensions; Represents the transposed features of the image in wide dimensions; Represents a 4D tensor.

[0208]

[0209] In formula (26), Y represents the output result of restoring the spatial dimension of the input feature after weighting the feature map by group; Reshape(X group ⊙σ(W)) indicates that the weighted result is transformed into a standard four-dimensional tensor; ⊙ indicates element-by-element multiplication; σ(W) indicates the weight matrix after the activation function (such as Siqmoid); Represents a 4D tensor.

[0210] X′ h ,X′ w =Split(H,[H,W],dim=2) (27);

[0211] In formula (27), X h ′ represents the attention feature along the H direction; X′ w Represents the attention feature along the w direction; Split(H,[H,W],dim=2) means dividing H into two parts; [H,W] represents the spatial dimension of the feature map; dim means operating on a certain dimension.

[0212] A spatial =σ(GroupNorm(X group ⊙X′ h ⊙(X′ w) T )) (28);

[0213] In formula (28), A spatial represents the spatial attention map; σ(GroupNorm(X group ⊙X′ h ⊙(X′ w ) T )) means that each group of features is normalized and then activated by the function; GroupNorm(X group ⊙X′ h ⊙(X′ w ) T ) means normalizing each set of features; (X′ w ) T Represents the transposed spatial attention features.

[0214] By matrix multiplication, we establish global-local feature associations, generate channel attention weights, apply the attention weights to the group features, and restore the output dimension to obtain Y:

[0215]

[0216] In formula (29), Q1 represents the channel attention vector; Flatten(AvgPool(X1)) represents the flattened vector after global average pooling of the feature map; AvgPool(X1) represents global average pooling of the feature map; X1 represents the input feature map; Represents a three-dimensional tensor.

[0217]

[0218] In formula (30), K1 represents the flattened local feature map; Flatten(X1) represents flattening (straightening) the tensor X1 into a two-dimensional tensor; Represents a three-dimensional tensor.

[0219]

[0220] In formula (31), Q2 represents the query vector Query of the second set of attention; Flatten(AvgPool(X2)) represents flattening into a two-dimensional tensor; AvgPool(X2) represents global average pooling of the spatial dimension (HxW) of X2; X2 represents the second set of input features.

[0221]

[0222] In formula (32), K2 represents the key of the second group of attention; Flatten(X2) means flattening the X2 spatial dimension;

[0223]

[0224] In formula (33), W represents the final spatial attention weight map, which is used to weight the input feature map; Indicates normalizing the similarity of the first feature map to a probability distribution; Transpose the key vector representing the first special graph; Indicates normalizing the similarity of the second feature map to a probability distribution; Transpose the key vector representing the second special graph; Represents a 4D tensor.

[0225]

[0226] In formula (34), Y represents the final spatial attention feature map, which has weighted features to enhance spatial sensitivity; Reshape(X group ⊙σ(W)) means restoring the weighted result to the standard image tensor shape; σ(W) means performing the activation function operation on the tensor W.

[0227] The EMA attention mechanism captures long-range spatial dependencies through horizontal and vertical pooling respectively, 3x3 convolution enhances the details of transmission line insulator defects, Sigmoid gating is used to suppress complex background interference, and cross-space learning is used to enhance the feature fusion of multimodal defect images of transmission line insulators.

[0228] Inner-Wise-MPDIOU is introduced to replace CIOU. The calculation formula of Inner-Wise-MPDIOU loss function is as follows:

[0229]

[0230] In formula (35), L Wise-Inner-MPDIoU represents the Inner-Wise-MPDIOU loss function; IoU is the Intersection over Union; d inner is the Euclidean distance between the feature center point inside the target box and the feature center point inside the prediction box; d center is the Euclidean distance between the center of the target box and the center of the prediction box; w union , h union The width and height of the bounding rectangle of the target box and the predicted box, respectively. λ1 and λ2 are hyperparameters used to adjust the weight of the loss term.

[0231] In 2.4, after the LSCD lightweight detection head extracts the feature map,

[0232] First, the channel is compressed by a 1×1 convolution (Conv_GN, 1×1);

[0233] Reduce the number of model parameters and computational complexity, and improve computational efficiency.

[0234] Then, two 3×3 convolutions with shared parameters (Conv_GN, 3×3) are used to extract features and enhance spatial perception;

[0235] Through two 3×3 convolution operations with shared parameters, the network can capture more spatial features in a larger receptive field. This helps capture the structural information of objects in the image, and the high-dimensional information of the features can be gradually extracted and refined, allowing the network to extract richer and more useful features at different scales.

[0236] Afterwards, the classification (CLS) and localization (REG) tasks are decoupled to improve detection accuracy;

[0237] The localization branch can focus more on predicting the target position and size, reducing errors and improving the accuracy of bounding box regression. The decoupled classification branch can make better use of contextual information and improve category differentiation capabilities.

[0238] Finally, dynamic scale adjustment is performed to adaptively adjust the output weights of feature maps of different scales to optimize the multi-scale detection effect. Figure 6 shown.

[0239] The traditional fixed weight method easily overlooks some key information, while the dynamic weight mechanism can adaptively focus on significant features at different scales, reduce the loss of key information, and ensure that the model's detection capabilities at different scales are more balanced.

[0240] This experiment uses PyTorch as the framework, Python 3.8 as the programming language, and GPU training. The training hardware environment includes an Intel(R) Xeon(R) Silver 4214R processor, an NVIDIA RTX 3080Ti GPU, and a Linux operating system. PyTorch version 1.10.0 and CUDA version 11.3 are used. Experimental parameter settings include 240 epochs, 40 batch sizes, SGD optimizer, amp set to Ture, initial learning rate of 0.01, momentum parameter of 0.937, and weight decay of 0.0005.

[0241] The experimental evaluation indicators used in this paper are precision (P), average precision at IOU 0.5 (map@0.5), parameter count (Para), and computational complexity (GFLOPs / G).

[0242] Figure 1This paper demonstrates the simple process for detecting insulator defects using this method. First, a multimodal insulator dataset was obtained: a multimodal transmission line dataset consisting of 3,000 high-definition drone inspection images, along with their infrared and UV discharge images, labeled with pollution-flashover, broken, and insulator. The labeled dataset was randomly divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The training set consisted of 2,400 images, and the validation set consisted of 300 images.

[0243] The collected dataset is enhanced through data enhancement techniques such as inversion to obtain a multimodal dataset.

[0244] Figure 7 This is a partial image after data enhancement implemented in the present invention. Figure 7 Defects in the insulator can be seen under visible and ultraviolet light conditions.

[0245] The data set is input into the insulator multi-type defect detection model of the EWL-YOLO model to obtain the detection results.

[0246] Figure 2 and Figure 3 To improve C2f, the present invention introduces the MDTA (Multi-Dconv Head Transposed Attention) module to capture global context and fuse local features while reducing computational complexity. It also uses the GDFN (Gated-Dconv Feed-Forward Network) module to filter effective features through a gating mechanism.

[0247] The MGDB module, through the collaborative design of channel attention and gated convolution, effectively implements global context modeling in target detection tasks, enhances the detection efficiency of small and occluded targets, and optimizes linear complexity to meet real-time requirements and is compatible with mainstream detection frameworks and edge devices. Therefore, the present invention uses the MAGB module to replace the Bottleneck portion of the C2f module in the YOLOv10n network to obtain C2f_MGDB. Subsequently, by replacing the C2f module in the Head portion of the YOLOv10n model with C2f_MGDB, the detection accuracy of multimodal transmission line insulator defect images is improved.

[0248] Figure 5The paper demonstrates the insertion of a multi-scale efficient attention mechanism (EMA) into the Neck part of the YOLOv10n network. This attention mechanism enhances the positional features of transmission line insulators and their defects from the feature extraction network, thereby reducing the interference of irrelevant information in complex environments and improving the model's ability to distinguish transmission line insulators and their defects.

[0249] In the YOLOv10n model, the above improvements may lead to a significant increase in the model's computational complexity and parameter count. To reduce the model's computational complexity and parameter count without reducing detection accuracy, the YOLOv10n built-in detection head is replaced with the LSCD lightweight detection head.

[0250] After the LSCD model extracts the feature map, it compresses the channels through a 1×1 convolution (Conv_GN, 1×1). It then extracts features through two 3×3 convolutions (Conv_GN, 3×3) with shared parameters to enhance spatial perception. It then decouples the classification (CLS) and localization (REG) tasks to improve detection accuracy. Finally, dynamic rescaling (scaling) is performed to adaptively adjust the output weights of feature maps at different scales to optimize multi-scale detection.

[0251] Although the model performance has improved to a certain extent after the above improvements, problems still exist, such as sensitivity to the aspect ratio penalty term and optimization stagnation caused by gradient saturation. Therefore, this paper introduces Inner-Wise-MPDIOU to replace CIOU. This loss function is a loss function designed specifically for optimizing bounding box regression in object detection. It combines the constraints of the distance between key points of the bounding box on the basis of traditional IOU and optimizes the regression loss of the object detection box. It aims to improve the detection model's regression accuracy for the bounding box position and shape, while improving the regression ability for objects with mismatched aspect ratios.

[0252] The calculation formula of the Inner-Wise-MPDIOU loss function is as follows:

[0253]

[0254] In order to verify the effectiveness of the improved module on the model improvement effect, the present invention designed an ablation experiment based on YOLOv10n. The ablation experiment is shown in Table 1:

[0255] Table 1. Ablation experiment data based on YOLOv10n

[0256]

[0257] The first set of experiments replaced the C2f module in the Head with C2f_MGDB. The second set of experiments inserted the EMA multi-scale efficient attention mechanism into the Neck. The third set of experiments replaced the built-in detection head of YOLOv10 with the LSCD lightweight detection head. The fourth set of experiments replaced the built-in CIOU in YOLOv10n with Inner-Wise-MPDIOU. Finally, experiments were conducted using permutations and combinations of these improvements.

[0258] In order to further test the detection performance of the improved algorithm, the mainstream algorithms SSD, Faster-RCNN, YOLOv3-tiny,

[0259] YOLOv5n, YOLOv8n, YOLOv9n, YOLOv10n were compared with the improved algorithm. The comparison results are shown in Table 2:

[0260] Table 2 Algorithm comparison experiment table

[0261]

[0262] This paper makes four key optimizations to the target detection model, significantly improving the performance and efficiency of transmission line insulator defect detection:

[0263] First, by introducing a multi-granularity dense block (MGDB) to replace the original C2f module, the model's ability to locate and classify targets in multimodal data (such as images and infrared images) is enhanced. This enables the model to more accurately identify subtle features of insulator defects in complex environments (such as weather changes or light interference).

[0264] Secondly, an exponential moving average (EMA) attention module is introduced in the Neck layer of the model to dynamically screen and enhance the characteristics of the target area, while suppressing irrelevant background information, effectively reducing the impact of interference factors such as messy wires and vegetation on the detection results in complex power transmission scenarios.

[0265] At the same time, in order to reduce the number of model parameters and improve computational efficiency, a lightweight multi-scale feature fusion (LSCD) detection head is used to replace the original detection head to simplify the calculation process and retain key information.

[0266] Finally, the CIOU loss is replaced with an improved Multi-Point Position Intersection-Over-Union (MPDIOU) loss, forcing the model to fit the target shape and edges more accurately. This significantly improves the regression accuracy of defect boundaries such as insulator damage and cracks.

[0267] Finally, the MEWL-YOLO model-based multi-modal and multi-type defect detection model for transmission line insulators has improved performance compared to the original model YOLOv10, as shown in Table 3:

[0268] Table 3 Performance comparison between MEWL-YOLO model and YOLOv10n model

[0269]

[0270]

[0271] The improved MEWL-YOLO model proposed in the present invention effectively improves the detection accuracy in image defect detection of transmission line insulators and is more suitable for multimodal defect detection of transmission line insulators.

Claims

1. A multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model, characterized by The following steps are involved: Step 1: Obtain a multimodal transmission line dataset; Step 2: Construct a MEWL-YOLO model for detecting multiple types of insulator defects. This includes: 2.1: Based on the YOLOv10n network framework, this model combines the multi-head attention mechanism and the gated convolutional feedforward network to design the multi-head gated convolutional feedforward network MGDB module and optimize the C2K layer; 2.2: Insert the multi-scale efficient attention mechanism EMA into the Neck part of the YOLOv10n network; 2.3: Use Inner-Wise-MPDIOU loss function to replace the traditional loss function; 2.4: Finally, the LSCD lightweight detection head is used to replace the built-in detection head in the YOLOv10n network. Step 3: Based on the insulator multi-type defect detection model of the MEWL-YOLO model constructed in step 2, defects of transmission line insulators are detected.

2. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 1 is characterized by: In step 1, the multimodal transmission line data set consists of multiple high-definition drone inspection images, multiple infrared images, and multiple ultraviolet discharge images, with labels such as flashover, defect, and insulator; The labeled dataset is randomly divided into training set, validation set and test set in the ratio of 8:1:

1.

3. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 1 is characterized by: In 2.1, the MDTA module and the GDFN module are integrated to obtain the multi-head gated convolutional feedforward network MGDB module; Let the input be First, perform layer normalization on the input to obtain X1: X1=LayerNorm1(X)(19); In formula (19), X1 represents the result of normalization processing of X; LayerNorm1(X) represents the operation of normalization processing of X; Then, the output X1 passes through the MDTA module and is added to the input X, and the output result is layer normalized to obtain X2: X2=LayerNorm2(X+MDTA(X1))(20); In formula (20), X2 represents the output result; MDTA(X1) represents the output result after X1 passes through the MDTA module; LayerNorm2(X+MDTA(X1)) represents the normalization processing of the output result; After the output X2 passes through the GDFN module, it is added to X and X1 to obtain the final output X out =X+X1+GDFN(X2)(21); In formula (21), X out Represents the output result; GDFN(X2) represents the output result of X2 through the GDFN module.

4. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 3 is characterized by: The MDTA module is introduced as follows: Let the input be Among them, X is input; H is height; W is width; C is channel; Represents a three-dimensional tensor; First, perform layer normalization on the input: X norm =LayerNorm(X)(1); In formula (1), X norm Represents the result after normalization of X; LayerNorm(X) represents the normalization of X; normalization is performed along the channel dimension: In formula (2), μ c represents the mean of channel c; X i,j,c Represents the result after normalization and scaling offset; i represents the horizontal coordinate in the feature map; j represents the vertical coordinate in the feature map; In formula (3), σ c represents the variance of channel c; In formula (4), X norm,i,j,c represents the output of the i, j, c position after normalization, scaling and offset; γ c represents the scaling parameter; β c represents the offset parameter; ∈ is a small constant; The number of input channels is expanded to C′ through 1×1 convolution: X proj =W 1×1 ·X norm , In formula (5), X proj Represents the result after the input feature undergoes 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution; X norm Represents the normalized input features; Indicates that the convolution kernel projects the number of channels from C to C′; Extract local spatial features through 3×3 depth-wise separable convolution: Q=DWConv 3×3 (X proj ), In formula (6), Q represents Query; DWConv 3×3 (X proj ) represents a 3×3 depth-wise separable convolution operation; Represents the spatial structure of the feature map; K=DWConv 3×3 (X proj ), In formula (7), K represents Key; V=DWConv 3×3 (X proj ), In formula (8), V represents Value; Reshape Q, K into The shape of a tensor represents R samples, each sample has C′ channels, each channel has a size of H×W, and then the spatial dimension is flattened to one dimension; Q reshape =Reshape(Q),K reshape =Reshape(K)(9); In formula (9), Q reshape Reshape(Q) represents the reshaping result of Q; Reshape(Q) represents the reshaping of Q; K reshape Reshape(K) represents the reshaping result of K.

5. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 4 is characterized by: Calculate the attention weight of the channel dimension and apply the attention weight to the value V: In formula (10), A represents the attention matrix of the channel dimension; Indicates that the convolution kernel projects the number of channels from C to C′; represents the transposed query matrix; Indicates that the channel attention weight is calculated by scaling the dot product; Output attn =A·V reshape (11); In formula (11), V reshape Reshape(V) represents the reshaping result of V. Represents a two-bit tensor; Output attn represents the weighted output; Reshape the output to the original spatial dimensions: In formula (12), Output reshape Reshape(Output attn ) indicates adjusting the attention output to the spatial dimension of the original image; Finally, the number of channels is restored and added to the original input, retaining the low-frequency information to obtain the output: Output final =W 1×1′ ·Output reshape , X out =X+Output final (14); In the above formula, Output final Represents the output of the reshaped tensor after 1×1 convolution; W 1×1′ represents the weight matrix of the 1×1 convolution kernel; X out Represents the features after adding attention adjustment output.

6. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 3 is characterized by: Reference the GDFN module as follows: Let the input be First, perform layer normalization on the input: X norm =LayerNorm(X); The input is then passed through a 1×1 convolution, mapping the input from C channels to C′ channels: X conv1 =W 1×1 *X norm (15); In formula (15), X conv1 represents the output of Xnorm after 1×1 convolution; W 1×1 represents the weight matrix of 1×1 convolution, Represents a three-dimensional tensor; * represents a convolution operation; Extract local spatial features through a 3×3 depth-wise separable convolution to reduce parameters: X dconv,c =W 3×3,c *X conv1,c ,c=1,2,...,C′ (16); In formula (16), X dconv,c Represents X through a 3×3 convolution kernel conv1,c The result of the convolution operation; W 3×3,c Represents a 3×3 depth-wise separable convolution; The convolution kernel for each channel; X conv1,c represents the result of Xconv1 of c channels; c represents the index of different channels; C′ represents the feature dimension; Through 1×1 convolution again, the feature dimension is restored from C′ to C, which is consistent with the input dimension; X conv2 =W 1×1′ *X dconv (17); In formula (17), X conv2 Represents X dconv The output result after 1×1 convolution; W 1×1′ represents the weight matrix of 1×1 convolution; X dconv Represents the result of convolution operation on Xconv1 through a 3×3 convolution kernel; Add the module output to the original input, preserving low-frequency information: X′=X+X conv2 (18); In formula (18), X′ represents the information retained after the module output is added to the original input.

7. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 1 is characterized by: In 2.2, let the input be B represents the batch size; Represents a four-dimensional tensor; First, the input is grouped according to the channel dimension and pooled vertically and horizontally; X group =Reshape(X,(B·G,C / G,H,W))(22); In formula (22), X group Indicates grouping by channel dimension and pooling results in vertical and horizontal directions; (B·G,C / G,H,W) represents the reshaped shape of the input data X; G represents the number of groups; C represents the number of channels; C / G represents the number of channels in each group; Reshape(X,(B·G,C / G,H,W)) represents grouping the input X according to the channel dimension and reshaping it into the shape (B,G,C / G,H,W); In formula (23), X h Indicates the result of the average pooling operation on Xgroup by height; AvgPool height (X group ) represents the average pooling operation of Xgroup by height; Represents a four-dimensional tensor with a width of 1; In formula (24), X w Indicates the result of the average pooling operation on Xgroup by height; AvgPool width (X group ) represents the average pooling operation of Xgroup by height; Splice the features from two directions and pass the features through a 1×1 convolution to obtain the spatial attention weight A spatial : In formula (25), H represents the result of the generated fusion features; It means that features of different dimensions are combined and 1x1 convolution is used to generate fused features; Indicates that two features are spliced along the channel direction; X h Represents the feature projection of the image in high dimensions; Represents the transposed features of the image in wide dimensions; Represents a four-dimensional tensor; In formula (26), Y represents the output result of restoring the spatial dimension of the input feature after weighting the feature map by group; Reshape(X group ⊙σ(W)) indicates that the weighted result is transformed into a standard four-dimensional tensor; ⊙ indicates element-by-element multiplication; σ(W) indicates the weight matrix after the activation function (such as Siqmoid); Represents a four-dimensional tensor; X h ′,X′ w =Split(H,[H,W],dim=2)(27); In formula (27), X h ′ represents the attention feature along the H direction; X′ w represents the attention feature along the w direction; Split(H,[H,W],dim=2) means dividing H into two parts; [H,W] represents the spatial dimension of the special graph; dim means operating on a certain dimension; A spatial =σ(GroupNorm(X group ⊙X h ′⊙(X′ w ) T ))(28); In formula (28), A spatial represents the spatial attention map; σ(GroupNorm(X group ⊙X h ′⊙(X′ w ) T )) means that each group of features is normalized and then activated by the function; GroupNorm(X group ⊙X h ′⊙(X′ w ) T ) indicates normalization of each set of features; (X′ w ) T Represents the transposed spatial attention features.

8. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 7 is characterized by: By matrix multiplication, we establish global-local feature associations, generate channel attention weights, apply the attention weights to the group features, and restore the output dimension to obtain Y: In formula (29), Q1 represents the channel attention vector; Flatten(AvgPool(X1)) represents the flattened vector after global average pooling of the feature map; AvgPool(X1) represents global average pooling of the feature map; X1 represents the input feature map; Represents a three-dimensional tensor; In formula (30), K1 represents the flattened local feature map; Flatten(X1) represents flattening (straightening) the tensor X1 into a two-dimensional tensor; Represents a three-dimensional tensor; In formula (31), Q2 represents the query vector Query of the second set of attention; Flatten(AvgPool(X2)) represents flattening into a two-dimensional tensor; AvgPool(X2) represents global average pooling of the spatial dimension (HxW) of X2; X2 represents the second set of input features; In formula (32), K2 represents the key of the second group of attention; Flatten(X2) means flattening the X2 spatial dimension; In formula (33), W represents the final spatial attention weight map, which is used to weight the input feature map; Indicates normalizing the similarity of the first feature map to a probability distribution; Transpose the key vector representing the first special graph; Indicates normalizing the similarity of the second feature map to a probability distribution; Transpose the key vector representing the second special graph; Represents a four-dimensional tensor; In formula (34), Y represents the final spatial attention feature map, which has weighted features that enhance spatial sensitivity; Reshape(X group ⊙σ(W)) means restoring the weighted result to the standard image tensor shape; σ(W) means performing the activation function operation on the tensor W.

9. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 1 is characterized by: In 2.3, Inner-Wise-MPDIOU is introduced to replace CIOU. The calculation formula of Inner-Wise-MPDIOU loss function is as follows In formula (35), L Wise-Inner-MPDIoU represents the Inner-Wise-MPDIOU loss function; IoU is the Intersection over Union; d inner is the Euclidean distance between the feature center point inside the target box and the feature center point inside the prediction box; d center is the Euclidean distance between the center of the target box and the center of the prediction box; w union , h union The width and height of the bounding rectangle of the target box and the prediction box respectively; λ1 and λ2 are hyperparameters used to adjust the weight of the loss term.

10. The multi-modal and multi-type defect detection method for transmission line insulators based on the MEWL-YOLO model according to claim 1, characterized in that: In 2.4, after the LSCD lightweight detection head extracts the feature map, it first compresses the channel through a 1×1 convolution; then, it extracts features through two 3×3 convolutions with shared parameters to enhance spatial perception; then, it decouples the classification CLS and positioning REG tasks to improve detection accuracy; finally, it performs dynamic scale adjustment to adaptively adjust the output weights of feature maps of different scales to optimize multi-scale detection effects.

Images