A lightweight target detection system integrating attention mechanism, optimizing feature fusion method and self-supervised learning

Abstract

The application relates to the technical field of computer vision, and particularly discloses a lightweight target detection model integrating an attention mechanism, an optimized feature fusion method and self-supervised learning, which comprises a backbone network, a feature fusion module, a detection head, a lightweight module and a self-supervised learning module; the model is improved on the basis of YOLOv7-tiny, adopts FasterNet as the backbone network, and integrates a convolution block attention module (CBAM) after each convolution layer; the feature fusion module (Neck) introduces a feature pyramid network (FPN) structure to optimize the multi-scale feature fusion effect; in addition, the backbone network is pre-trained through a self-supervised learning method (such as SimCLR); and the model is lightweight through channel pruning and channel distillation technology. The lightweight target detection model integrating the attention mechanism, the optimized feature fusion method and the self-supervised learning has good practical application value while maintaining high detection precision and significantly reducing model parameters and inference time.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a lightweight object detection system that integrates an attention mechanism, an optimized feature fusion method, and self-supervised learning. Background Technology

[0002] With the development of the power industry, the length and complexity of transmission lines are constantly increasing. Insulators, as key components, bear the dual functions of electrical insulation and mechanical support. However, insulators are exposed to harsh environments for extended periods, making them susceptible to natural factors such as lightning strikes, pollution, and extreme cold, leading to insulator defects such as insulator damage and flashover. These defects can cause transmission lines to come into contact with towers or other lines, resulting in power outages or even large-scale power failures. Therefore, regular inspection of insulators on transmission lines is crucial.

[0003] Traditional insulator inspection mainly relies on manual inspection, which suffers from low efficiency, high cost, and poor safety. With the development of UAV (Unmanned Aerial Vehicle) technology, it has become possible to use UAVs for image acquisition of insulators in high-voltage transmission lines. However, the wide-angle and variable perspective of UAV images present challenges such as small target detection and complex backgrounds, making it difficult to meet practical needs in terms of detection accuracy and efficiency. Deep learning, especially target detection algorithms, has become a research hotspot in this context, but existing methods still suffer from low detection accuracy and high model complexity. Therefore, developing an efficient, lightweight, and high-accuracy insulator defect detection system is of significant practical importance. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight target detection system that integrates an attention mechanism, an optimized feature fusion method, and self-supervised learning, specifically designed for efficient and accurate detection of defects in power insulators. By improving upon YOLOv7-tiny, it integrates a CBAM attention module and an FPN feature fusion structure, and employs self-supervised learning for backbone network pre-training. Furthermore, it combines channel pruning and channel distillation techniques to achieve a lightweight and efficient model suitable for deployment on edge devices such as drones, meeting the requirements for real-time detection.

[0005] To achieve the above objectives, this invention provides a lightweight target detection system that integrates an attention mechanism, an optimized feature fusion method, and self-supervised learning, including a backbone network, a feature fusion module (Neck), a detection head, a lightweight module, and a self-supervised learning module.

[0006] The backbone network is based on the FasterNet architecture, and the Convolutional Block Attention (CBAM) module is integrated into some of the PConv and PWConv layers in the architecture to enhance the attention capabilities of channel and spatial features.

[0007] The feature fusion module introduces the Feature Pyramid Network (FPN) structure and combines it with the OD-SlimNeck module to optimize the fusion effect of multi-scale features;

[0008] The detection head uses a decoupled detection head, which includes independent classification and localization branches, used to handle the prediction of the target category and the prediction of the target location, respectively.

[0009] The lightweight module reduces the number of model parameters through channel pruning and channel distillation techniques;

[0010] The self-supervised learning module uses a contrastive learning method to pre-train the backbone network.

[0011] Preferably, the CBAM module includes two sub-modules: channel attention and spatial attention, which are used to enhance the channel information and spatial information of the feature map, respectively. The mathematical expression for the channel attention mechanism is as follows:

[0012] M c =σ(W2·δ(W1·AvgPool(X))+W2·δ(W1·MaxPool(X)));

[0013] Among them, M c X represents the channel attention map, with a size of (C, 1, 1) indicating the importance of each channel; X represents the input feature map, with a size of (C, H, W), where C is the number of channels, H is the height, and W is the width; AvgPool(·) represents global average pooling of the input feature map in the spatial dimension, with an output size of (C, 1, 1); MaxPool(·) represents global max pooling of the input feature map in the spatial dimension, with an output size of (C, 1, 1); W1 and W2 are learnable weight matrices, corresponding to the weights of the fully connected layer and convolutional layer of the channel attention, respectively, determining the weighting method in the channel dimension; δ(·) is the ReLU activation function; σ(·) is the Sigmoid function, with an output range of (0, 1);

[0014]

[0015] Among them, Y c This represents the feature output map after channel attention weighting, and its size is the same as X. For element-wise multiplication, M... cAfter being broadcast to the same spatial dimension as X, it is multiplied with X channel by channel;

[0016] The mathematical expression for spatial attention mechanism is:

[0017] M s =σ(Conv 7×7 ([AvgPool(Y c MaxPool(Y) c )]));

[0018] Among them, AvgPool(Y c ) indicates that Y is in the channel dimension c Perform a pixel-wise averaging operation, with an output size of (1, H, W), MaxPool(Y) c ) represents the relationship between Y and the channel dimension. c Perform a pixel-wise maximum value operation, outputting a size of (1, H, W); [AvgPool(Y c MaxPool(Y) c [] indicates that the average pooling result and the max pooling result are concatenated along the channel dimension, with a size of (2, H, W); Conv 7×7 (·) represents a 7×7 convolution operation used to extract spatial attention information from the concatenated features; M s The inter-attention map is of size (1, H, W), representing the importance of each location in the spatial dimension;

[0019] The mathematical expression for the overall attention mechanism Y:

[0020]

[0021] Where Y represents the final output feature map after applying channel attention and spatial attention, with a size of (C, H, W).

[0022] Preferably, the feature fusion module combines multi-scale feature maps and utilizes global and local features to enhance the representational power of the feature maps. The FPN structure fuses feature maps of different scales layer by layer through a top-down path and lateral connections. The top-down path performs feature fusion as follows:

[0023]

[0024] in, For fusion features, obtain them in level l; P l This is the original feature map. This indicates the feature map that has been fused in the previous layer; UpSample(·) indicates an upsampling operation, which increases the resolution of the high-level features and then adds them element-wise to the low-level features. l represents the index of the feature map, and L represents the index of the highest layer.

[0025] Horizontal connection: for each After adjusting the number of channels using a 1×1 convolution, it is compared with the original P. l Add elements one by one.

[0026] Preferably, the channel pruning technique of the lightweight module reduces the number of model parameters and computational overhead by evaluating the scaling factor of the Batch Normalization layer and pruning channels with scaling factors close to zero.

[0027] The lightweight module's channel distillation technique uses the original YOLOv7 model as the teacher model and performs channel distillation on the pruned student model. Knowledge transfer is then performed on samples correctly predicted by the teacher model, and the distillation loss function is used.

[0028]

[0029] Among them, w ij For weights; used to measure the contribution of different channels or different samples to the loss; z i and z j To extract enhanced feature representations of the two views using FasterNet, τ is a temperature parameter that controls the degree of softening distribution; z i and z j This represents the feature vector obtained from different augmented views of the same image, typically with a size of (d, 1) or (1, d), sim(z) i ,z j ) represents the similarity function between vectors, defined as cosine similarity; 2N is the total number of sample views, N samples, each sample generates two views; l [k≠i] This is an indicator function that takes the value 1 when k ≠ i and 0 otherwise, used to exclude the influence of the same view itself; and These are the channel outputs for the student model and the teacher model, respectively; ||·|| refers to the Euclidean distance or other norm distance, in this case, the square of the L2 norm. ||·|| 2 ; To mitigate channel distillation loss, student model performance is improved by minimizing the difference between the student and teacher model outputs;

[0030] Overall loss function for

[0031]

[0032] Where λ and β are weight parameters that control the contribution of each loss to the overall training; The supervised loss for the object detection task itself, such as bounding box regression loss and classification loss; This is the self-supervised contrastive loss part of SimCLR; This refers to the channel distillation loss.

[0033] Preferably, the self-supervised learning module generates different views by applying random cropping, color jittering, and rotation data augmentation operations to the input image, wherein the self-supervised contrastive loss function SimCLR is calculated as follows:

[0034]

[0035] The self-supervised contrastive loss function is used to optimize the feature representation of the backbone network, so that different views of the same image are close together in the feature space, and features of different images are far apart.

[0036] Preferably, the lightweight module also includes a model fine-tuning step, which is used to fine-tune the pruned model.

[0037] Preferably, the decoupled detection head adopts a hybrid channel strategy, partially sharing weights, for classification prediction and localization prediction.

[0038] An application method for a lightweight object detection system integrating attention mechanisms, optimized feature fusion methods, and self-supervised learning includes the following steps:

[0039] S1. Data Acquisition: Use drones to collect images of electrical insulators on high-voltage transmission lines;

[0040] S2. Data preprocessing: Standardize the acquired images, including cropping, scaling, and enhancement;

[0041] S3. Model pre-training: The backbone network is pre-trained on large-scale unlabeled data using a self-supervised learning method;

[0042] S4. Model Integration: Integrate the pre-trained backbone network with the feature fusion module and the decoupled detection head to build a complete target detection model;

[0043] S5. Model Training: Supervised training of the integrated model using labeled data to optimize detection performance;

[0044] S6. Model Compression: By using channel pruning and channel distillation techniques, the number of model parameters is reduced, thus achieving model lightweighting.

[0045] S7. Model Deployment: Deploy the compressed model onto the drone to achieve real-time power insulator defect detection.

[0046] Preferably, in S5, a hybrid loss function is used during training, combining supervised detection loss and self-supervised contrastive loss.

[0047] Preferably, in S6, a global pruning threshold is set, all channels with scaling factors below the threshold are pruned, and model performance is restored by fine-tuning the training.

[0048] Therefore, the lightweight object detection system of the present invention, which integrates the attention mechanism, optimizes the feature fusion method, and employs self-supervised learning, has the following beneficial effects:

[0049] (1) This invention improves the model’s ability to detect defects in small targets and complex backgrounds by integrating the CBAM attention module and the FPN feature fusion structure.

[0050] (2) This invention significantly reduces the number of model parameters and computational overhead through channel pruning and channel distillation techniques, making it suitable for real-time deployment on edge devices.

[0051] (3) This invention improves the feature representation capability of the backbone network through self-supervised learning, thereby enhancing the generalization performance of the model.

[0052] (4) The lightweight model in this invention maintains high detection accuracy while significantly shortening the inference time, thus meeting the requirements of real-time detection. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall architecture of a lightweight object detection system embodiment that integrates attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention;

[0054] Figure 2 This is a schematic diagram of the FasterNet convolutional block structure of the integrated CBAM attention module in an embodiment of a lightweight object detection system that integrates attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention.

[0055] Figure 3 This is a schematic diagram of the FPN feature fusion structure of an embodiment of a lightweight target detection system integrating attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention;

[0056] Figure 4 This is a schematic diagram of the pre-training process of self-supervised learning (SimCLR) in an embodiment of a lightweight object detection system that integrates attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention.

[0057] Figure 5 This is a schematic diagram of the channel pruning process of an embodiment of a lightweight target detection system integrating attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention;

[0058] Figure 6 This is a schematic diagram of the channel distillation process of an embodiment of a lightweight target detection system integrating attention mechanism, optimized feature fusion method and self-supervised learning according to the present invention. Detailed Implementation

[0059] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0060] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0061] Example

[0062] like Figure 1 As shown, a lightweight object detection system integrating attention mechanism, optimized feature fusion method, and self-supervised learning includes a backbone network, a feature fusion module, a detection head, a lightweight module, and a self-supervised learning module. The specific model implementation is as follows:

[0063] 1. Model Architecture Design

[0064] The target detection model of this invention is improved based on YOLOv7-tiny as follows:

[0065] Backbone: Replaced with the FasterNet architecture, and integrated with a Convolutional Block Attention Module (CBAM) after each PConv and PWConv convolutional layer to enhance the attention capabilities for channel and spatial features.

[0066] Neck section: Introducing the Feature Pyramid Network (FPN) structure, combined with the OD-SlimNeck module, to optimize the fusion effect of multi-scale features and improve the accuracy of small target detection.

[0067] Detection Head: A decoupled detection head is used to handle classification and localization tasks separately, improving detection accuracy.

[0068] Lightweighting techniques: By using channel pruning and channel distillation techniques, the number of model parameters is reduced, thus achieving lightweighting of the model, making it suitable for deployment on edge devices.

[0069] 2. Integration of attention mechanisms

[0070] like Figure 2 As shown, after each PConv and PWConv layer of FastNet, an integrated attention mechanism CBAM module is implemented, specifically including a Channel Attention module and a Spatial Attention module. Their mathematical expressions are as follows:

[0071] Mathematical expression for channel attention mechanism:

[0072] M c =σ(W2·δ(W1·AvgPool(X))+W2·δ(W1·MaxPool(X)));

[0073] Among them, M c X represents the channel attention map, with a size of (C, 1, 1) indicating the importance of each channel; X represents the input feature map, with a size of (C, H, W), where C is the number of channels, H is the height, and W is the width; AvgPool(·) represents global average pooling of the input feature map in the spatial dimension, with an output size of (C, 1, 1); MaxPool(·) represents global max pooling of the input feature map in the spatial dimension, with an output size of (C, 1, 1); W1 and W2 are learnable weight matrices, corresponding to the weights of the fully connected layer and convolutional layer of the channel attention, respectively, determining the weighting method in the channel dimension; δ(·) is the ReLU activation function; σ(·) is the Sigmoid function, with an output range of (0, 1);

[0074]

[0075] Among them, Y c This represents the feature output map after channel attention weighting, and its size is the same as X. For element-wise multiplication, M... c After being broadcast to the same spatial dimension as X, it is multiplied with X channel by channel;

[0076] Mathematical expression of spatial attention mechanism:

[0077] M s =σ(Conv 7×7 ([AvgPool(Y c MaxPool(Y) c )]));

[0078] Among them, AvgPool(Y c ) indicates that Y is in the channel dimension c Perform a pixel-wise averaging operation, with an output size of (1, H, W), MaxPool(Y) c ) represents the channel dimension of Yc Perform a pixel-wise maximum value operation, outputting a size of (1, H, W); [AvgPool(Y c MaxPool(Y) c [] indicates that the average pooling result and the max pooling result are concatenated along the channel dimension, with a size of (2, H, W); Conv 7×7 (·) represents a 7×7 convolution operation used to extract spatial attention information from the concatenated features; M s The inter-attention map is of size (1, H, W), representing the importance of each location in the spatial dimension;

[0079] The mathematical expression for the overall attention mechanism Y:

[0080]

[0081] Where Y represents the final output feature map after applying channel attention and spatial attention, with a size of (C, H, W).

[0082] 3. Optimization of Feature Fusion (FPN) Method

[0083] like Figure 3 As shown, an FPN structure is introduced into the Neck section and combined with the OD-SlimNeck module. The specific process is as follows:

[0084] (1) First, perform feature fusion according to the top-down path:

[0085]

[0086] in, To achieve feature fusion, it is obtained at level l; P l This is the original feature map. This indicates the feature map that has been fused in the previous layer; UpSample(·) indicates an upsampling operation, which increases the resolution of the high-level features and then adds them element-wise to the low-level features. l represents the index of the feature map, and L represents the index of the highest layer.

[0087] (2) Horizontal connection: For each After adjusting the number of channels using a 1×1 convolution, it is compared with the original P. l Add elements one by one.

[0088] 4. Introduction of self-supervised learning

[0089] like Figure 4 As shown, FasterNet is pre-trained using self-supervised contrastive loss (SimCLR). The specific steps are as follows:

[0090] (1) Data augmentation: Apply augmentation operations such as random cropping, color jittering, and rotation to each input image to generate two different views.

[0091] (2) Feature extraction: FasterNet is used to extract the feature representation z of the two enhanced views. i and z j .

[0092] (3) Self-supervised contrast loss calculation (SimCLR): Calculate the similarity between positive sample pairs and negative sample pairs to optimize the contrast loss.

[0093]

[0094] Among them, z i and z j To extract enhanced feature representations of the two views using FasterNet, τ is a temperature parameter that controls the degree of softening distribution; z i and z j This represents the feature vector obtained from different augmented views of the same image, typically with a size of (d, 1) or (1, d), sim(z) i ,z j ) represents the similarity function between vectors, defined as cosine similarity; 2N is the total number of sample views, N samples, each sample generates two views; l [k≠i] The indicator function takes the value 1 when k ≠ i and 0 otherwise. It is used to exclude the influence of the same view itself. The self-supervised contrastive loss function is used to optimize the feature representation of the backbone network, so that different views of the same image are close in the feature space and the features of different images are far apart.

[0095] 5. Lightweight technology

[0096] like Figures 5-6 As shown, the model is lightweighted through channel pruning and channel distillation. The specific steps are as follows:

[0097] Channel pruning:

[0098] (1) Sparse training is performed on the model after attention mechanism and feature fusion optimization, and the scaling factor of BatchNormalization layer is used as the basis for pruning.

[0099] (2) Set a global threshold to remove channels with scaling factors close to zero and reduce the number of model parameters.

[0100] (3) Restore model accuracy by fine-tuning.

[0101]

[0102] Among them, w ij These are weights used to measure the contribution of different channels or different samples to the loss. and These are the channel outputs for the student model and the teacher model, respectively; ||·|| refers to the Euclidean distance or other norm distance, which is often the square of the L2 norm in this case. 2 . Channel Distillation Loss is used to improve student model performance by minimizing the difference between the student model's output and the teacher model's output.

[0103] Overall loss function for

[0104]

[0105] Where λ and β are weight parameters; they control the contribution of each loss to the overall training. The supervised loss for the object detection task itself includes box regression loss and classification loss. This is the self-supervised contrastive loss (SimCLR) component; This refers to the channel distillation loss.

[0106] 6. Training Process

[0107] (1) Self-supervised pre-training stage

[0108] We use the SimCLR method to pre-train FasterNet+CBAM on large-scale unlabeled data to optimize feature representation.

[0109] (2) Transfer learning and fine-tuning stage

[0110] The pre-trained FasterNet+CBAM is integrated into the OD-YOLOV7-tiny model, and FPN is used to optimize feature fusion.

[0111] Supervised fine-tuning of the entire detection model using labeled data optimizes detection performance.

[0112] (3) Lightweighting stage

[0113] Channel pruning is applied to the fine-tuned model to reduce the number of model parameters.

[0114] The detection accuracy of the pruned model can be further improved by using channel distillation.

[0115] (4) Final model deployment

[0116] The lightweight OD-YOLOv7-tiny model can be deployed to edge devices such as drones to achieve real-time defect detection of power insulators.

[0117] Algorithm model implementation effect

[0118] This invention significantly improves the performance of the target detection model in power insulator defect detection by integrating an attention mechanism, optimizing the feature fusion method, and introducing self-supervised learning. Specifically:

[0119] Improved detection accuracy: The integration of the CBAM attention module and FPN feature fusion structure enhances the model's ability to detect defects in small targets and complex backgrounds.

[0120] Lightweight model: Through channel pruning and channel distillation techniques, the number of model parameters and computational overhead are significantly reduced, making it suitable for real-time deployment on edge devices.

[0121] Feature representation enhancement: Self-supervised learning methods improve the feature representation capabilities of the backbone network, thereby enhancing the model's generalization performance.

[0122] Real-time performance guarantee: The lightweight model maintains high detection accuracy while significantly shortening inference time, meeting the needs of real-time detection.

[0123] The effectiveness of the proposed model was verified through experiments on a power insulator defect dataset. Experimental results show that the detection accuracy of the model is significantly improved after integrating the attention mechanism and optimizing the feature fusion method. By using channel pruning and channel distillation, the number of model parameters is reduced by more than 50%, and the inference speed is improved by about 37%. While maintaining high detection accuracy, the model achieves lightweight and high efficiency, making it suitable for real-time deployment on edge devices.

[0124] Therefore, the present invention adopts a lightweight target detection system that integrates attention mechanism, optimized feature fusion method and self-supervised learning, which is particularly suitable for efficient and accurate detection of defects in power insulators. Through various improvements based on YOLOv7-tiny, the high performance and lightweight nature of the model are organically combined, and it has broad application prospects.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A lightweight object detection system integrating attention mechanisms, optimized feature fusion, and self-supervised learning, characterized in that: It includes a backbone network, a feature fusion module (Neck), a detection head, a lightweight module, and a self-supervised learning module; The backbone network is based on the FasterNet architecture, and the Convolutional Block Attention (CBAM) module is integrated into some of the PConv and PWConv layers in the architecture to enhance the attention capabilities of channel and spatial features. The feature fusion module introduces the Feature Pyramid Network (FPN) structure and combines it with the OD-SlimNeck module to optimize the fusion effect of multi-scale features; The detection head is a decoupled detection head, which includes independent classification and localization branches, used to handle the prediction of target category and target location, respectively. The lightweight module reduces the number of model parameters through channel pruning and channel distillation techniques; The self-supervised learning module uses a contrastive learning method to pre-train the backbone network.

2. The lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning according to claim 1, characterized in that, The CBAM module comprises two sub-modules: channel attention and spatial attention. These are used to enhance the channel and spatial information of the feature map, respectively. The mathematical expression for the channel attention mechanism is as follows: ； in, This is a channel attention map with a size of (C, 1, 1), representing the importance of each channel. The input feature map has a size of (C, H, W), where C is the number of channels, H is the channel height, and W is the channel width. This indicates that global average pooling is performed on the input feature map in the spatial dimension, and the output size is (C, 1, 1). This indicates that global max pooling is performed on the input feature map in the spatial dimension, and the output size is (C, 1, 1). and The weight matrix is ​​a learnable matrix, corresponding to the weights of the fully connected layer and the convolutional layer of the channel attention, respectively, which determines the weighting method in the channel dimension; It is the ReLU activation function; This is the Sigmoid function, with an output range of (0,1). ； in, The output feature map is the feature map after channel attention weighting, and its size is... Same size; For element-wise multiplication, that is... spread to After the same spatial dimension, with Perform channel-wise product; The mathematical expression for spatial attention mechanism is: ； in, This indicates that in the channel dimension... Perform a pixel-wise averaging operation, and the output size is (1, H, W). Represents the channel dimension Perform a pixel-by-pixel maximum value operation, and output a size of (1, H, W); This means concatenating the average pooling result and the max pooling result along the channel dimension, with a size of (2, H, W); Represent a The convolution operation is used to extract spatial attention information from the concatenated features; The inter-attention map is of size (1, H, W), representing the importance of each location in the spatial dimension; Overall attention mechanism The mathematical expression: ； in, This represents the final output feature map after applying both channel attention and spatial attention, with a size of (C, H, W).

3. The lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning according to claim 2, characterized in that, The feature fusion module combines multi-scale feature maps and utilizes global and local features to enhance the representational power of the feature maps. The FPN structure fuses feature maps of different scales layer by layer through a top-down path and lateral connections. The top-down path performs feature fusion as follows: ； in, To integrate features, at the level Obtained; This is the original feature map. This represents the feature maps that have already been fused from the previous layer; This indicates an upsampling operation, which increases the resolution of high-level features before adding them element-wise to low-level features. Indicates the index of the feature map. Indicates the highest-level index; Horizontal connection: for each After adjusting the number of channels using a 1×1 convolution, compared with the original... Add elements one by one.

4. The lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning according to claim 3, characterized in that, The channel pruning technique of the lightweight module reduces the number of model parameters and computational overhead by evaluating the scaling factor of the Batch Normalization layer and pruning channels with scaling factors close to zero. The lightweight module's channel distillation technique uses the original YOLOv7 model as the teacher model and performs channel distillation on the pruned student model. Knowledge transfer is then performed on samples correctly predicted by the teacher model, and the distillation loss function is used. : ； in, Weights are used to measure the contribution of different channels or different samples to the loss. , Temperature is a parameter that controls the degree of softening distribution. and This represents the feature vector obtained from different augmented views of the same image. The similarity function between vectors is defined as cosine similarity; This represents the total number of sample views. N For each sample, two views are generated. For indicator functions, when The value is 1 if the view is identical to itself, and 0 otherwise, to exclude the influence of the same view itself. and These are the channel outputs for the student model and the teacher model, respectively. This refers to the Euclidean distance or other norm distance, in this case, the square of the L2 norm. Channel distillation loss improves student model performance by minimizing the difference between the student model's and the teacher model's outputs; Overall loss function for ； in, and These are weight parameters that control the contribution of each loss to the overall training. The supervised loss for the object detection task itself includes bounding box regression loss and classification loss; This is the self-supervised contrastive loss part of SimCLR; Channel distillation loss.

5. A lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning as described in claim 4, characterized in that, The self-supervised learning module generates different views by applying random cropping, color jittering, and rotational data augmentation operations to the input image. The self-supervised contrastive loss function, SimCLR, is calculated as follows: ； The self-supervised contrastive loss function is used to optimize the feature representation of the backbone network, so that different views of the same image are close together in the feature space, and features of different images are far apart.

6. The lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning according to claim 5, characterized in that, The lightweight module also includes a model fine-tuning step, which is used to fine-tune the pruned model.

7. A lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning as described in claim 6, characterized in that, The decoupled detection head employs a hybrid channel strategy, partially sharing weights and using them for classification and localization prediction.

8. An application method of a lightweight target detection system integrating attention mechanism, optimized feature fusion, and self-supervised learning as described in any one of claims 1-7, characterized in that, Includes the following steps: S1. Data Acquisition: Use drones to collect images of electrical insulators on high-voltage transmission lines; S2. Data preprocessing: Standardize the acquired images, including cropping, scaling, and enhancement; S3. Model pre-training: The backbone network is pre-trained on large-scale unlabeled data using a self-supervised learning method; S4. Model Integration: Integrate the pre-trained backbone network with the feature fusion module and the decoupled detection head to build a complete target detection model; S5. Model Training: Supervised training of the integrated model using labeled data to optimize detection performance; S6. Model Compression: By using channel pruning and channel distillation techniques, the number of model parameters is reduced, thus achieving model lightweighting. S7. Model Deployment: Deploy the compressed model onto the drone to achieve real-time power insulator defect detection.

9. The application method of the lightweight target detection system integrating attention mechanism, optimized feature fusion and self-supervised learning as described in claim 8, characterized in that, In S5, the training process uses a hybrid loss function, combining supervised detection loss and self-supervised contrastive loss.

10. The application method of the lightweight target detection system integrating attention mechanism, optimized feature fusion and self-supervised learning according to claim 9, characterized in that, In S6, a global pruning threshold is set to prune all channels with scaling factors below the threshold, and model performance is restored by fine-tuning the training.

Images