Insulator defect real-time detection method and system based on channel trimming and distillation

Abstract

The invention relates to the technical field of electric power operation and maintenance, and particularly discloses an insulator defect real-time detection method and system based on channel trimming and channel distillation, and the method comprises the steps: carrying out the image collection of an insulator; the collected images are preprocessed; inputting the preprocessed image into an original detection model; optimizing a backbone network, a neck part and a detection head of the original detection model to obtain a lightweight detection model of which the model complexity and the calculated amount are lower than those of the original detection model; performing channel trimming and channel distillation on the lightweight detection model to obtain a distillation model; and deploying the distillation model after knowledge migration on mobile equipment to realize real-time detection of insulator defects. According to the insulator defect real-time detection method and system based on channel trimming and channel distillation provided by the invention, high detection precision can be kept while miniaturization of the model is realized, so that the actual requirements of mobile equipment such as an unmanned aerial vehicle are met.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power operation and maintenance, and in particular to a real-time detection method and system for insulator defects based on channel pruning and channel distillation. Background Art

[0002] As China's power industry develops, transmission lines are becoming longer and more complex. Insulators, as key components, have a dual role: providing mechanical support and ensuring electrical insulation. Insulators are exposed to the atmosphere all year round and are vulnerable to natural factors such as lightning strikes, pollution and severe cold. Failure of insulators may cause transmission lines to come into contact with towers or other lines, resulting in power outages and potential large-scale blackouts. Therefore, regular inspections of porcelain insulators on transmission lines are essential for reliable power transmission throughout the grid. Traditionally, insulator inspections are performed manually. However, inspecting high-voltage transmission lines presents significant challenges in terms of difficulty, efficiency and safety.

[0003] The rapid development of unmanned aerial vehicle (UAV) technology has enabled power systems to use drones to capture images of insulators along high-voltage transmission lines. These images are then analyzed and diagnosed using image processing algorithms. Despite this progress, the wide angle and variable viewing angle of drone photography present challenges such as small target size and complex background, resulting in low detection accuracy, weak feature map representation, and limited extraction of key information. Deep learning-based object detection algorithms have been shown to be very effective in identifying defects in target objects, such as external defects in capacitors. As a result, researchers are increasingly leveraging image processing and deep learning techniques to detect insulator defects.

[0004] Object detection technology can be roughly divided into traditional methods and deep learning-based methods. The introduction of AlexNet marks the transition between the two. At present, the mainstream deep learning-based object detection algorithms are divided into two categories: one is a two-stage object detection algorithm based on region proposal, and the other is an end-to-end single-stage object detection algorithm based on regression.

[0005] Typical two-stage target detection algorithms include R-CNN, Fast R-CNN, and Faster R-CNN. These methods involve generating region proposals and then classifying these proposals, which improves detection accuracy but also results in a large model size, which is not conducive to deployment on mobile devices such as drones with limited computing resources.

[0006] Representative single-stage target detection algorithms include the YOLO series of algorithms, which frame target detection as a regression problem and achieve end-to-end fast detection. Single-stage target detection algorithms usually perform poorly in detecting small targets because these targets occupy fewer pixels in the image, have less obvious features, and are easily overlooked. Therefore, single-stage detection algorithms are usually not as accurate as two-stage detection algorithms (such as Faster R-CNN) in accurately locating targets.

[0007] Therefore, the present invention is devoted to proposing a real-time detection method for insulator defects, aiming to achieve miniaturization of the model while maintaining high detection accuracy to meet the actual needs of mobile devices such as drones.

[0008] The above information disclosed in this Background section is included only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is currently known to a person of ordinary skill in the art. Summary of the invention

[0009] One object of the present invention is to provide a real-time detection method and system for insulator defects based on channel pruning and channel distillation, aiming to achieve miniaturization of the model while maintaining high detection accuracy to meet the actual needs of mobile devices such as drones.

[0010] To achieve the above objectives, on the one hand, the present invention provides a real-time detection method for insulator defects based on channel pruning and channel distillation, comprising:

[0011] S10: Using drones to collect images of insulators along high-voltage transmission lines;

[0012] S20: preprocessing the collected images, including image denoising, enhancement, and cropping, to highlight the defect features of the insulator;

[0013] S30: inputting the preprocessed image into an original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image;

[0014] S40: Optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model;

[0015] S50: performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure, and number of parameters of the model;

[0016] S60: Migrating the knowledge of the original detection model to the pruned model by channel distillation to obtain a distilled model that maintains the detection accuracy of the pruned model;

[0017] S70: Deploy the distillation model after knowledge migration on a mobile device to achieve real-time detection of insulator defects.

[0018] Optionally, the original detection model is an existing YOLOv7-tiny model, and the lightweight detection model is an improved YOLOv7-tiny model.

[0019] Optional,

[0020] Step S40 is specifically as follows:

[0021] Based on the existing YOLOv7-tiny model, the FasterNet architecture is used as the backbone network, the neck part is replaced, and a decoupled detection head is used for detection to obtain an improved YOLOv7-tiny model with higher detection accuracy and efficiency.

[0022] Optionally, the neck portion is replaced, including:

[0023] Build the GS bottleneck module and integrate the GS bottleneck module on top of GSConv;

[0024] Replace the neck of the existing YOLOv7-tiny model with the GSConv module after integrating the GS bottleneck module.

[0025] Optionally, the neck portion is replaced, including:

[0026] Construct OD-Slimneck module based on slimNeck module and ODConv module;

[0027] Replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module.

[0028] Optional,

[0029] The channel pruning includes: pruning based on the sparsity of the convolution kernel to simultaneously maintain the detection accuracy and detection speed of the model. The pruned model maintains or improves the detection accuracy and detection efficiency while reducing the size and computational complexity.

[0030] In another aspect, a detection system is provided, comprising:

[0031] drones for image acquisition of insulators along high-voltage transmission lines;

[0032] An image preprocessing module is used to preprocess the collected images, including image denoising, enhancement and cropping, to highlight the defect features of the insulator;

[0033] The original detection module is used to input the preprocessed image into the original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image;

[0034] A detection model optimization module, used for optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model;

[0035] A channel pruning module, used for performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure and number of parameters;

[0036] A channel distillation module, used to transfer the knowledge of the original detection model to the pruned model by means of channel distillation, so as to obtain a distilled model that maintains the detection accuracy of the pruned model;

[0037] The mobile device deployment module is used to deploy the distillation model after knowledge migration on the mobile device to realize real-time detection of insulator defects.

[0038] Optionally, the detection model optimization module includes a basic model unit for providing an existing YOLOv7-tiny model, a backbone network replacement unit for adopting a FasterNet architecture as a backbone network, a neck replacement unit for replacing a neck part, and a decoupled detection head.

[0039] Optionally, the neck replacement unit is used to:

[0040] Build the GS bottleneck module and integrate the GS bottleneck module on top of GSConv;

[0041] Replace the neck of the existing YOLOv7-tiny model with the GSConv module after integrating the GS bottleneck module.

[0042] Optionally, the neck replacement unit is used to:

[0043] Construct OD-Slimneck module based on slimNeck module and ODConv module;

[0044] Replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module.

[0045] The beneficial effect of the present invention is to provide a real-time detection method for insulator defects based on channel pruning and channel distillation:

[0046] 1. Enhanced feature fusion network: An OD-SlimNeck network architecture for drone detection of insulator defects is proposed. This architecture enhances the detection capability of small defects and effectively avoids the loss of important details. By integrating multi-scale features and making full use of local and global characteristics, the network's mapping representation of features is strengthened, thereby improving the overall detection performance. In addition, full-dimensional dynamic convolution allows the network to extract regions of interest from complex backgrounds and images with more interference, focusing on key targets and further improving the performance of the model;

[0047] 2. Compress the model through channel pruning: The improved model is compressed by channel pruning. Before pruning, the model is first trained for sparsity to ensure that the model performance does not drop significantly within a certain pruning limit. By pruning the channel, the number of parameters and the amount of calculation are reduced, a lighter network is achieved, and its effectiveness is fully evaluated;

[0048] 3. Channel distillation to improve accuracy: The YOLOv7 model is used as the original detection model to perform structural improvement, channel pruning, and channel distillation in sequence. This method improves the accuracy of insulator defect detection without increasing the number of parameters, provides a feasible solution for deploying insulator defect detection models on mobile devices such as drones, and ensures accurate identification of insulator defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 A flow chart of a real-time detection method for insulator defects based on channel pruning and channel distillation provided in an embodiment;

[0051] Figure 2 An algorithm framework diagram of the existing YOLOv7-tiny model provided in the embodiment;

[0052] Figure 3 An algorithm framework diagram of the YOLOv7-tiny improved model provided in the embodiment;

[0053] Figure 4 A schematic diagram of the structure of a decoupled detection head provided in an embodiment;

[0054] Figure 5 A schematic diagram comparing different convolution modes provided for the embodiments;

[0055] Figure 6 A schematic diagram of the structure of FasteNet Block provided in the embodiment;

[0056] Figure 7 A schematic diagram of the structure of the GSConv model provided in the embodiment;

[0057] Figure 8 A schematic diagram of the structure of the GS bottleneck module provided in the embodiment;

[0058] Fig. 9 A schematic diagram of the structure of the VoV-GSCSP module provided in the embodiment;

[0059] Fig.10 A schematic diagram of the structure of the ODConv module provided in the embodiment;

[0060] Fig.11 A schematic diagram of the structure of the OD-Slimneck model provided in the embodiment;

[0061] Fig.12 A pruning principle diagram of a channel pruning algorithm provided in an embodiment;

[0062] Fig.13 A schematic diagram of a channel pruning process provided by an embodiment;

[0063] Fig.14 A schematic diagram of the distillation process structure provided for an embodiment;

[0064] Fig.15 Schematic diagram of the acquisition process of the OD-YOLOV7-tiny network model. DETAILED DESCRIPTION

[0065] The reference to "embodiment" in the present invention means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The word "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present invention, as long as there is no technical contradiction or conflict, the various technical features mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0066] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which the present invention belongs; the use of relevant terms herein is only for describing specific embodiments and is not intended to limit the present invention.

[0067] In the description of the present invention, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in a logical relationship of "or".

[0068] In the present invention, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0069] Without further restrictions, in the present invention, the words "include", "comprises", "has" or other similar expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0070] Similar to the understanding in the Examination Guidelines, in the present invention, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of the present invention, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0071] In the description of the embodiments of the present invention, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present invention or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0072] Unless otherwise expressly specified or limited, in the description of the embodiments of the present invention, the terms such as "install", "connect", "connect", "fix", "set" and the like used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art of the technical field to which the present invention belongs, the specific meanings of the above terms in the embodiments of the present invention can be understood according to the specific circumstances.

[0073] The present invention provides a real-time detection method for insulator defects based on channel pruning and channel distillation, aiming to achieve miniaturization of the model while maintaining high detection accuracy to meet the actual needs of mobile devices such as drones.

[0074] See also Figure 1 The real-time detection method for insulator defects based on channel pruning and channel distillation provided in this embodiment includes the following steps:

[0075] S10: Using drones to collect images of insulators along high-voltage transmission lines;

[0076] In the daily operation and maintenance of high-voltage transmission lines, the status monitoring of insulators is an important part of ensuring the safe operation of the power grid. Since transmission lines are usually located in remote or difficult-to-reach areas, traditional inspection methods are not only time-consuming and labor-intensive, but also difficult to achieve high-frequency and comprehensive inspections of a large number of insulators. Therefore, the present invention uses drone technology for image acquisition, which has the following advantages:

[0077] Efficiency: Drones can quickly cover large areas of power transmission lines, significantly improving detection efficiency compared to manual inspections.

[0078] High-resolution images: The high-resolution camera can capture subtle defects on the surface of the insulator and provide clear image data for subsequent defect identification.

[0079] Safety: Drone inspections avoid direct contact between personnel and high-voltage lines, reducing safety risks.

[0080] Controllability: The UAV can adjust the flight altitude and angle as needed to obtain insulator images from different perspectives to ensure comprehensive defect detection.

[0081] Real-time: Images collected by drones can be transmitted to ground stations in real time for timely analysis and processing.

[0082] By collecting images using drones, the present invention can provide a large amount of diverse and high-quality training data for deep learning models, thereby improving the accuracy and efficiency of insulator defect detection. This step is the basis of the entire detection process and directly affects the effects of subsequent image preprocessing, model training and defect identification.

[0083] S20: preprocessing the collected images, including image denoising, enhancement, and cropping, to highlight the defect features of the insulator;

[0084] The images collected by drones may contain various noises and irrelevant information, which may interfere with the subsequent defect detection process. Therefore, image preprocessing is a key step to ensure detection accuracy. The following are the specific functions and purposes of preprocessing:

[0085] ① Image denoising: Due to environmental factors (such as wind speed, raindrops, dust, etc.) and the limitations of the imaging device itself, the collected images may contain noise. Denoising can reduce these random noises, improve image quality, and make insulator defects clearer.

[0086] ② Image enhancement: Enhancement processing can improve the contrast and brightness of the image, making the defect characteristics of the insulator more obvious. By adjusting the brightness and contrast of the image, the difference between the defect and the background can be enhanced, which is convenient for subsequent defect identification.

[0087] ③ Image cropping: The cropping process is to remove irrelevant parts of the image, such as the sky, distant scenery, etc., so as to focus on the insulator area. This helps to reduce the amount of calculation and allows the model to focus on the defect area of ​​interest.

[0088] Through these preprocessing steps, the following goals can be achieved:

[0089] ① Improve the availability of image data and provide higher quality input data for deep learning models.

[0090] ②Reduce interference factors in model processing and improve the accuracy of defect detection.

[0091] ③Reduce the computational complexity of subsequent processing steps and improve the efficiency of the overall detection process.

[0092] The preprocessed images will be more suitable for training deep learning models and ultimately achieve higher accuracy and faster processing speed in real-time detection. This step is an important prerequisite to ensure that subsequent models can effectively learn and identify insulator defects.

[0093] S30: inputting the preprocessed image into an original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image;

[0094] After the image preprocessing step is completed, the resulting image has a high quality and is suitable as input data for the deep learning model.

[0095] Specifically, the selection of the original detection model gives priority to the detection accuracy of the model, followed by the detection speed of the model, which can be an existing deep learning model with high detection accuracy.

[0096] In this embodiment, the original detection model may be a YOLOV7-Tiny model.

[0097] As one of the most prominent target detection algorithms, the YOLO series of models effectively balances detection speed and accuracy. YOLOv7 has multiple versions, each of which differs in the width and depth of the network model, but all share a similar structure. Considering the importance of speed in drone aerial photography insulator defect detection, the smallest and fastest version can be selected: YOLOv7-tiny. The specific structure of YOLOv7-tiny is as follows Figure 2 shown.

[0098] S40: Optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model;

[0099] In this embodiment, the original detection model is the existing YOLOv7-tiny model, and the lightweight detection model is the YOLOv7-tiny improved model.

[0100] Step S40 is specifically as follows:

[0101] Based on the existing YOLOv7-tiny model, the FasterNet architecture is used as the backbone network, the neck part is replaced, and a decoupled detection head is used for detection to obtain an improved YOLOv7-tiny model with higher detection accuracy and efficiency.

[0102] (1) YOLOv7-tiny model

[0103] YOLOv7-tiny is a popular single-stage object detection algorithm that is known for its fast speed and relatively high accuracy, but there is still room for improvement.

[0104] (2) Using FasterNet architecture as the backbone network

[0105] The backbone network is the part of the target detection model used to extract image features. The FasterNet architecture is a new neural network architecture that reduces redundant calculations and improves computational efficiency through partial convolution (PConv) technology. Using FasterNet as the backbone network of YOLOv7-tiny can enhance the speed and efficiency of the model in extracting features.

[0106] When using drone aerial photography for insulator defect detection tasks, it is critical to ensure that the detection model can be deployed on mobile devices such as drones while maintaining high detection speed. To achieve this goal, this embodiment focuses on designing a cost-effective and high-speed neural network.

[0107] The traditional YOLOv7-tiny neural network architecture includes some redundant calculations, resulting in a higher number of floating point operations (flops). The increase in FLOPs leads to model latency. The relationship between latency and FLOPs is as follows:

[0108] (1)

[0109] FLOPs stands for floating point operations, and FLOPs stands for floating point operations per second. The division of FLOPs and FLOPs can be used as a measure of computational latency. As can be seen from formula (1), the FasterNet backbone network improves detection speed by effectively reducing FLOPs while increasing FLOPs, thereby reducing latency and increasing computational speed without affecting accuracy.

[0110] This embodiment improves the neural network architecture of the existing YOLOv7-tiny model by means of deep convolution (DWConv).

[0111] Deep convolution (DWConv) is a commonly used optimization technique in backbone networks. Unlike traditional convolution methods, DWConv assigns a convolution kernel to each channel, ensuring that each channel is processed by only one kernel. This approach significantly reduces redundant computations and floating-point operations per second (FLOPs). However, replacing regular convolution with DWConv alone may result in reduced network accuracy. To address this issue, point convolution (PWConv) is often used together with DWConv to improve accuracy. In this setting, the number of channels of DWConv is expanded from c to c', exceeding the original number of channels c of standard convolution to offset the accuracy loss caused by DWConv. However, the increase in channels requires more memory access, which increases latency and reduces overall computational efficiency. The memory access of DWConv is detailed in Equation (2), where h and w represent the height and width of the image, respectively, c represents the number of channels, and the size of the convolution kernel is K×K.

[0112] (2)

[0113] The memory access of regular convolution is as follows:

[0114] (3)

[0115] When c' exceeds c, the memory requirement of DWConv exceeds that of standard convolution. Therefore, it is necessary to develop a new convolution module to replace the inefficiencies associated with regular convolution and DWConv, thereby improving the detection speed.

[0116] Compared with standard convolution and DWConv, the partial convolution (PConv) in FasterNet only needs to apply regular convolution to part of the input channels to extract spatial features, and the remaining channels remain unchanged. If the feature map is stored continuously or regularly in memory, the first or last continuous channel can be regarded as the representative of the entire feature map. According to experimental results, PConv significantly reduces the amount of computation compared with standard convolution, and the amount of computation required is only 1 / 16 of that of regular convolution. At the same time, the memory access of PConv is also reduced to 1 / 4 of that of regular convolution. Similar to DWConv, the FasterNet module adds point convolution (PWConv) to PConv to capture the correlation between input channels.

[0117] This combination forms two structures, including T-shaped convolution and two independent convolution structures. Comparison of different convolution modes is shown in Figure 5 shown.

[0118] Unlike traditional convolution, t-convolution gives priority to the center position more obviously. Although the t-convolution module can be used directly for more efficient calculations, it requires more floating point operations per second (FLOPs) compared to PConv and PWConv. For the same input and output features, the FLOPs of t-convolution are as follows:

[0119] (4)

[0120] The Flops of PConv and PWConv are as follows:

[0121] (5)

[0122] In continuous memory access, is the first or last consecutive channel number, and Obviously, the number of floating point operations (FLOPs) of T-convolution is greater than that of PConv and PWConv.

[0123] Each FasterNet block consists of a partial convolution (PConv) layer and two point-wise convolution (PWConv) layers, as shown in Figure 6 As shown. In addition, normalization and activation layers play a vital role in neural networks and are indispensable components. In each FasterNet module, batch normalization (BN) and rectified linear units (ReLU) are integrated into two PWConv layers. BN improves training speed and accuracy, while ReLU as an activation function accelerates model training and alleviates the gradient vanishing problem. Placing normalization and activation layers between two PWConv layers in each FasterNet module helps reduce latency and maintain feature diversity.

[0124] Replacing the YOLOv7 backbone network with FasterNet effectively reduces the number of FLOPs and increases the quality of FLOPs, thereby reducing memory access and latency. This improvement reduces the detection time per image during the detection process, thereby improving the FPS value.

[0125] (3) Introducing the OD-SlimNeck feature extraction framework in the neck part

[0126] The neck part usually refers to the part between the backbone network and the detection head, which is responsible for further processing and fusing the features extracted by the backbone network. OD-SlimNeck is a feature extraction framework that aims to integrate and refine features in a more efficient way, reduce the complexity and computation of the model, while maintaining or improving the detection performance.

[0127] Object detection is a key downstream task in computer vision. For various mobile devices, models with a large number of parameters often fail to meet the requirements of real-time detection. In order to accelerate the prediction calculation, the input image in CNN must undergo an almost similar transformation process in the backbone network: the spatial information is gradually transferred to the channel. Each feature map of spatial compression and channel expansion may result in local loss of semantic information. Dense convolution calculations maximize the retention of hidden connections between channels, while sparse convolutions completely disconnect these connections.

[0128] 1) Replace the neck of the existing YOLOv7-tiny model based on the GSConv module

[0129] Figure 7The structure of the GSConv module is shown. GSConv (short for Generative Surgery Convolution, an improved structure of a convolutional neural network (CNN) used in computer vision, especially in image generation tasks) can maintain connectivity throughout the application process, but deploying it in all stages of the model leads to an increase in network depth. This additional depth hinders data flow and significantly prolongs inference time. When feature maps reach the neck stage, they are maximally compressed and minimized in the channel dimension and minimized in the spatial dimension, and further transformations become unnecessary.

[0130] Therefore, an alternative strategy is to use only GSConv modules in the neck. First, GSConv replaces the lightweight convolution method SC, with a computational cost of about 60-70% of SC, while still having a comparable impact on the learning ability of the model. Subsequently, the GS bottleneck layer is integrated on top of GSConv to further improve the performance of the model.

[0131] The construction of GS bottleneck module is as follows Figure 8 After that, the cross-layer part of the network, namely the VoV-GSC SP module, is designed using a one-shot aggregation method. The VoV-GSCSP module reduces the computational and architectural complexity while maintaining good accuracy. Fig. 9 The construction of the VoV-GSCSP module is described.

[0132] 2) Build the OD-Slimneck module based on the slimNeck module and the ODConv module; replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module

[0133] In addition, in some other embodiments, an OD-Slimneck module may be constructed based on the slimNeck module and the ODConv module; and the neck of the existing YOLOv7-tiny model may be replaced with the OD-Slimneck module.

[0134] The slimNeck module is designed with hardware friendliness in mind, facilitating easier deployment and operation on a variety of hardware platforms. Therefore, replacing the neck of the existing YOLOv7-tiny model with the slimNeck module not only maintains good performance but also improves efficiency and broadens the potential range of applications.

[0135] Due to the parameter sharing characteristics of convolutional neural networks (CNNs), traditional feature filters use convolution kernel parameters that are fixed for the entire image. Therefore, when optimizing CNNs, it is usually necessary to increase the model size and parameters, as well as to utilize larger data sets. This requirement inevitably leads to a significant increase in computational overhead. Omni-dimensional dynamic convolution (ODConv) expands on the traditional dynamic convolution kernel by combining the size of the spatial kernel and the number of input and output channels in the kernel space. The ODConv module is as follows: Fig.10 shown.

[0136] In this embodiment, the ODConv module is used to replace the standard convolution in the backbone network. This replacement enhances the performance of the network, making it more efficient and streamlined. Based on the concepts introduced by SlimNeck and ODConv, the goal is to reduce the inference speed and complexity of the neck part of the network while improving the accuracy of the model in detecting insulator defects.

[0137] Therefore, if Fig.11 As shown, in this embodiment, the ODConv module is integrated after the GSConv module to obtain a new structure: OD-Slimneck, which can significantly improve the accuracy of detecting small-scale targets such as insulator defects while minimizing the increase in the number of parameters of the model.

[0138] The slimNeck module and the ODConv module may be complementary in functionality, with slimNeck focusing on reducing the complexity and computation of the neck structure, while ODConv focuses on improving the flexibility and efficiency of the convolution operation.

[0139] The OD-SlimNeck structure replaces the neck network of YOLOV7-tiny and takes advantage of the lightweight SlimNeck design to enhance the detection performance of small objects compared to the standard convolutional layer.

[0140] OD-Slimneck is a part or an improved version of the entire network structure, which combines the slimNeck module and ODConv technology. The purpose of OD-Slimneck is to use the efficient feature fusion capability of slimNeck and the dynamic convolution characteristics of ODConv to improve the performance and efficiency of the network.

[0141] (4) Use a decoupled detection head for detection

[0142] In the object detection model, the "Detection Head" is a key component of the model, which is responsible for predicting the category and location of the target based on the feature map extracted by the model. In the YOLOv7-tiny model, the detection head is usually located at the end of the network. It receives feature maps from the network backbone (such as CSPDarknet) and the neck (such as FPN or PAN) and outputs the final detection result.

[0143] Decoupled detection head means that the design of the detection head separates the localization of the bounding box (i.e. the location and size of the object) and the category prediction (i.e. what the object is) into different steps or modules. This design can make the model more focused and efficient in performing these two tasks, thereby improving the accuracy of detection.

[0144] Through the above improvements, the ultimate goal is to create an optimized YOLOv7-tiny improved model (such as Figure 3 As shown in the figure), the YOLOv7-tiny improved model reduces the number of parameters and improves the accuracy of insulator defect detection, making it more suitable for target detection in drone aerial images.

[0145] The following is a further introduction to the decoupled detection head:

[0146] The detection head of the existing YOLOv7-tiny model is a coupled head, which usually directly passes the feature map output from the convolutional layer into several fully connected layers or convolutional layers. This generates outputs of target locations and categories and shares parameters between classification and localization branches. Although the network architecture is relatively simple, it requires a large number of parameters and computing resources, which may lead to overfitting.

[0147] This embodiment designs a decoupling head module that can be integrated into YOLOv7-tiny. Different from the traditional coupled prediction head, the decoupling head module fully shares weights in category prediction, but only partially shares weights in positioning and confidence scoring. Combined with the hybrid channel strategy, detection is more efficient. The decoupling head reduces latency while maintaining accuracy, promotes network convergence, and enhances the detection performance of deep convolutional neural networks. The structure of the decoupling head is shown in Figure 1. Figure 4 By replacing the coupling head in the existing YOLOv7-tiny model with a decoupling head, the number of parameters and computational complexity can be effectively reduced, the generalization ability and robustness of the model can be enhanced, and the performance of target detection can be significantly improved.

[0148] S50: performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure, and number of parameters of the model;

[0149] Although the YOLOv7-tiny improved model has fewer parameters, deploying it on drones for real-time insulation inspection remains a major challenge due to the limited memory and computing power of embedded devices. To address this issue, a channel pruning algorithm is applied to the trained YOLOv7-tiny improved model to further simplify it for easier deployment on lightweight devices with limited processing power.

[0150] The most common model pruning method is channel pruning, which requires minimal software and hardware resources and effectively balances fine-grained and coarse-grained methods. Fig.12 As shown, channel pruning significantly reduces the size, structure, and number of parameters of the model.

[0151] When performing channel pruning, initially, the YOLOv7-tiny improved model was trained using sparse regularization to assign a scaling factor to each channel to represent its importance. Subsequently, channel sparsity training was performed to distinguish between essential and non-essential channels.

[0152] Batch Normalization (BN) layer is a powerful operation that ensures fast convergence and enhances generalization. Connected after the convolutional layers in most traditional network architectures, BN layer normalizes feature activations using batch statistics.

[0153] (6)

[0154] µ and σ represent the mean and standard deviation of the features in a mini-batch (Mini-batch: refers to a small number of samples randomly selected from the entire training dataset, usually containing dozens to hundreds of samples). γ and β are trainable factors for scaling and shifting, respectively, and Zout is the output of the batch normalization BN layer. The batch normalization BN layer is inserted after the convolutional layer with trainable factors, where the scaling factor is used as an indicator of channel importance. The loss function used in the sparse training process is:

[0155] (7)

[0156] Item 1 represents the training loss of the network, where x and y represent the input and output of the training respectively, and W represents the training parameters of the network. is the L1 regularization of the BN layer scale coefficient γ. The penalty factor α is used to balance the two terms, while γ is introduced as a scaling factor for each channel. By minimizing L, the model tends to drive the γ coefficient of the BN layer close to zero, thereby achieving structural sparsity.

[0157] After training with sparse regularization, a model with scaling factors approaching zero is obtained. Channels with scaling factors close to zero can then be pruned by removing all relevant incoming and outgoing connections and their respective weights. Channels are pruned using a global threshold in all layers, which is defined as a certain percentile of all scaling factor values.

[0158] The channel pruning lightweight model may initially experience some accuracy loss, which is compensated by a subsequent fine-tuning process on the pruned network, recalibrating the parameters of the pruned model to compensate for the accuracy loss caused by channel removal, achieving a balance between efficiency and performance. Fig.13 shown.

[0159] S60: Migrating the knowledge of the original detection model to the pruned model by channel distillation to obtain a distilled model that maintains the detection accuracy of the pruned model;

[0160] Knowledge distillation has important applications in computer vision, especially in the field of object detection, where it is used to reduce model parameters and improve the accuracy of existing models. After channel pruning, in order to quickly restore the accuracy of the model, channel distillation is used to assist the training of the pruned model. Different from traditional knowledge distillation (KD), the channel distillation (CD) method transfers channel information from the teacher to the student.

[0161] In SENet, the channel-wise attention mechanism enables the model to learn the weight of each channel, which is then multiplied by the original channel. This process enhances the features of important channels while weakening the features of less important channels, making the extracted features more directional and improving the prediction performance of the network. The weight of each channel is calculated as follows:

[0162] (8)

[0163] The variable wc represents the weight of the cth channel. H and W represent the spatial dimensions of the feature map, and uc(i, j) represents the activation rate.

[0164] Each channel in the feature map corresponds to a visual pattern, but the importance of visual patterns varies between channels. Given that the teacher model (original detection model) outperforms the student model (pruned model), this suggests that the visual patterns learned by the teacher model are more accurate. Therefore, the goal is to have the student model simulate these patterns. Global Average Pooling (GAP) is used to evaluate the importance of the feature map of each channel, representing the attention information of each channel, and these attention data are regarded as knowledge. Both the student and the teacher calculate the attention information from their respective feature maps. Then, the teacher guides the student to learn and acquire this attention information. In this way, the student can gain insight into the attention details of the teacher's specific channels, thereby improving its performance. For simplicity, channel distillation is only applied between networks with reduced spatial resolution. In addition, if the number of channels does not match, the FitNets method is adopted to scale the dimension using 1 × 1 convolution. First, the student's feature map is upscaled to match the number of channels in the teacher model, and then channel distillation is performed. The definition formula of channel distillation (CD) loss is as follows:

[0165] (9)

[0166] In the provided formula (9), CD(s,t) represents the CD loss between the student and the teacher, represents the weight of the jth channel of the i-th student sample, The weight of the jth channel of the i-th sample of the teacher is represented by c, which represents the number of channels, and n represents the number of samples.

[0167] After performing channel distillation on different channels, guided knowledge distillation (GKD) is performed following the softmax prediction. Fig.14 The structure of the distillation process is illustrated.

[0168] GKD is designed based on the basic principle of knowledge distillation (KD). The core concept of KD involves calculating the prediction distribution between the teacher model and the student model. By gradually minimizing the difference between these two distributions, the output distribution of the student model becomes similar to the output distribution of the teacher model. The calculation formula of KD is as follows:

[0169] (10)

[0170] (11)

[0171] When using the softmax function to calculate the probability distribution, p represents the probability distribution calculated by the logit function, T represents the temperature parameter, which controls the "smoothness" of the output probability distribution. Using a higher T value will make the calculated probability distribution softer or smoother. n refers to the batch size, and KD (s, t) represents and The average Kullback-Leibler divergence (KL) between .

[0172] Although the teacher network is more accurate than the student network, the teacher model still has some prediction errors during batch processing, which causes significant problems. When the teacher makes incorrect predictions, this incorrect knowledge is passed on to the student, worsening the student's performance. To solve this problem, an improved version of knowledge distillation (KD) called guided knowledge distillation (GKD) was developed. This method ensures that only the positive prediction distribution from the teacher is passed to the student, while the negative predictions are ignored. The method only involves backpropagating the KD loss on the samples correctly predicted by the teacher network, while ignoring the errors. The definition formula of GKD is as follows:

[0173] (12)

[0174] In this formula, represents the probability distribution predicted by the teacher model for the i-th sample, represents the probability distribution predicted by the student model for the i-th sample. I represents an indicator function that is equal to 1 when the teacher's output matches the true label, expressed as , otherwise 0. KL divergence is used to measure the difference between two probability distributions. Suppose a batch contains n samples, of which the teacher model correctly predicts n1 samples. In this case, the GKD (Guided Knowledge Distillation) loss only calculates these n1 correctly predicted samples, and only considers the samples correctly predicted by the teacher model, avoiding the influence of the teacher model's incorrect predictions during the student model learning process.

[0175] After the above enhancements are made to the network model, channel pruning is performed, followed by channel distillation to obtain the final OD-YOLOV7-tiny network model (i.e., distillation model). The whole process is as follows Fig.15 shown.

[0176] S70: Deploy the distillation model after knowledge transfer on mobile devices such as drones to achieve real-time detection of insulator defects.

[0177] In this embodiment, it is verified through experiments that the optimized model is superior to the existing target detection algorithm in terms of detection speed, accuracy and model size.

[0178] Specifically, training was performed on the original YOLOv7-tiny model and the improved version of the YOLOv7-tiny model as well as the teacher model YOLOv7. The resolution of the training images was set to 640 × 640 pixels. The batch size was configured to 16, with a total of 300 training iterations, the Adam optimizer was selected, and the initial learning rate was 0.001. The original dataset used in this study includes two types of defects: insulator damage and insulator flashover. The dataset includes insulator defects of different sizes. In order to improve the generalization of the model, the images of the original dataset are enhanced by random rotation, flipping, brightness and contrast enhancement. The expanded dataset contains 2,800 different images.

[0179] In terms of model speed, the inference speed of the original YOLOv7-tiny detection model is 1.25 ms, while the inference speed of the final OD-YOLOv7-tiny model is 0.78 ms, an increase of 37.6%, which significantly improves the speed. In terms of accuracy, the accuracy of the OD-YOLOv7-tiny model is 92.67% and 92.32%, respectively, which is 3.9% and 5.3% higher than the original YOLOv7-tiny model. These enhancements in model size and speed were achieved without affecting accuracy.

[0180] In summary, the real-time insulator defect detection method based on channel pruning and channel distillation provided in this embodiment adopts an improved deep learning network architecture to detect insulator defects, and reduces the weight of the model through channel pruning and channel knowledge distillation, so that it can be deployed on mobile devices such as drones.

[0181] The contributions of this technology are mainly reflected in the following aspects:

[0182] 1. Enhanced feature fusion network: An OD-SlimNeck network architecture for drone detection of insulator defects is proposed. This architecture enhances the detection capability of small defects and effectively avoids the loss of important details. By integrating multi-scale features and making full use of local and global characteristics, the network's mapping representation of features is strengthened, thereby improving the overall detection performance. In addition, full-dimensional dynamic convolution allows the network to extract regions of interest from complex backgrounds and images with more interference, focusing on key targets and further improving the performance of the model;

[0183] 2. Compress the model through channel pruning: The improved model is compressed by channel pruning. Before pruning, the model is first trained for sparsity to ensure that the model performance does not drop significantly within a certain pruning limit. By pruning the channel, the number of parameters and the amount of calculation are reduced, a lighter network is achieved, and its effectiveness is fully evaluated;

[0184] 3. Channel distillation to improve accuracy: The YOLOv7 model is used as the original detection model to perform structural improvement, channel pruning, and channel distillation in sequence. This method improves the accuracy of insulator defect detection without increasing the number of parameters, provides a feasible solution for deploying insulator defect detection models on mobile devices such as drones, and ensures accurate identification of insulator defects.

[0185] Example 2

[0186] This embodiment provides a detection system that can execute the method provided by any embodiment of the present invention and has functional modules and beneficial effects corresponding to the execution method.

[0187] On the basis of the above-mentioned embodiments, for the features not explained in this embodiment, the explanations in other embodiments are adopted and will not be repeated here.

[0188] The detection system provided in this embodiment includes:

[0189] drones for image acquisition of insulators along high-voltage transmission lines;

[0190] An image preprocessing module is used to preprocess the collected images, including image denoising, enhancement and cropping, to highlight the defect features of the insulator;

[0191] The original detection module is used to input the preprocessed image into the original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image;

[0192] A detection model optimization module, used for optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model;

[0193] A channel pruning module, used for performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure and number of parameters;

[0194] A channel distillation module, used to transfer the knowledge of the original detection model to the pruned model by means of channel distillation, so as to obtain a distilled model that maintains the detection accuracy of the pruned model;

[0195] The mobile device deployment module is used to deploy the distillation model after knowledge migration on the mobile device to realize real-time detection of insulator defects.

[0196] Optionally, the detection model optimization module includes a basic model unit for providing an existing YOLOv7-tiny model, a backbone network replacement unit for adopting a FasterNet architecture as a backbone network, a neck replacement unit for replacing a neck part, and a decoupled detection head.

[0197] Optionally, the neck replacement unit is used to:

[0198] Build the GS bottleneck module and integrate the GS bottleneck module on top of GSConv;

[0199] Replace the neck of the existing YOLOv7-tiny model with the GSConv module after integrating the GS bottleneck module.

[0200] Optionally, the neck replacement unit is used to:

[0201] Construct OD-Slimneck module based on slimNeck module and ODConv module;

[0202] Replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module.

[0203] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concept of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A real-time detection method for insulator defects based on channel pruning and channel distillation, characterized in that: include: S10: Using drones to collect images of insulators along high-voltage transmission lines; S20: preprocessing the collected images, including image denoising, enhancement, and cropping, to highlight the defect features of the insulator; S30: inputting the preprocessed image into an original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image; S40: Optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model; S50: performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure, and number of parameters of the model; S60: Migrating the knowledge of the original detection model to the pruned model by channel distillation to obtain a distilled model that maintains the detection accuracy of the pruned model; S70: Deploy the distillation model after knowledge migration on a mobile device to achieve real-time detection of insulator defects.

2. The real-time detection method for insulator defects based on channel pruning and channel distillation according to claim 1 is characterized in that: The original detection model is the existing YOLOv7-tiny model, and the lightweight detection model is the improved YOLOv7-tiny model.

3. The real-time detection method for insulator defects based on channel pruning and channel distillation according to claim 2 is characterized in that: Step S40 is specifically as follows: Based on the existing YOLOv7-tiny model, the FasterNet architecture is used as the backbone network, the neck part is replaced, and a decoupled detection head is used for detection to obtain an improved YOLOv7-tiny model with higher detection accuracy and efficiency.

4. The real-time detection method for insulator defects based on channel pruning and channel distillation according to claim 3 is characterized in that: The neck portion is replaced, comprising: Build the GS bottleneck module and integrate the GS bottleneck module on top of GSConv; Replace the neck of the existing YOLOv7-tiny model with the GSConv module after integrating the GS bottleneck module.

5. The real-time detection method for insulator defects based on channel pruning and channel distillation according to claim 3 is characterized in that: The neck portion is replaced, comprising: Construct OD-Slimneck module based on slimNeck module and ODConv module; Replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module.

6. The real-time detection method for insulator defects based on channel pruning and channel distillation according to claim 1 is characterized in that: The channel pruning includes: pruning based on the sparsity of the convolution kernel to simultaneously maintain the detection accuracy and detection speed of the model. The pruned model maintains or improves the detection accuracy and detection efficiency while reducing the size and computational complexity.

7. A detection system for executing the method according to any one of claims 1 to 6, characterized in that: include: drones for image acquisition of insulators along high-voltage transmission lines; An image preprocessing module is used to preprocess the collected images, including image denoising, enhancement and cropping, to highlight the defect features of the insulator; The original detection module is used to input the preprocessed image into the original detection model; wherein the original detection model is used to identify and detect insulator defects according to the input image; A detection model optimization module, used for optimizing the backbone network, the neck part, and the detection head of the original detection model to obtain a lightweight detection model with lower model complexity and calculation amount than the original detection model; A channel pruning module, used for performing channel pruning on the lightweight detection model to obtain a pruned model with reduced size, structure and number of parameters; A channel distillation module, used to transfer the knowledge of the original detection model to the pruned model by means of channel distillation, so as to obtain a distilled model that maintains the detection accuracy of the pruned model; The mobile device deployment module is used to deploy the distillation model after knowledge migration on the mobile device to realize real-time detection of insulator defects.

8. The detection system according to claim 7, characterized in that: The detection model optimization module includes a basic model unit for providing an existing YOLOv7-tiny model, a backbone network replacement unit for adopting a FasterNet architecture as a backbone network, a neck replacement unit for replacing a neck part, and a decoupled detection head.

9. The detection system according to claim 8, characterized in that: The neck replacement unit is used to: Build the GS bottleneck module and integrate the GS bottleneck module on top of GSConv; Replace the neck of the existing YOLOv7-tiny model with the GSConv module after integrating the GS bottleneck module.

10. The detection system according to claim 8, characterized in that: The neck replacement unit is used to: Construct OD-Slimneck module based on slimNeck module and ODConv module; Replace the neck of the existing YOLOv7-tiny model with the OD-Slimneck module.

Images