Insulator identification and string falling defect positioning method
By integrating CCA attention, Ghost, LFEM and ASFF modules into the YOLOv7-tiny neural network, and combining with the U-net model, the problem of insulator identification and string loss defect positioning in complex environments is solved, and a fast and accurate detection effect is achieved.
Patent Information
- Application Number
- CN202510393427.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-04
AI Technical Summary
The prior art is difficult to quickly and accurately identify insulators and locate string defects in complex environments. The manual inspection efficiency is low and the missed inspection is serious. The detection method based on computer vision is not ideal in complex backgrounds, especially in small sample conditions, which is difficult to meet the efficient and accurate detection needs.
The CCA attention submodule, Ghost submodule, LFEM module and ASFF module are introduced into the YOLOv7-tiny neural network. Combined with the U-net neural network, the insulator identification and string loss defect positioning are achieved through the combination of the improved YOLOv7-tiny model and the U-net model.
It improves the accuracy of insulator identification and the positioning accuracy of string-loss defects, solves the problems of complex background and small target identification, and realizes fast and accurate insulator identification and string-loss defect detection.
Smart Images
Figure CN120259261A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of visual detection, and particularly to a method for insulator recognition and string-drop defect localization. Background Art
[0002] Insulators are key components in power transmission systems, whose main functions are to provide mechanical support and electrical insulation, and their performance directly affects the safety and stability of power systems. With the rapid growth of the economy, the power demand continues to rise, the scale of the power grid is constantly expanding, and the length and structural complexity of transmission lines also increase accordingly, resulting in a more complex and diverse operating environment for insulators. When exposed to the natural environment for a long time, insulators are affected by multiple factors such as mechanical stress, electrical load, environmental pollution, and climate change, and thus various defects may occur, among which the string-drop defect is particularly serious. Once a string-drop defect occurs, it may lead to major accidents such as line short circuits and power outages, causing significant losses to the social economy and having a serious impact on people's production and life. Therefore, timely and accurately identifying insulators and locating their string-drop defects is crucial for ensuring the reliable operation of power systems. Currently, the main methods for insulator defect detection include manual inspection and computer vision-based detection techniques.
[0003] As a traditional means of insulator defect detection, manual inspection can identify defects to a certain extent, but it has significant limitations. First of all, the efficiency of manual inspection is low, it is difficult to achieve comprehensive and timely detection, and due to the high labor intensity, the detection cost is relatively high. Secondly, the accuracy of manual detection highly depends on the experience, skill level, and working status of the inspectors, and it is prone to missed detections or false detections, and it is difficult to guarantee the reliability and consistency of the detection results. On the other hand, the direct detection effect of computer vision-based detection methods on insulator defects is not ideal under complex environments and small sample conditions. In particular, it is easy to misjudge the connection between the insulator and the clamp as a string-drop defect, reducing the detection accuracy. In addition, the aerial images taken by drones during power line inspections have a complex background. Affected by factors such as lighting conditions, shooting angles, and shooting distances, the scale of insulators in the images varies greatly, and the insulator strings are easily blocked, further increasing the detection difficulty. At the same time, the defect area of insulators is usually much smaller than their own size, which poses higher technical requirements for defect detection. The existing visual detection methods still have deficiencies in the accuracy and real-time performance of rapid insulator recognition and accurate string-drop defect localization. Especially under actual working conditions with complex backgrounds and multiple interference factors, it is difficult to meet the requirements of efficient and accurate detection of string-drop defects of insulators during power line inspections. Summary of the Invention
[0004] To overcome the deficiencies of the prior art, the purpose of the present invention is to provide a method for insulator identification and string-drop defect localization, which can quickly and accurately identify insulators and precisely locate their string-drop defects.
[0005] To achieve the above purpose, the present invention provides the following solutions:
[0006] A method for insulator identification and string-drop defect localization, comprising:
[0007] Introduce a CCA attention sub-module into the ELAN module of the YOLOv7-tiny neural network, replace the CBS module in the ELAN module with a Ghost sub-module, add an LFEM module between the backbone network and the neck network of the YOLOv7-tiny neural network, and introduce an ASFF module at the output end of the neck network to obtain an improved YOLOv7-tiny model;
[0008] Collect aerial images of the drone, and obtain an insulator annotation image set and an insulator string-drop defect annotation image set according to the aerial images;
[0009] Input the insulator annotation image set into the improved YOLOv7-tiny model for feature extraction and training to obtain an insulator identification model;
[0010] Input the insulator string-drop defect annotation image set into a preset U-net neural network detection model for feature extraction and training to obtain a string-drop defect identification model;
[0011] Embed the insulator identification model and the string-drop defect identification model into the drone to obtain a string-drop defect localization device;
[0012] Use the string-drop defect localization device to collect on-site images of the target detection line, use the insulator identification model to identify and crop the insulator area in the on-site images to obtain insulator cropped images, and use the string-drop defect identification model to identify the string state of the insulator cropped images to obtain string-drop defect detection results.
[0013] Preferably, collecting aerial images of the drone and obtaining an insulator annotation image set and an insulator string-drop defect annotation image set according to the aerial images includes:
[0014] Adjust the resolution of the aerial images to 640×640;
[0015] Use the LabelImg tool to annotate the insulator devices in the aerial images to obtain the insulator annotation image set;
[0016] Use the LabelMe tool to label the insulator string dropping area in the aerial image to obtain the insulator string dropping defect labeled image set.
[0017] Preferably, the working process of the LFEM module includes:
[0018] Use three global average poolings to reduce the width and height of the input feature map to be processed to 1 / 2, 1 / 4, and 1 / 8 of the original image size respectively, obtaining the first compressed feature map, the second compressed feature map, and the third compressed feature map;
[0019] Use 1×1 convolution to reduce the first compressed feature map, the second compressed feature map, and the third compressed feature map to 1 / 16 of the output channel number, obtaining the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map;
[0020] Use the bilinear interpolation upsampling algorithm to restore the sizes of the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map to the original image size, obtaining the first restored feature map, the second restored feature map, and the third restored feature map;
[0021] Perform splicing and 1×1 convolution operations on the first restored feature map, the second restored feature map, the third restored feature map, and the feature map to be processed to obtain the insulator feature.
[0022] Preferably, the operations of the ASFF module include: identity scaling, weighted fusion coefficient calculation, and adaptive fusion.
[0023] Preferably, the image ratio of the insulator labeled image set and the insulator string dropping defect labeled image set participating in the training process and the testing process during model training is 7:3.
[0024] Preferably, the identity scaling includes: 1 / 2 downsampling, 1 / 4 downsampling, and upsampling.
[0025] Preferably, the calculation formula of the weighted fusion coefficient includes:
[0026]
[0027] and
[0028]
[0029] where are the first learning weight, the second learning weight, and the third learning weight respectively; are the first feature map, the second feature map, and the third feature map respectively; l is the dimension of the adaptive fusion feature layer; (i,j) represents a vector.
[0030] Preferably, the expression of the adaptive fusion is as follows:
[0031]
[0032] Wherein, is the updated feature map; are the first output feature map, the second output feature map, and the third output feature map, respectively.
[0033] The present invention discloses the following technical effects:
[0034] The present invention provides a method for insulator recognition and string-drop defect location. By integrating the CCA attention sub-module, Ghost sub-module, LFEM module, and ASFF module into the traditional YOLOv7-tiny neural network, the problem that the traditional YOLOv7-tiny neural network has poor detection effect on insulator defects directly in complex environments and small-sample cases is solved, and the extraction of the position and channel information of the insulator, the adaptive learning of the fusion spatial weights of feature maps at each scale, the lateral expansion of the network width, the increase of the receptive field, and the improvement of the network's ability to extract weak semantic features are realized; by combining the improved YOLOv7-tiny model and the U-net neural network detection model, the negative impact caused by complex backgrounds and small proportions of insulator regions in direct string-drop recognition on the original image is solved, and the recognition of local images of insulators and the further recognition of string-drop defects are realized. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts.
[0036] Figure 1 is a schematic diagram of the insulator recognition and string-drop defect location process provided by an embodiment of the present invention;
[0037] Figure 2 is a structural diagram of the improved YOLOv7-tiny model provided by an embodiment of the present invention;
[0038] Figure 3 is a structural diagram of the CCA module provided by an embodiment of the present invention;
[0039] Figure 4 is a structural diagram of the ELAN-C module provided by an embodiment of the present invention;
[0040] Figure 5The structural diagram of the LFEM module provided by the embodiment of the present invention;
[0041] Figure 6 The structural diagram of the ASFF module provided by the embodiment of the present invention. Detailed implementation manners
[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0043] The purpose of the present invention is to provide a method for insulator identification and string-drop defect location, which can quickly and accurately identify insulators and precisely locate their string-drop defects.
[0044] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.
[0045] Figure 1 The schematic diagram of the insulator identification and string-drop defect location process provided by the embodiment of the present invention. As Figure 1 shown, the present invention provides a method for insulator identification and string-drop defect location, including:
[0046] Step 100: Introduce a CCA attention sub-module into the ELAN module of the YOLOv7-tiny neural network, use the Ghost sub-module to replace the CBS module in the ELAN module, add an LFEM module between the backbone network and the neck network of the YOLOv7-tiny neural network, and introduce an ASFF module at the output end of the neck network to obtain an improved YOLOv7-tiny model;
[0047] Step 200: Collect aerial images of the drone, and obtain an insulator annotation image set and an insulator string-drop defect annotation image set according to the aerial images;
[0048] Step 300: Input the insulator annotation image set into the improved YOLOv7-tiny model for feature extraction and training to obtain an insulator identification model;
[0049] Step 400: Input the insulator string-drop defect annotation image set into a preset U-net neural network detection model for feature extraction and training to obtain a string-drop defect identification model;
[0050] Step 500: Embed the insulator identification model and the string-drop defect identification model into the drone to obtain a string-drop defect location device;
[0051] Step 600: Use the string-drop defect location device to collect the on-site image of the target detection line, use the insulator recognition model to recognize and crop the insulator area in the on-site image to obtain the insulator cropped image, and use the string-drop defect recognition model to recognize the string state of the insulator cropped image to obtain the string-drop defect detection result.
[0052] Preferably, collect the aerial images of the unmanned aerial vehicle, and obtain the insulator annotation image set and the insulator string-drop defect annotation image set according to the aerial images, including:
[0053] Adjust the resolution of the aerial image to 640×640;
[0054] Use the LabelImg tool to annotate the insulator devices in the aerial image to obtain the insulator annotation image set;
[0055] Use the LabelMe tool to annotate the insulator string-drop areas in the aerial image to obtain the insulator string-drop defect annotation image set.
[0056] Furthermore, the working process of the LFEM module includes:
[0057] Use three global average poolings to reduce the width and height of the input feature map to be processed to 1 / 2, 1 / 4, and 1 / 8 of the original image size respectively, to obtain the first compressed feature map, the second compressed feature map, and the third compressed feature map;
[0058] Use 1×1 convolution to reduce the first compressed feature map, the second compressed feature map, and the third compressed feature map to 1 / 16 of the output channel number, to obtain the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map;
[0059] Use the bilinear interpolation upsampling algorithm to restore the sizes of the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map to the original image size, to obtain the first restored feature map, the second restored feature map, and the third restored feature map;
[0060] Perform splicing and 1×1 convolution operations on the first restored feature map, the second restored feature map, the third restored feature map, and the feature map to be processed to obtain the insulator feature.
[0061] Specifically, the operations of the ASFF module include: identity scaling, weighted fusion coefficient calculation, and adaptive fusion.
[0062] Optionally, the image ratio of the insulator annotation image set and the insulator string-drop defect annotation image set participating in the training process and the testing process during the model training is 7:3.
[0063] Preferably, the identity scaling includes: 1 / 2 downsampling, 1 / 4 downsampling, and upsampling.
[0064] Specifically, the calculation formula of the weighted fusion coefficient includes:
[0065]
[0066] and
[0067]
[0068] wherein, are the first learning weight, the second learning weight, and the third learning weight respectively; are the first feature map, the second feature map, and the third feature map respectively; l is the dimension of the adaptive fusion feature layer; (i, j) represents a vector.
[0069] Furthermore, the expression of the adaptive fusion is:
[0070]
[0071] wherein, is the updated feature map; are the first output feature map, the second output feature map, and the third output feature map respectively.
[0072] Preferably, a channel attention mechanism and a coordinate attention mechanism are introduced into the backbone network ELAN module to extract the position and channel information of the insulator. Considering the complex background of the aerial images of transmission lines and the wide distribution of insulators, it is necessary to adopt a feature extraction method that can capture long-range dependencies. Although the channel attention mechanism can significantly improve the performance of deep learning algorithms, it often ignores the position information, which is crucial for generating spatially selective attention maps. Methods such as Squeeze-and-Excitation (SE) and Convolutional Block Attention Module (CBAM) obtain position information by reducing the number of channels and using large-size convolutions, but they ignore the position information and the spatial structure, resulting in difficulty in retaining the spatial position information of the insulator target in the channel. The Coordinate Attention mechanism CA decomposes the channel attention into two one-dimensional feature encoding processes, aggregates features along two spatial directions. In this way, the CA attention mechanism can capture long-range dependencies while retaining accurate position information. Compared with other attention mechanisms, the advantage of the CA module is that it is flexible and lightweight enough. Aiming at the complex background of aerial images and the more difficult multi-scale feature information localization of insulators, in this embodiment, based on the channel attention mechanism and the CA (Coordinate Attention) mechanism, a novel Channel Coordinate Attention Module CCA is designed (as Figure 3 shown), that is, to extract the position and channel information of the insulator simultaneously. Combining the CA attention mechanism structureFigure 3 , and its implementation process is as follows: First, use a one-dimensional global pooling kernel to encode each channel along two directions, horizontal X and vertical Y, and aggregate them into two separate directional attention feature maps. Then, concatenate the feature maps of the two directions that obtain the global receptive field, perform a transformation operation using a 1×1 convolution, and input a non-linear activation function to obtain an intermediate feature map of the spatial information in the horizontal and vertical directions. Next, divide the intermediate feature map into two separate tensors, perform a 1×1 convolution according to the original height and width to obtain a feature map with the same number of channels as the input feature map, and obtain the attention weights of the feature map in the height and width directions after passing through the sigmoid activation function. Finally, perform a multiplication weighting calculation on the original feature map transformed by the channel attention mechanism to obtain an output feature map with attention weights. Integrate this module into the ELAN module (80×80, 40×40) of the backbone network to form an ELAN-C module.
[0073] Specifically, the structure of ELAN-C is as Figure 4 shown. By adding the CCA attention mechanism to the ELAN module, the feature extraction and target localization capabilities of the backbone network can be improved with only a slight increase in computational complexity. In order to deploy the model to edge devices for real-time identification of insulators in aerial images, a lightweight Ghost convolution module is introduced into the original ELAN module to replace the traditional CBS convolution module, and a lightweight and efficient aggregation network is constructed. The Ghost module decomposes the ordinary convolution into two steps. First, perform traditional convolution calculations with a strictly controlled number, generate Ghost feature maps through a series of simple linear transformations, and then concatenate all the Ghost feature maps together to generate the final feature map, thereby eliminating the redundancy of the feature map. The ELAN-C module consists of several Ghost convolution modules and CCA attention modules, and can learn and converge more effectively by controlling the shortest and longest gradient paths.
[0074] Preferably, a low-level feature enhancement module is added between the effective feature layer of the backbone network and the neck network to horizontally expand the network width and increase the receptive field, and improve the network's ability to extract weak semantic features. In order to strengthen the extraction of low-level features in the backbone network and better adapt to the requirements of the transmission line insulator detection task, in this embodiment, a low-level feature enhancement module (LFEM) is designed based on the pyramid pooling module (PPM). LFEM uses a multi-branch structure aggregated by feature pyramids of different scales, forms a multi-channel feature map through cascading, horizontally expands the network width and increases the receptive field, and thus improves the network's ability to extract weak semantic features.
[0075] Furthermore, the network structure of LFEM is as Figure 5As shown in the figure. First, three global average poolings are used to reduce the width and height of the feature map to 1 / 2, 1 / 4, and 1 / 8 of the original image size respectively to obtain feature information at different scales, and then 1×1 convolution is used to reduce the number of channels to 1 / 16 of the output channel number; then, the bilinear interpolation upsampling algorithm is used to restore the image to the input size; finally, the feature maps of the four branches are concatenated, and the number of channels is adjusted to the output channel number through 1×1 convolution, so as to extract finer insulator features through pooling operations at different scales.
[0076] Preferably, an adaptive spatial feature fusion module is introduced at the output end of the neck network to adaptively learn the fusion spatial weights of feature maps at each scale, achieve feature fusion spatially, and ensure the consistency of features at different scales. In the insulator recognition task, feature layers at different scales contain different semantic information. However, in aerial images taken by drones, more feature information is concentrated in the high-resolution layer and is easily lost during the feature fusion process. Aiming at the problem of low accuracy in recognizing small targets and multiple targets in aerial images with complex backgrounds, an adaptive spatial feature fusion (ASFF) module is constructed in the neck of the YOLOv7-tiny model. The ASFF network structure diagram is as Figure 6 shown.
[0077] Furthermore, the adaptive spatial feature fusion module is a feature adaptive fusion strategy that adaptively learns the fusion spatial weights of feature maps at each scale, achieves feature fusion spatially, and ensures the consistency of features at different scales. The adaptive spatial feature fusion module includes identity scaling, weighted fusion coefficients, and adaptive fusion; identity scaling includes 1 / 2 downsampling, 1 / 4 downsampling, and upsampling; the weighted fusion coefficients (α l , β l , γ l ) are adaptively learned by the network and shared across all channels; adaptive fusion is achieved by fusing the first three feature maps. The calculation formula for the weighted fusion coefficients is as follows:
[0078]
[0079]
[0080] where,
[0081] Even further, the calculation formula for the output feature map of adaptive fusion is:
[0082]
[0083] Specifically, this embodiment incorporates an Adaptive Spatial Feature Fusion (ASFF) module. Combining low-level feature information with dimensions of 80×80×256 and 40×40×512 with high-level detailed feature information with dimensions of 20×20×1048 can fuse rich cross-scale semantic information, thereby making the recognition of small targets and multi-targets more accurate.
[0084] Specifically, the drone is equipped with a trained network model to collect images according to the inspection task. The collected images are first input into the improved YOLOv7-tiny model for insulator recognition. The detected insulators are cropped out by YOLOv7-tiny and input into the U-net network model. U-net performs insulator string-drop defect detection and outputs the results of insulator recognition and string-drop defect detection.
[0085] The beneficial effects of the present invention are as follows:
[0086] By incorporating the CCA attention sub-module, Ghost sub-module, LFEM module, and ASFF module into the traditional YOLOv7-tiny neural network, the present invention realizes the extraction of the position and channel information of the insulator and the adaptive learning of the fusion spatial weights of feature maps at each scale, horizontally expands the network width and increases the receptive field, and improves the ability of the network to extract weak semantic features; through the combination of the improved YOLOv7-tiny model and the U-net neural network detection model, the negative impact caused by the complex background and the small proportion of the insulator area in the direct string-drop recognition on the original image is avoided, and the detection accuracy of the model is improved.
[0087] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.
[0088] Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.
Claims
1. An insulator recognition and string-drop defect location method, characterized in that, Including: Introduce the CCA attention sub-module in the ELAN module of the YOLOv7-tiny neural network, replace the CBS module in the ELAN module with the Ghost sub-module, add the LFEM module between the backbone network and the neck network of the YOLOv7-tiny neural network, and introduce the ASFF module at the output end of the neck network to obtain an improved YOLOv7-tiny model; Collect the aerial images of the drone, and obtain the insulator annotation image set and the insulator string-drop defect annotation image set according to the aerial images; Input the insulator annotation image set into the improved YOLOv7-tiny model for feature extraction and training to obtain an insulator recognition model; Input the insulator string-drop defect annotation image set into a preset U-net neural network detection model for feature extraction and training to obtain a string-drop defect recognition model; Embed the insulator recognition model and the string-drop defect recognition model into the drone to obtain a string-drop defect positioning device; Use the string-drop defect positioning device to collect the on-site images of the target detection line, use the insulator recognition model to identify and crop the insulator area in the on-site images to obtain insulator cropped images, and use the string-drop defect recognition model to identify the string state of the insulator cropped images to obtain the string-drop defect detection results.
2. The method for identifying insulators and locating the defect of string dropping according to claim 1, wherein, Collect the aerial images of the drone, and obtain the insulator annotation image set and the insulator string-drop defect annotation image set according to the aerial images, including: Adjust the resolution of the aerial images to 640×640; Use the LabelImg tool to annotate the insulator devices in the aerial images to obtain the insulator annotation image set; Use the LabelMe tool to annotate the insulator string-drop areas in the aerial images to obtain the insulator string-drop defect annotation image set.
3. The insulator identification and string-drop defect location method according to claim 1, characterized in that The working process of the LFEM module includes: Use three global average poolings to reduce the width and height of the input feature map to be processed to 1 / 2, 1 / 4, and 1 / 8 of the original image size respectively, to obtain the first compressed feature map, the second compressed feature map, and the third compressed feature map; Use 1×1 convolution to reduce the first compressed feature map, the second compressed feature map, and the third compressed feature map to 1 / 16 of the output channel number to obtain the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map; Use the bilinear interpolation upsampling algorithm to restore the sizes of the first convolutional feature map, the second convolutional feature map, and the third convolutional feature map to the original image size to obtain the first restored feature map, the second restored feature map, and the third restored feature map; Perform splicing and 1×1 convolution operations on the first restored feature map, the second restored feature map, the third restored feature map, and the feature map to be processed to obtain insulator features.
4. The insulator identification and string-drop defect location method according to claim 1, wherein The operations of the ASFF module include: identity scaling, weighted fusion coefficient calculation, and adaptive fusion.
5. The method for insulator identification and string-drop defect location according to claim 2, wherein The image ratio of the insulator annotation image set and the insulator string-drop defect annotation image set participating in the training process and the testing process during model training is 7:
3.
6. The insulator identification and string-drop defect location method according to claim 4, wherein, The identity scaling includes: 1 / 2 downsampling, 1 / 4 downsampling, and upsampling.
7. A method for identifying insulators and locating the defect of string dropping according to claim 4, characterized in that, The calculation formula of the weighted fusion coefficient includes: and Among them, are the first learning weight, the second learning weight, and the third learning weight respectively; are the first feature map, the second feature map, and the third feature map respectively; l is the dimension of the adaptive fusion feature layer; (i, j) represents a vector.
8. A method for insulator identification and string-drop defect location according to claim 7, characterized in that, The expression of the adaptive fusion is: Among them, is the updated feature map; are the first output feature map, the second output feature map, and the third output feature map respectively.