An end-to-end power transmission line detection method based on Yolov7

Abstract

The application discloses an end-to-end power transmission line detection method based on Yolov7, comprising the following steps: S1: collecting power transmission line images and constructing a data set; S2: through an input module, input images are scaled to a uniform size of network input; S3: feature group learning is adopted to improve the feature extraction capability of the network; S4: a network bottom-up path is established to fuse deep features and shallow features and further improve the ability of extracting detailed information; and S5: a Rep structure outputs different scale features, adjusts the number of image channels, and calculates faster and more flexibly. Compared with a traditional power transmission line detection algorithm, the application can overcome the defects of low detection accuracy and unstable detection in a complex background, and successfully solves the problem of detection real-time in real-time inspection.

Description

Technical Field

[0001] This invention relates to the field of computer vision target detection, and more particularly to an end-to-end transmission line detection method based on Yolov7. Background Technology

[0002] Overhead transmission lines are commonly used for power transmission. These lines are widely laid across various regions, sometimes traversing harsh environments (scorching deserts, mountains, dense forests, and bodies of water). To ensure efficient and reliable transmission of high-voltage power, these lines require regular inspections to detect and maintain faults early. Fault detection and location along transmission lines are crucial. During power system inspections, the complex backgrounds and long distances of transmission lines present numerous challenges, including inaccurate detection and slow inspection speeds. Traditionally, power company personnel inspect high-voltage transmission lines visually using small helicopters or manually controlled flying robots. However, the main problems with existing technologies are: Since inspections rely primarily on human vision, the transmission lines may blend into the background in certain natural environments, or become blurred and difficult to distinguish over time. This creates a high risk for inspectors, as excessive visual strain can lead to collisions between the helicopter or inspection robot and the high-voltage transmission lines, resulting in injuries or fatalities. Therefore, using machine vision to replace human eye observation is the main direction of technological improvement.

[0003] Traditional object detection algorithms primarily originate from machine learning. This involves designing features to describe object characteristics and then integrating them with a classifier. In recent years, deep learning (DL), especially convolutional neural networks (CNNs), has made significant progress and promising advancements, thus receiving considerable attention on the global stage of computer vision research. Currently, deep learning applications in object detection are mainly divided into two types: two-step object detection, which involves first identifying candidate regions and then classifying them. Classic object detection networks include R-CNN, SPP-net, Fast R-CNN, and Faster R-CNN. These algorithms have high accuracy but poor real-time performance and are ineffective at detecting small objects, making them suitable for high-precision object detection. End-to-end object detection: This uses a single network to detect objects in one step. The input image is taken as input, and the output includes the types and locations of objects. Classic object detection networks include YOLO and SSD. These algorithms have high real-time performance but low accuracy in detecting groups of objects and small objects, making them suitable for real-time object detection. Implementing end-to-end object detection algorithms for power transmission line inspection is a primary goal for engineers. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an end-to-end transmission line detection method based on YOLOv7.

[0005] One technical solution to achieve the above objective is: an end-to-end transmission line detection method based on Yolov7, comprising the following steps:

[0006] S1: Collect images of power transmission lines and construct a dataset;

[0007] S2: The Input module scales the input image to the uniform size of the network input.

[0008] S3: Employ feature group learning to enhance the network's feature extraction capabilities;

[0009] S4: Establish a bottom-up path for the network, integrate deep and shallow features, and further enhance the ability to extract detailed information;

[0010] S5: The Rep structure outputs features at different scales, adjusts the number of image channels, and makes calculation output faster and more flexible.

[0011] Furthermore, the specific steps of step S1 are as follows:

[0012] S101: A transmission line dataset is constructed by combining visible light transmission line images and transmission line images collected from the internet.

[0013] Furthermore, the specific steps of step S2 are as follows:

[0014] S201: Data augmentation of the image is performed using the Mosaic method, which stitches together nine different images as a new input image to enhance the robustness of the network.

[0015] S202: Use linear interpolation to scale all input image sizes.

[0016] Furthermore, in step S202, all input images are scaled to 640*640*3.

[0017] Furthermore, the specific methods for learning various features in step S3 are as follows:

[0018] S301: Employs an efficient aggregation network structure, using four layers of CBS convolutional blocks, E-ELAN convolutions, and MPConv convolutional layers to extract image features at different scales.

[0019] Furthermore, to improve the efficient network structure, CBS consists of Conv+BN+SiLU. First, there is a CBS convolutional block with a kernel size of 3 and a stride of 1, followed by a convolutional block with a kernel size of 3 and a stride of 2. Two of each type of CBS convolutional block are alternately connected. After four CBS, the feature map becomes 160*160*128.

[0020] Furthermore, E-ELAN is composed of multiple CBSs, whose input and output feature sizes remain unchanged. The number of channels changes in the first two CBSs, while the subsequent input and output channels remain consistent. The output of the last CBS is the required channel.

[0021] Furthermore, the MPConv convolutional layer has the same input and output channels. A Maxpool layer is added on top of the BConv layer to form two branches. The upper branch halves the image width and height through Maxpool and halves the image channels through the BConv layer. The lower branch halves the image channels through the first BConv layer and halves the image width and height through the second BConv layer. Finally, the Cat operation is used to fuse the features extracted from the upper and lower branches.

[0022] Furthermore, the specific steps for establishing the bottom-up path of the network in step S4 are as follows:

[0023] S401: Bottom-up fusion of deep and shallow features is a PAFPN structure. The CSP module in YOLOv5 is replaced with the ELAN-H module, and the downsampling is changed to the MP2 layer. For the 32x downsampled feature map C5 output by the backbone, it is then processed by SPPCSP, and the number of channels is reduced from 1024 to 512. First, it is fused with C4 and C3 in a top-down manner to obtain P3, P4 and P5. Then, it is fused with P4 and P5 in a bottom-up manner to achieve the joint fusion of deep and shallow features.

[0024] Furthermore, the specific steps of the Rep structure in step S5 are as follows:

[0025] For P3, P4, and P5 output by PAFPN, the number of channels is adjusted by RepConv, and finally a 1x1 convolution is used to predict the three parts: class, confidence, and anchor box.

[0026] This invention discloses an end-to-end transmission line detection method based on Yolov7. In terms of architecture, the ELAN module is a highly efficient network structure that, by controlling the shortest and longest gradient paths, enables the network to learn more features and exhibits stronger robustness. By improving the feature fusion of the E-ELAN feature extraction module through the residual structure in the Rep module, the network can simultaneously leverage the high performance advantages of multi-branch model training and the fast and memory-efficient characteristics of single-path model inference, thereby improving network performance. The Backbone module incorporates an efficient pyramid compression attention mechanism, which effectively extracts finer-grained multi-scale spatial feature information, enhancing the network's feature extraction capability. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0028] Figure 2 This is a diagram of the input image feature extraction network structure of the present invention;

[0029] Figure 3 This is a schematic diagram of the Backbone portion of the present invention;

[0030] Figure 4 This is a schematic diagram of the Head section structure of the present invention;

[0031] Figure 5 This is a schematic diagram of the ELAN-H module structure of the present invention;

[0032] Figure 6 This is a schematic diagram of the ELAN module structure of the present invention. Detailed Implementation

[0033] To better understand the technical solution of the present invention, detailed descriptions are provided below through specific embodiments:

[0034] Please see Figure 1 The above is an overall flowchart of the present invention. An end-to-end transmission line detection method based on Yolov7 according to the present invention includes the following steps:

[0035] S1: Collect images of transmission lines and construct a dataset, including:

[0036] S101: A transmission line dataset is constructed using visible light transmission line images created in collaboration with the Turkish Electricity Transmission Company and its research team, along with transmission line images collected from the internet.

[0037] S2: Input module, scales the input image to a uniform size for the network input, including:

[0038] S201: Image data augmentation is performed using the Mosaic method, which stitches together nine different images as a new input image to enhance the robustness of the network.

[0039] S202: Use linear interpolation to scale all input images to 640*640*3.

[0040] S3: Employs multiple feature group learning methods to enhance the network's feature extraction capabilities, including:

[0041] S301: Employs a high-efficiency aggregation network structure, extracting image features at different scales through four layers of CBS convolutional blocks, E-ELAN convolutions, and MPConv convolutional layers. The CBS layer primarily consists of Conv+BN+SiLU. First, a CBS convolutional block with a kernel size of 3 and a stride of 1 is formed, followed by another convolutional block with a kernel size of 3 and a stride of 2. These two types of CBS convolutional blocks are alternately connected, resulting in a feature map of 160*160*128 after four CBS layers. The E-ELAN layer consists of multiple CBS layers, maintaining constant input and output feature sizes. The number of channels changes in the first two CBS layers, while subsequent input and output channels remain consistent. The final CBS layer outputs the desired number of channels. The MPConv convolutional layer has the same input and output channels. A Maxpool layer is added to the BConv layer, forming two branches. The upper branch halves the image width and height using Maxpool, and halves the image channels using the BConv layer. The lower branch halves the image channels using the first BConv layer, halves the image width and height using the second BConv layer, and finally uses the Cat operation to fuse the features extracted from the upper and lower branches. The entire backbone layer consists of several BConv layers, E-ELAN layers, and MPConv convolutional layers that alternately halve the width and height, double the channels, and extract features.

[0042] S4: Establish a bottom-up path for the network, fusing deep and shallow features to further enhance the ability to extract detailed information, including:

[0043] S401: The bottom-up fusion of deep and shallow features follows a PAFPN structure, similar to Yolov4 and Yolov5. The difference lies in replacing the CSP module in Yolov5 with the ELAN-H module, and changing the downsampling to an MP2 layer. ELAN-H differs slightly from the ELAN in the Backbone in the number of Cats and the number of channels is halved. The Backbone's final output, a 32x downsampled feature map C5, is then processed through SPPCSP, reducing the number of channels from 1024 to 512. First, it is fused with C4 and C3 in a top-down manner to obtain P3, P4, and P5; then, it is fused with P4 and P5 in a bottom-up manner to achieve the joint fusion of deep and shallow features.

[0044] S5: The Rep structure outputs features at different scales, adjusts the number of image channels, and calculates outputs faster and more flexibly. This includes: for P3, P4, and P5 output by PAFPN, the number of channels is adjusted by RepConv, and finally a 1x1 convolution is used to predict the three parts: class, confidence, and anchor box.

[0045] Please see Figure 2 The specific steps for image preprocessing are as follows: First, images are filtered, filled, and integrated. After basic image processing, the LabelMe data annotation tool is used for annotation. The detection target is named Power Line, generating a ".xml" file for each image. Second, the ".xml" file for each image is converted to a corresponding ".txt" file. The data is then used as a training set (80%) and a test set (20%). The images corresponding to the ".txt" files are then input into the detection network, and the images are processed using linear interpolation to transform them to a 640*640*3 image size as the final input image.

[0046] Please see Figure 3The third step in the entire process employs an efficient aggregation network structure to enhance the network's feature extraction capabilities. It consists of several BConv convolutional layers, E-ELAN convolutional layers, and MPConv convolutional layers. The BConv layer comprises convolutional layers, batch normalization (BN) layers, and a LeakyReLU activation function, used to extract image features at different scales. The E-ELAN convolutional layer maintains the original ELAN design architecture, guiding computational blocks of different feature groups to learn more diverse features, improving the network's learning ability without disrupting the original gradient path. The MPConv convolutional layer adds a Maxpool layer to the BConv layer, forming two branches. The upper branch halves the image's width and height using Maxpool and halves the image channels using the BConv layer. The lower branch halves the image channels using the first BConv layer and halves the image's width and height using the second BConv layer. Finally, a Cat operation is used to fuse the features extracted by the upper and lower branches, improving the network's feature extraction capabilities.

[0047] Please see Figure 4 The S4 Head module of the entire process consists of a PathAggregation Feature Pyramid Network (PAFPN) structure. By introducing bottom-up paths, it makes it easier for lower-level information to be transmitted to higher levels, thereby achieving efficient fusion of features at different levels.

[0048] Please see Figure 5 This is an extended efficient long-range attention network (ELAN-H) in YOLOv7, which is an improvement on the efficient long-range attention network (ELAN). For example... Figure 5 As shown, compared to Figure 6 The ELAN network, specifically ELAN-H, uses group convolutions to increase the cardinality of features and combines features from different groups using shuffle and merge operations. This structure enhances feature extraction capabilities, improves parameter usage and computational efficiency, and improves the network's learning ability without disrupting the original gradient path, while saving significant computational resources during inference. However, this attention mechanism leads to a slight decrease in network performance.

[0049] Those skilled in the art should recognize that the above embodiments are merely illustrative of the present invention and are not intended to limit the present invention. Any variations or modifications to the above embodiments that are within the spirit and essence of the present invention will fall within the scope of the claims of the present invention.

Claims

1. A method for end-to-end transmission line detection based on Yolov7, characterized in that, Includes the following steps: S1: Collect images of power transmission lines and construct a dataset; S2: The Input module scales the input image to the uniform size of the network input. S3: Employ feature group learning to enhance the network's feature extraction capabilities; S4: Establish a bottom-up path for the network, fusing deep and shallow features to further enhance the ability to extract detailed information; S5: The Rep structure outputs features at different scales, adjusts the number of image channels, and makes calculations faster and more flexible. The specific methods for learning various features in step S3 are as follows: S301: Employs an efficient aggregation network structure, using 4 layers of CBS convolutional blocks, E-ELAN convolutions, and MPConv convolutional layers to extract image features at different scales; The improved, efficient network structure, CBS, consists of Conv + BN + SiLU. First, a CBS convolutional block with a kernel size of 3 and a stride of 1 is formed, followed by a convolutional block with a kernel size of 3 and a stride of 2. These two types of CBS convolutional blocks are alternately connected in pairs. After four CBS convolutions, the feature map becomes 160. 160 128, E-ELAN consists of multiple CBSs, whose input and output feature sizes remain constant. The number of channels changes in the first two CBSs, and the input and output channels remain consistent in subsequent CBSs. The output of the last CBS is the required channel. The MPConv convolutional layer has the same input and output channels. It adds a Maxpool layer on top of the BConv layer, forming two branches. The upper branch halves the image width and height through Maxpool and halves the image channels through the BConv layer. The lower branch halves the image channels through the first BConv layer and halves the image width and height through the second BConv layer. Finally, the Cat operation is used to fuse the features extracted from the upper and lower branches. The specific steps for establishing the bottom-up path of the network in step S4 are as follows: S401: Bottom-up fusion of deep and shallow features is a PAFPN structure. The CSP module in YOLOv5 is replaced with the ELAN-H module, and the downsampling is changed to the MP2 layer. For the 32x downsampled feature map C5 output by the backbone, it is then processed by SPPCSP, reducing the number of channels from 1024 to 512. First, it is fused with C4 and C3 in a top-down manner to obtain P3, P4, and P5. Then, it is fused with P4 and P5 in a bottom-up manner to achieve the joint fusion of deep and shallow features. The specific steps of the Rep structure in step S5 are as follows: For P3, P4, and P5 output by PAFPN, the number of channels is adjusted by RepConv, and finally a 1x1 convolution is used to predict the three parts: class, confidence, and anchor box.

2. The end-to-end transmission line detection method based on YOLOv7 according to claim 1, characterized in that, The specific steps of step S1 are as follows: S101: A transmission line dataset is constructed by combining visible light transmission line images and transmission line images collected from the internet.

3. The end-to-end transmission line detection method based on YOLOv7 according to claim 1, characterized in that, The specific steps of step S2 are as follows: S201: Data augmentation of the image is performed using the Mosaic method, which stitches together nine different images as a new input image to enhance the robustness of the network. S202: Scale all input image sizes using linear interpolation.

4. The end-to-end transmission line detection method based on Yolov7 according to claim 1, characterized in that, In step S202, all input images are scaled to 640. 640 3.

Images