High-voltage transmission line nest detection method based on improved YOLOv5s

Abstract

The invention relates to an improved YOLOv5s-based high-voltage transmission line nest detection method, which is applied to an unmanned terminal and is characterized in that a nest detection model which is trained in advance based on a nest small data set is utilized to predict the category and area of a target in an image captured by the unmanned terminal so as to realize high-voltage transmission line nest detection, the bird nest detection model is constructed on the basis of YOLOv5s, and the bird nest detection model comprises a DenseNet-based Backbone network, a bidirectional fusion FPN-based Neck network and a Head network which are connected in sequence. The model is based on the DenseNet and the bidirectional fusion FPN, the capability of extracting features from the image is effectively improved, in addition, the characteristic that the object size is small in a high-voltage transmission line nest detection scene is fully considered, each network is organized by providing one of inclusive and exclusive connection modes, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of high-voltage transmission line operation and inspection, and in particular to a high-voltage transmission line bird nest detection method based on improved YOLOv5s. Background Art

[0002] Among the causes of high-voltage transmission line failures, bird activity is the third most common cause after lightning strikes and external damage. In addition, bird activity on high-voltage transmission towers accounts for 90% of failures. Therefore, developing an excellent bird nest detection method can better prevent power failures caused by bird activity.

[0003] Since most transmission towers are located in the wild and bird nests are usually built high up on the towers, traditional manual inspections are time-consuming and inefficient. With the continuous development of artificial intelligence technology, unmanned aerial vehicle (UAV) inspections have gradually replaced manual inspections. Many intelligent target detection algorithms based on airborne image recognition technology have been applied to bird nest detection.

[0004] In the aerial images taken by drone inspections, bird nest detection can be divided into two main categories. One is the traditional transmission line target detection algorithm based on feature extraction and classification models; however, due to the irregular shape of bird nests and the color that changes with light, the robustness of this type of algorithm is poor.

[0005] The second category is the target detection algorithm based on deep learning, including two-stage and single-stage algorithms. The two-stage detection algorithm is performed in two steps: first, the candidate box area is obtained, and then the target in the area is classified. Representative algorithms include R-CNN, Fast R-CNN, Mask R-CNN, Faster R-CNN, etc. The single-stage detection algorithm directly predicts the category and area of ​​the target through the target detection network. Representative algorithms include SSD (Single Shot Multi-Box Detector) and YOLO (You Only LookOnce) series. Among them, YOLOv5s achieved a detection speed of 73FPS on the COCO dataset, and the model size is only 27.8M. However, for small targets and targets in complex backgrounds, there are still cases of false detection and missed detection. The complex improvement of the feature fusion layer makes the model too large and difficult to apply to embedded devices.

[0006] Therefore, how to provide a method that can effectively identify bird nests in aerial images of power transmission lines under complex backgrounds has become an urgent problem to be solved by those skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a high-voltage transmission line bird nest detection method based on improved YOLOv5s. YOLOv5s is improved according to the characteristics of bird nest detection to improve the detection accuracy.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] One aspect of the present invention provides a method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s, which is applied to unmanned terminals. A bird nest detection model pre-trained based on a small bird nest data set is used to predict the category and area of ​​targets in images captured by the unmanned terminal to achieve bird nest detection on high-voltage transmission lines. The bird nest detection model is constructed based on YOLOv5s, and the bird nest detection model includes a Backbone network based on DenseNet, a Neck network based on bidirectional fusion FPN, and a Head network connected in sequence.

[0010] As a preferred technical solution, the process of processing the input features by the Neck network based on the bidirectional fusion FPN to obtain the output features includes:

[0011] Multi-scale feature extraction is performed on the input features to obtain multi-layer feature maps. The high-layer feature maps are upsampled and added to the low-layer feature maps. At the same time, the low-layer feature maps are downsampled and added to the high-layer feature maps to achieve bidirectional fusion and obtain output features.

[0012] As a preferred technical solution, the Backbone network, Neck network and Head network are connected in an inclusive manner, wherein the Backbone network includes multiple end-to-end connected convolutional layers, and the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer.

[0013] As a preferred technical solution, the Backbone network includes multiple end-to-end connected convolutional layers, and the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer and do not include the last convolutional layer.

[0014] As a preferred technical solution, when training the bird's nest detection model, the bird's nest small data set is divided into a training set, a validation set and a test set in advance, and the bird's nest detection model is trained, validated and tested respectively.

[0015] As a preferred technical solution, the bird's nest small data set includes samples of bird's nests blocked by transmission towers, bird's nests blocked by birds, and unfinished bird's nests photographed from a drone perspective.

[0016] Another aspect of the present invention provides a high-voltage transmission line bird nest detection device based on improved YOLOv5s, which is set in an unmanned terminal. The high-voltage transmission line bird nest detection device is constructed based on YOLOv5s, and includes:

[0017] Backbone network module, built on the DenseNet network, is used to perform preliminary feature extraction on images captured by unmanned terminals;

[0018] The Neck network module is constructed based on a bidirectional fusion FPN network and is used to perform multi-scale feature extraction on the features output by the Backbone network module;

[0019] The head network module is used to obtain the category and area of ​​the target in the image captured by the unmanned terminal based on the features output by the neck network module, so as to realize the bird nest detection on the high-voltage transmission line.

[0020] As a preferred technical solution, the Backbone network module includes multiple end-to-end connected convolutional layers, the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer.

[0021] As a preferred technical solution, the Backbone network module includes multiple end-to-end connected convolutional layers, and the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer and do not include the last convolutional layer.

[0022] Another aspect of the present invention provides a computer-readable storage medium, comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned high-voltage transmission line bird nest detection method based on improved YOLOv5s.

[0023] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0024] (1) Improving the ability to extract features from captured images: The bird's nest detection model of the present invention includes a Backbone network based on DenseNet, a Neck network based on bidirectional fusion FPN, and a Head network that are connected in sequence, which effectively improves the ability to extract features from images.

[0025] (2) High model prediction accuracy: The present invention fully considers the small size of objects in the bird nest detection scenario of high-voltage transmission lines, and organizes the Backbone network, Neck network and Head network by providing two optional connection methods: inclusive and exclusive, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of a high-voltage transmission line bird nest detection device that improves YOLOv5s in an embodiment;

[0027] Figure 2 A schematic diagram of the relationship between the position, width and height of a small data set instance in the embodiment;

[0028] Figure 3 Schematic diagram of an improved FPN network used in the embodiment;

[0029] Figure 4 This is the YOLOv5s structure diagram;

[0030] Figure 5 A schematic diagram of the connection method of YOLOv5s in the embodiment;

[0031] Figure 6 is a schematic diagram of an inclusive connection method in an embodiment;

[0032] Figure 7 Schematic diagram of an exclusive connection method in the embodiment;

[0033] Figure 8 Schematic diagram of YOLOv5s recognition results;

[0034] Fig. 9 A schematic diagram of the identification result using the embodiment method;

[0035] Fig.10 is a schematic diagram of an electronic device in an embodiment;

[0036] Among them, 1. Backbone network module, 2. Neck network module, 3. Head network module. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] Example 1

[0039] In view of the problems existing in the aforementioned prior art, this embodiment provides a bird nest detection method for high-voltage transmission line based on improved YOLOv5s, which is applied to unmanned terminals such as drones and unmanned vehicles. In the actual detection process, a bird nest detection model pre-trained based on a small bird nest data set is used to predict the category and area of ​​the target in the image captured by the unmanned terminal, so as to realize bird nest detection on high-voltage transmission line.

[0040] Preferably, after acquiring the image captured by the unmanned terminal, the image is also pre-processed such as denoising and filtering.

[0041] This embodiment uses a small data set of bird nests taken from the perspective of a drone. The data set includes 109 images, including samples of bird nests blocked by transmission towers, bird nests blocked by birds, and unfinished bird nests. Such a data set simulates the situation of most bird nests on high-voltage transmission lines. When training the bird nest detection model, the data set is pre-divided into a training set, a validation set, and a test set, with a ratio of 65:15:20. The relationship between the position, width, and height of the instances in the data set is as follows: Figure 2 shown.

[0042] In this embodiment, the bird's nest detection model is built based on YOLOv5s, and the bird's nest detection model includes a Backbone network based on DenseNet, a Neck network based on bidirectional fusion FPN, and a Head network that are sequentially connected.

[0043] In this embodiment, the Backbone part uses the DenseNet model. This improvement shows a significant performance improvement without significantly increasing the inference time.

[0044] In this embodiment, the Neck part is implemented by using an improved FPN. Specifically, the multi-scale image features are extracted through the ResNet network, and then the high-level feature map is upsampled and added to the low-level feature map. At the same time, the low-level feature map is downsampled and added to the high-level feature map, and finally a bidirectional fusion is achieved to obtain the feature map output by the Neck part. For details, see Figure 3 It is a schematic diagram of the structure of the improved FPN in this embodiment.

[0045] In this embodiment, the Backbone, Neck and Head networks are organized in an inclusive connection mode, see Figure 4 , Figure 5 They are respectively a schematic diagram of the structure of YOLOv5s and a schematic diagram of the connection method between networks. Figure 5 In the figure, Conv represents the convolutional layer and Detect represents the detection head. Figure 6 , Figure 7Similarly, in general, each time the input image passes through a convolutional layer, the size of the feature map obtained will be reduced a little, such as Figure 5 As shown in the figure, taking four convolutional layers as an example, three feature maps of different sizes can be obtained from the last three convolutional layers. These three feature maps are sorted into feature vectors after Neck, and finally used for Head part detection. Figure 6 The detection of the feature map obtained by the first convolutional layer is added. Since the feature map obtained by the first convolutional layer has the largest size and the highest resolution, this feature map is more suitable for detecting small-sized targets. The corresponding connection method is called the inclusive connection method. Figure 7 exist Figure 6 The feature map obtained by the last convolution layer is discarded on the basis of the last convolution layer. The feature map obtained by the last convolution layer has the smallest size and the lowest resolution, so this feature map is more suitable for detecting large-sized targets. In the images taken by drones, a large number of bird nests are small in size. If the feature map with the lowest resolution is discarded, the recognition accuracy can be kept unchanged while the reasoning speed can be improved. Correspondingly, this connection method becomes an exclusive connection method.

[0046] It should be noted that Figure 5 Figure 6 Figure 7 What is shown is just a schematic diagram of the connection method. The actual model is more complicated. Taking the YOLOv5s model as an example, the actual detection head detects the feature maps obtained by the 17th, 20th, and 23rd convolutional layers. Figure 5 Figure 6 Figure 7 Only for understanding inclusive and exclusive ways of connecting.

[0047] Compare Figure 5 and Figure 6 , we can see that compared with YOLOv5s, the inclusive connection method adds the detection of high-resolution feature maps. Since the details of high-resolution feature maps are more complete and the object positions are more accurate, high-resolution feature maps are more suitable for identifying small-sized objects.

[0048] Example 2

[0049] Based on Example 1, this example provides another method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s. The difference is that, see Figure 7In this embodiment, the Backbone, Neck and Head networks are organized in an exclusive connection mode, wherein the Backbone network includes N forward cascaded convolution blocks, the Neck network includes N reverse cascaded pyramid layers, the Head network includes N-1 detection heads, and each convolution block is connected one-to-one with each pyramid layer, and the N-1 pyramid layers at the end are connected one-to-one with each detection head. In this embodiment, N=4, and the specific value of N can be adjusted without conflict.

[0050] Compare Figure 6 and Figure 7 As you can see, Figure 6 Not only does it add detection for high-resolution feature maps, but it also removes detection for low-resolution images. Low-resolution images are suitable for the recognition of large-sized objects. If most of the objects in the image are small-sized objects, using an exclusive connection method can reduce the inference time.

[0051] To illustrate that the aforementioned connection method has performance advantages over YOLOv5s, see Table 1 for the experimental results of different connection methods, and Table 2 for the comparison results between the exclusive connection method and YOLOv5s.

[0052] Table 1 Experimental results of various connection methods

[0053] YOLOv5s Inclusive extraction of small item features Exclusively extract small item features mAP@0.5 0.826 0.853 0.847 Frame rate 102 82 99

[0054] Table 2 Comparison results between exclusive connection mode and YOLOv5s

[0055] method mAP@0.5 Reasoning time YOLOv5s 0.826 102 Exclusive approach 0.853 105

[0056] As can be seen from Table 1, the two connection methods achieve higher recognition accuracy compared to YOLOv5s without much loss in frame rate. As can be seen from Table 2, the inference time of the exclusive connection method is close to that of YOLOv5s, but the mAP@0.5 score is 3% higher than that of YOLOv5s.

[0057] See also Figure 8 , Fig. 9 The following are the recognition results of the same batch of samples using YOLOv5s and the exclusive connection method provided by this embodiment. By comparison, it can be seen that the small target bird's nest that cannot be recognized by YOLOv5s can be recognized using the method of this embodiment.

[0058] Example 3

[0059] See also Figure 1 Based on the above-mentioned embodiment, this embodiment provides a high-voltage transmission line bird nest detection device based on improved YOLOv5s, which is set in an unmanned terminal. The detection device is constructed based on YOLOv5s, and includes the following connected in sequence:

[0060] Backbone network module 1, built on the DenseNet network, is used to perform preliminary feature extraction on images captured by unmanned terminals;

[0061] Neck network module 2, built based on a bidirectional fusion FPN network, is used to extract multi-scale features from the features output by the Backbone network module;

[0062] The Head network module 3 is used to obtain the category and area of ​​the target in the image captured by the unmanned terminal based on the features output by the Neck network module, so as to realize the bird nest detection on the high-voltage transmission line.

[0063] Specifically, in this embodiment, the Backbone network module includes N forward cascaded convolution blocks, the Neck network module includes N reverse cascaded pyramid layers, the Head network module includes N detection heads, and each convolution block is connected one-to-one to each pyramid layer, and each pyramid layer is connected one-to-one to each detection head.

[0064] Example 4

[0065] Based on Example 3, this embodiment provides another high-voltage transmission line bird nest detection device based on improved YOLOv5s, the difference being that the Backbone network module of this embodiment includes N forward cascaded convolution blocks, the Neck network module includes N reverse cascaded pyramid layers, the Head network module includes N-1 detection heads, and each convolution block is connected one-to-one to each pyramid layer, and the N-1 pyramid layers at the end are connected one-to-one to each detection head.

[0066] Example 5

[0067] See also Fig.10 This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the high-voltage transmission line bird nest detection method based on the improved YOLOv5s as described in Example 1 or Example 2.

[0068] At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it.

[0069] Example 6

[0070] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the high-voltage transmission line bird nest detection method based on improved YOLOv5s as described in Example 1 or Example 2.

[0071] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A bird nest detection method for high-voltage transmission line based on improved YOLOv5s, characterized in that: Applied to unmanned terminals, the bird's nest detection model pre-trained based on a small bird's nest data set is used to predict the category and area of ​​the target in the image captured by the unmanned terminal to realize bird's nest detection on high-voltage transmission lines, wherein the bird's nest detection model is built based on YOLOv5s, and the bird's nest detection model includes a Backbone network based on DenseNet, a Neck network based on bidirectional fusion FPN, and a Head network connected sequentially.

2. According to a method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s according to claim 1, it is characterized in that: The process of processing the input features by the Neck network based on the bidirectional fusion FPN to obtain the output features includes: Multi-scale feature extraction is performed on the input features to obtain multi-layer feature maps. The high-layer feature maps are upsampled and added to the low-layer feature maps. At the same time, the low-layer feature maps are downsampled and added to the high-layer feature maps to achieve bidirectional fusion and obtain output features.

3. The method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s according to claim 1, characterized in that: The Backbone network, Neck network and Head network are connected in an inclusive manner, wherein the Backbone network includes multiple end-to-end connected convolutional layers, and the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer.

4. The method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s according to claim 1, characterized in that: The Backbone network, Neck network and Head network are connected in an exclusive connection manner, wherein the Backbone network includes multiple end-to-end connected convolutional layers, and the Neck network and the Head network are used to detect feature maps obtained by multiple convolutional layers preset in the Backbone network, and the preset multiple convolutional layers include the first convolutional layer and do not include the last convolutional layer.

5. The method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s according to claim 1, characterized in that: When training the bird's nest detection model, the bird's nest small data set is divided into a training set, a verification set and a test set in advance, and the bird's nest detection model is trained, verified and tested respectively.

6. The method for detecting bird nests on high-voltage transmission lines based on improved YOLOv5s according to claim 1, characterized in that: The bird nest small data set includes samples of bird nests blocked by transmission towers, bird nests blocked by birds, and unfinished bird nests photographed from a drone perspective.

7. A high-voltage transmission line bird nest detection device based on improved YOLOv5s, characterized in that: Set up in an unmanned terminal, the high-voltage transmission line bird nest detection device is built based on YOLOv5s, including the following connected in sequence: Backbone network module (1), built on the basis of DenseNet network, is used to perform preliminary feature extraction on images captured by unmanned terminals; A Neck network module (2), constructed based on a bidirectional fusion FPN network, for performing multi-scale feature extraction on features output by the Backbone network module; The head network module (3) is used to obtain the category and area of ​​the target in the image captured by the unmanned terminal based on the features output by the neck network module, so as to realize the detection of bird nests on high-voltage transmission lines.

8. The high-voltage transmission line bird nest detection device based on improved YOLOv5s according to claim 7, characterized in that: The Backbone network module (1) includes a plurality of convolutional layers connected end to end, and the Neck network module (2) and the Head network module (3) are used to detect feature maps obtained by the plurality of convolutional layers preset in the Backbone network module (1), wherein the plurality of convolutional layers preset include the first convolutional layer.

9. The high-voltage transmission line bird nest detection device based on improved YOLOv5s according to claim 7, characterized in that: The Backbone network module includes a plurality of convolutional layers connected head to tail, and the Neck network module (2) and the Head network module (3) are used to detect feature maps obtained by the plurality of convolutional layers preset in the Backbone network module (1), wherein the plurality of convolutional layers preset include the first convolutional layer and do not include the last convolutional layer.

10. A computer-readable storage medium, characterized in that: It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the high-voltage transmission line bird nest detection method based on improved YOLOv5s as described in any one of claims 1-6.

Images