Bird nest detection method for transmission lines in complex environments based on improved yolox algorithm

Abstract

The present invention discloses a method for detecting bird nests on power transmission lines in complex environments based on an improved Yolox algorithm. The method is specifically implemented in the following steps: Step 1: Collect bird nest image data, annotate and divide the data set into a training set, a validation set and a test set; Step 2: Establish an improved Yolox algorithm network model, and use a deep channel attention module and a loss function to improve the Yolox target detection algorithm; Step 3: Input the images in the training set into the improved Yolox algorithm network model for training, and obtain an optimized Yolox algorithm network model through training; Step 4: Input the images in the test set into the optimized Yolox model for detection, and obtain a bird nest detection result graph. The present invention solves the problem of low accuracy in bird nest detection in complex environments in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for detecting bird nests on power transmission lines in complex environments based on an improved Yolox algorithm. Background Art

[0002] To ensure a stable and reliable national power supply, the construction of a new generation of high-efficiency, long-distance, and intelligent power grid infrastructure, exemplified by ultra-high voltage (UHV) technology, has become a key development direction for my country. Due to my country's vast territory, major power failures caused by bird activity are common during long-distance power transmission. Typical failures caused by bird damage include short circuits caused by direct electric shock, short circuits caused by bird nesting, and flashovers caused by bird droppings covering insulation components. Analysis of the causes of these failures reveals that the nesting habits of birds result in large numbers of birds constantly confined to power towers and power lines, posing a significant risk to long-distance power transmission. Furthermore, due to the complex environment, traditional algorithms are more difficult to detect failures caused by bird nests.

[0003] To prevent such failures, power transmission operation and maintenance regulations require power maintenance departments to assign inspectors to conduct regular manual line inspections of fixed lines. Because transmission lines are often located in remote forests and mountainous areas with complex terrain, and birds often nest in hard-to-observe corners such as tower tops and pylons, manual line inspections for bird nest detection are costly and inefficient. In recent years, routine transmission line inspections have gradually moved away from manual methods, and drone-based video inspections have significantly improved the difficulty of power line inspections. Intelligent fault detection and identification technology based on drone inspection videos has recently become a research hotspot for promoting smart grid applications. However, due to the influence of lighting conditions, camera angle, shooting distance, line inspection flight speed, and complex terrain, automated detection based on drone inspection videos still faces challenges in the accuracy and robustness of detection algorithms.

[0004] Because the distance between the imaging end and the target object varies with the camera's position during drone inspections, some details of bird nests exhibit different scale characteristics as the distance changes. To accurately detect bird nests on transmission lines, this paper proposes a method for detecting bird nests on transmission lines in complex environments based on an improved Yolox algorithm. This method achieves accurate detection of bird nests on transmission lines. The algorithm first obtains images of bird nests on transmission lines using aerial photography. These images are then processed using image preprocessing methods. The processed images are divided into training, validation, and test sets. Finally, a deep learning network (Yolox algorithm model) is used to extract feature information about the bird nests. This extracted feature information is then used to make accurate predictions, enabling efficient detection of bird nests on transmission lines in complex environments. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting bird nests in power transmission lines under complex environments based on an improved Yolox algorithm, which solves the problem of low accuracy in detecting bird nests in complex environments in the prior art.

[0006] The technical solution adopted by the present invention is a method for detecting bird nests on power transmission lines in a complex environment based on an improved Yolox algorithm, which is specifically implemented in the following steps:

[0007] Step 1: Collect bird nest image data, annotate and divide the dataset into training set, validation set and test set;

[0008] Step 2: Establish an improved Yolox algorithm network model and use the deep channel attention module and loss function to improve the Yolox target detection algorithm;

[0009] Step 3: Input the images in the training set into the improved Yolox algorithm network model for training, and obtain the optimized Yolox algorithm network model through training;

[0010] Step 4: Input the images in the test set into the optimized Yolox model for detection to obtain the bird's nest detection result image.

[0011] The present invention is also characterized in that:

[0012] Step 1 is implemented as follows:

[0013] Step 1.1: Use the high-definition camera onboard the drone to capture a large number of inspection videos, including images of transmission line bird nests in various complex backgrounds, as well as distant, close-up, and close-up images of transmission line bird nests. The number of images of each type of transmission line bird nest should be consistent.

[0014] Step 1.2: rotating, flipping, scaling, and arbitrarily cropping the bird's nest image obtained in step 1.1 to obtain an expanded bird's nest image;

[0015] Step 1.3: The sum of the bird's nest image obtained in step 1.1 and the expanded bird's nest image obtained in step 1.2 is used as the bird's nest sample image library. A corresponding label file is created for each image sample in the bird's nest sample image library. The corresponding label file contains the category label of the bird's nest. The category label of the bird's nest is nest. The label file meets the XML label file standard of the PascalVOC format. The XML label file includes the image ID, image path, image name, pixel height and width of the image, where the pixel height and width of the image are represented by the four coordinates of a rectangular box, including x min ,y min , x max ,y max , where (x min ,ymin ) is the coordinate of the upper left vertex of the rectangle, (x max ,y max ) are the coordinates of the lower right vertex of the rectangle.

[0016] Step 1.4: Divide the bird nest sample image library into training set, validation set, and test set in a ratio of 8:1:1;

[0017] Step 2 is implemented as follows:

[0018] Step 2.1: Improve the Yolox algorithm network model. Keep the original Yolox network's Focus layer, SPP layer, CBS layer, Neck layer, and prediction layer's combined network unchanged; add a deep channel attention module to the CSP1_X layer in the backbone network;

[0019] Step 2.2, the structure of the deep channel attention module is as follows:

[0020] For the input feature map X(X 1in 、X 2in 、X 3in ), X∈R CxWxH , C is the number of channels, W and H are the height and width of the feature map;

[0021] First, the first branch input feature map X is extracted from two perspectives using maximum pooling and average pooling. 1in Features:

[0022] X max1 =f maxpool (X 1in ) (1)

[0023] X avg1 =f avgpool (X 1in ) (2)

[0024] Among them, X 1in Represents the first branch input feature map, f maxpool Indicates maximum pooling, f avgpool Indicates average pooling, X max1 represents the result of the maximum pooling and X max1 ∈R C×H / 2×W / 2 , X avg1 represents the average result and X avg1 ∈R C ×H / 2×W / 2 ; Use maximum pooling to extract important information of the input feature map, use average pooling to extract global information of the input feature map, stack the results of maximum pooling and average pooling, and output them through the Sigmoid function:

[0025] A c =σ[Z concat (X max1 +X avg1 )] (3)

[0026] Among them, σ represents the Sigmoid function, Z concat Indicates stacking the results of maximum pooling and average pooling, A c Represents the output result of the Sigmoid function and A c ∈R C×1×1 , A c Each element in represents the importance level of its corresponding channel;

[0027] Secondly, the input feature map X 1in It is divided into two paths. One path is fused with the output of the Sigmoid function; the other path first performs a 3x3 convolution, then passes through a depth-wise separable convolution and then passes through the ReLU function output:

[0028] X 1-1 =A c *X 1in

[0029] X 1-2 =f conv3 (X 1in )

[0030] X 1-3 =Relu[f conv3×3 (X 1-2 )+f conv1×1 (X 1-2 )] (4)

[0031] where X 1-1 Represents the input feature map X 1in And the Sigmoid function output A c The fusion result, f conv3 Indicates a 3x3 convolution operation, X 1-2 Represents the result of the 3x3 convolution operation, f conv3x3 represents channel-by-channel convolution, f conv1x1 Represents point-by-point convolution, channel-by-channel convolution f conv3x3 And point-wise convolution f conv1x1 It is collectively called depth-separable convolution, Relu means Relu activation operation, X 1-3 Represents the result of Relu activation operation;

[0032] Finally, X 1-1 With X 1-3 To perform stack output:

[0033] X 1-out =X 1-1 +X 1-3 (5)

[0034] Among them, X 1-out Represents X 1-1 and X 1-3 The result of stacking is that the above feature extraction process is depth extraction.

[0035] The second branch of the deep channel attention module inputs the feature map X 2in The width and height are upsampled by 2 times and the third branch input feature map X 3in The width and height are upsampled by 4 times and then input into the depth extraction to extract features; the second branch inputs the feature map X 2in After deep feature extraction, the maximum pooling output is obtained. 2-out And the third branch input feature map X 3in After deep feature extraction, the average pooling output is obtained to obtain X 3-out :

[0036] X 2-out =f max pool [F DM (UP2(X 2in ))] (6)

[0037] X 3-out =f avgpool [F DM (UP4(X 3in ))] (7)

[0038] where X 2in Represents the second branch input feature map, X 3in Represents the third branch input feature map, UP2 represents the second branch input feature map X 2in The width and height are upsampled by 2 times, UP4 represents the input feature map X of the third branch 3in The width and height are upsampled by 4 times, F DM Indicates that the upsampled feature map is input into the depth extraction to extract features, X 2-out represents the output result of the second branch after the maximum pooling, X 3-out Represents the output result of the third branch after average pooling;

[0039] The last part of the deep channel attention module is to output the three branches of the input feature map X 1-out 、X 2-out 、X 3-out Stack them, then perform weighted fusion with the input feature map X to obtain the target weight, and output the new feature map:

[0040] X′=X×(X 1-out +X 2-out +X 3-out ) (8)

[0041] Where X' represents the output result of three branches X 1-out 、X 2-out 、X 3-out New feature map after weighted fusion;

[0042] Step 2.3, add a deep channel attention module between the two CBS structures of the Resunit layer, name the improved Resunit layer as the Resunit* layer, and introduce the Resunit* layer into the CSP1_X layer to extract the target, name the improved CSP1_X layer as the CSP1_X* layer, and introduce the CSP1_X* layer into the Yolox algorithm to obtain the improved Yolox algorithm network model.

[0043] Step 2.4: In the prediction layer, select the EIoU loss function with the best performance at this stage to replace the original IoU loss function. The EIOU loss function calculation formula is as follows:

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] Among them, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, b and b gt are the center points of the predicted box and the real box respectively, c is the diagonal distance of the minimum closed area that can contain both the predicted box and the real box. p is the Euclidean distance between the two center points, α is the weight factor, v is the similarity of the aspect ratio, w gt Indicates the width of the real box, h gt Indicates the height of the real frame, w indicates the width of the predicted frame, h indicates the height of the predicted frame, Loss CIoU Represents the loss of CIoU function, Loss EIoU Represents the loss of the EIoU function; Loss EIoU The loss function consists of three parts: overlap loss 1-IoU, center distance loss Width and height loss The first two parts continue the method in CIOU, but Loss EIoU The width and height loss of the loss function directly minimizes the difference between the width and height of the target box and the predicted box, making the convergence faster;

[0051] Based on EIOU and combined with the loss function Focal Loss, a Focal EIOU Loss is proposed. From the perspective of gradient, high-quality anchor frames are separated from low-quality anchor frames. The formula is as follows:

[0052] L Focal-EIoU =IoU γ Loss EIoU (15)

[0053] Among them, y is a parameter that controls the degree of outlier suppression, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, and L Focal-EIoU Represents the proposed Focal EIOU Loss, and obtains the improved yolox algorithm network model.

[0054] Step 3 is implemented as follows:

[0055] Step 3.1: Input the training set images divided in step 1.4 into the improved Yolox algorithm network model obtained in step 3 for training, set batch_size to 64, use stochastic gradient descent with momentum of 0.9, train for 300 rounds, and obtain the training results;

[0056] Step 3.2. Use the validation set to validate the improved Yolox algorithm network model, obtain the validation results, and analyze the differences between the training results and the validation results, such as whether they are overfitting or underfitting. If overfitting occurs, it means that the model performs very well on the training set, but very poorly on the validation set, that is, the generalization ability of the model is very poor, which needs to be solved by increasing the number of images in the training set, reducing the complexity of the model, and fine-tuning the hyperparameters. If underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which needs to be solved by increasing the complexity of the model, fine-tuning the hyperparameters, and adding more target features.

[0057] Step 3.3. Fine-tune the hyperparameters of the Yolox algorithm network model on the validation set, fine-tune the learning rate to 0.003 for better robustness, set the batch_size to 32, and obtain the optimized Yolox model.

[0058] Step 4 is implemented as follows:

[0059] Step 4.1. Input the test set into step 3.3 to obtain the optimized yolox model. The specific optimization process is as follows: yolox hyperparameter setting: input image pixel size 640*640, freeze training for 100 rounds of batch_size32, thaw training for 200 rounds of batch_size4, num_workers2, Adam optimizer, decay weight coefficient 5*10-4, initial learning rate 1*10-5, set the IoU threshold to 0.5 for testing when testing the test set, and fine-tune the learning rate to 0.003 during verification to obtain better robustness. Set batch_size to 32, freeze training for the first 100 rounds, and the loss decreases rapidly. After 200 rounds of thaw training, the network is continuously fine-tuned. After 150 rounds of the total 300 rounds, the loss change of the verification set gradually decreases, and the optimized yolox model is obtained.

[0060] Step 4.2: Use the test set to test the performance of the optimized YOLOX model and obtain the detection results.

[0061] The present invention provides a method for detecting bird nests on power transmission lines in complex environments based on an improved Yolox algorithm. This method addresses the problem of insufficient sample datasets by expanding the sample database. The improved Yolox algorithm is used to detect bird nests on power transmission lines, improving detection accuracy and efficiency. First, a Deep Channels Attention Module is introduced to perceive the importance of each channel in the feature map, enhancing important channels and suppressing redundant channels, which is of great significance for improving target detection accuracy. Second, the original IoU loss function of the Yolox algorithm is replaced by the EIoU loss function to increase detection speed. The proposed method can improve the accuracy and recognition efficiency of the bird nest detection algorithm, reduce the computational complexity, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a method for detecting bird nests on power transmission lines in a complex environment based on an improved yolox algorithm;

[0063] Figure 2 is a schematic diagram of the network structure of the Yolox target detection algorithm in the method for detecting bird nests on power transmission lines in a complex environment based on the improved Yolox algorithm of the present invention; wherein, Figure 2(a) is a schematic diagram of the structure of the input end (Input) and the network backbone module (Backbone); Figure 2(b) is a schematic diagram of the structure of the feature enhancement module layer (Neck); Figure 2(c) is a schematic diagram of the structure of the prediction module (Prediction); Figure 2(d) is a schematic diagram of the SPP structure in the Backbone layer of the network backbone module; Figure 2(e) is a schematic diagram of the network backbone module (Backbone), the feature enhancement module layer (Neck) and the prediction module (Prediction). ction) structure; Figure 2(f) is a schematic diagram of the Resunit structure in the network backbone module (Backbone), feature enhancement module layer (Neck) and prediction module (Prediction) structure; Figure 2(g) is a schematic diagram of the CSP1_X structure in the network backbone module Backbone layer; Figure 2(h) is a schematic diagram of the CSP2_X structure in the input end (Input) and network backbone module (Backbone) layer; Figure 2(i) is a schematic diagram of the Focus structure in the network backbone module Backbone layer; Figure 2(j) is a schematic diagram of the introduced Deep Channels Attention Module structure; Figure 2(k) is a schematic diagram of the improved Resunit* structure; Figure 2(l) is a schematic diagram of the improved CSP1_X* structure;

[0064] Figure 3 This is a schematic diagram of a sample transmission line bird nest image database in the transmission line bird nest detection method in a complex environment based on the improved Yolox algorithm of the present invention;

[0065] Figure 4 It is a schematic diagram of rotating a transmission line bird nest image database sample to achieve capacity expansion in the transmission line bird nest detection method in a complex environment based on the improved Yolox algorithm of the present invention;

[0066] Figure 5 is a diagram of bird nest detection results in the transmission line bird nest detection method in a complex environment based on the improved yolox algorithm of the present invention, wherein Figure 5(a) is a diagram of bird nest detection results in a residential environment, and Figure 5(b) is a diagram of bird nest detection results in a complex tree background. DETAILED DESCRIPTION

[0067] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] The present invention is based on the improved yolox algorithm for detecting bird nests in power transmission lines under complex environments. The flow chart is as follows: Figure 1 As shown, please follow the steps below:

[0069] Step 1: Collect bird nest image data, annotate and divide the dataset into training set, validation set and test set;

[0070] Step 1 is implemented as follows:

[0071] Step 1.1: Use the high-definition camera on the drone to shoot a large number of inspection videos, including images of bird nests on power transmission lines under various complex backgrounds. Figure 3 As shown, the transmission line bird nest images are shown in perspective, close-up and close-up, and the number of each type of transmission line bird nest images is kept consistent;

[0072] Step 1.2: Rotate (180° to -180°), flip (horizontally and vertically), scale, and crop the bird's nest image obtained in step 1.1 to obtain an expanded bird's nest image as shown in Figure 4.

[0073] Step 1.3: The sum of the bird's nest image obtained in step 1.1 and the expanded bird's nest image obtained in step 1.2 is used as the bird's nest sample image library. A corresponding label file is created for each image sample in the bird's nest sample image library. The corresponding label file contains the category label of the bird's nest. The category label of the bird's nest is nest. The label file meets the XML label file standard of the Pascal VOC format. The XML label file includes the image ID, image path, image name, pixel height and width of the image, where the pixel height and width of the image are represented by the four coordinates of a rectangular box, including x min ,y min , x max ,y max , where (x min ,y min ) is the coordinate of the upper left vertex of the rectangle, (x max ,y max ) are the coordinates of the lower right vertex of the rectangle.

[0074] Step 1.4: Divide the bird nest sample image library into training set, validation set, and test set in a ratio of 8:1:1;

[0075] Step 2, establish an improved yolox algorithm network model, adopt the deep channel attention module (Deep ChannelsAttention Module) and loss function to improve the yolox target detection algorithm, and improve the detection accuracy; Figure 2 is a schematic diagram of the network structure of the yolox target detection algorithm in the power transmission line bird nest detection method under a complex environment based on the improved yolox algorithm of the present invention; the yolox algorithm network structure is mainly composed of four parts, namely the input end, the network backbone module, the feature enhancement module layer and the prediction module, that is, Figure 2 (a) is a schematic diagram of the structure of the input end (Input) and the network backbone module (Backbone); Figure 2 (b) is a schematic diagram of the structure of the feature enhancement module layer (Neck); Figure 2 (c) is a schematic diagram of the structure of the prediction module (Prediction); Figure 2 (d) is a schematic diagram of the SPP structure in the Backbone layer of the network backbone module; Figure 2 (e) is a schematic diagram of the network backbone Figure 2(f) is a schematic diagram of the CBS structure in the network backbone module (Backbone), feature enhancement module layer (Neck) and prediction module (Prediction) structure; Figure 2(g) is a schematic diagram of the CSP1_X structure in the network backbone module Backbone layer; Figure 2(h) is a schematic diagram of the CSP2_X structure in the input end (Input) and network backbone module (Backbone) layer; Figure 2(i) is a schematic diagram of the Focus structure in the network backbone module Backbone layer; Figure 2(j) is a schematic diagram of the introduced Deep Channels Attention Module structure; Figure 2(k) is a schematic diagram of the improved Resunit* structure; Figure 2(l) is a schematic diagram of the improved CSP1_X* structure;

[0076] Step 2 is implemented as follows:

[0077] Step 2.1. Improve the Yolox algorithm network model. Keep the Focus layer of the original Yolox network as shown in Figure 2(i), the SPP layer as shown in Figure 2(d), the CBS layer as shown in Figure 2(e), the Neck layer as shown in Figure 2(b), the CSP2_X layer as shown in Figure 2(h), and the prediction layer as shown in Figure 2(c) unchanged; add a deep channel attention module to the Resunit layer (as shown in Figure 2(f)) in the backbone network Backbone (as shown in Figure 2(a)) and introduce it into the CSP1_X layer (as shown in Figure 2(g)). This can enhance the representation ability of the feature map, suppress the background noise of the detection object in irrelevant images, and enhance the channel information of the target object;

[0078] Step 2.2: The Deep Channels Attention Module proposed in this paper has a structure as shown in Figure 2(j). The structure is as follows:

[0079] For the input feature map X(X 1in 、X 2in 、X 3in ), X∈R CxWxH , C is the number of channels, W and H are the height and width of the feature map;

[0080] First, the first branch input feature map X is extracted from two perspectives using maximum pooling and average pooling. 1in Features:

[0081] X max1 =f max pool (X 1in ) (1)

[0082] X avg1 =f avgpool (X 1in ) (2)

[0083] Among them, X 1in Represents the first branch input feature map, f maxpool Indicates maximum pooling, f avgpool Indicates average pooling, X max1 represents the result of the maximum pooling and X max1 ∈R C×H / 2×W / 2 , X avg1 represents the average result and X avg1 ∈R C ×H / 2×W / 2 ; Use maximum pooling to extract important information of the input feature map, use average pooling to extract global information of the input feature map, stack the results of maximum pooling and average pooling (concat), and output through the Sigmoid function:

[0084] A c =σ[Z concat (X max1 +X avg1 )] (3)

[0085] Among them, σ represents the Sigmoid function, Z concat Indicates stacking the results of maximum pooling and average pooling, A c Represents the output result of the Sigmoid function and A c ∈R C×1×1 , A c Each element in represents the importance level of its corresponding channel;

[0086] Secondly, the input feature map X 1in It is divided into two paths. One path is fused with the output of the Sigmoid function; the other path first performs a 3x3 convolution, then passes through a depth-wise separable convolution and then passes through the ReLU function output:

[0087] X 1-1 =A c *X 1in

[0088] X 1-2 =f conv3 (X 1in )

[0089] X 1-3 =Relu[f conv3×3 (X 1-2 )+f conv1×1 (X 1-2 )] (4)

[0090] where X 1-1 Represents the input feature map X 1in And the Sigmoid function output A c The fusion result, f conv3 Indicates a 3x3 convolution operation, X 1-2 Represents the result of the 3x3 convolution operation, f conv3x3 represents channel-by-channel convolution, f conv1x1 Represents point-by-point convolution, channel-by-channel convolution f conv3x3 And point-wise convolution f conv1x1 It is collectively called Depth wise convolution, which effectively reduces the number of parameters and the amount of calculation. Relu represents Relu activation operation, X 1-3 Represents the result of Relu activation operation;

[0091] Finally, X 1-1 With X 1-3 To perform stack output:

[0092] X 1-out =X 1-1 +X 1-3 (5)

[0093] Among them, X 1-out Represents X 1-1 and X 1-3 The result of stacking is that the above feature extraction process is called Deep Extraction Module.

[0094] The second branch of the Deep Channels Attention Module inputs the feature map X 2in The width and height are upsampled by 2 times and the third branch input feature map X 3in The width and height of the image are upsampled by 4 times and then input into the deep extraction module to extract features; the second branch inputs the feature map X 2in After the deep extraction module extracts features, the maximum pooling output is obtained. 2-out And the third branch input feature map X 3in After the deep extraction module extracts the features, the average pooling output is obtained. 3-out :

[0095] X 2-out =f max pool [F DM (UP2(X 2in ))] (6)

[0096] X 3-out =f avgpool [F DM (UP4(X 3in ))] (7)

[0097] where X 2in Represents the second branch input feature map, X 3in Represents the third branch input feature map, UP2 represents the second branch input feature map X 2in The width and height are upsampled by 2 times, UP4 represents the input feature map X of the third branch 3in The width and height are upsampled by 4 times, F DM Indicates that the upsampled feature map is input into the Deep Extraction Module to extract features, X 2-out represents the output result of the second branch after the maximum pooling, X 3-out Represents the output result of the third branch after average pooling;

[0098] The last part of the Deep Channels Attention Module is to output the three branches of the input feature map X to the result X 1-out 、X 2-out 、X 3-out Stack them, then perform weighted fusion with the input feature map X to obtain the target weight, and output the new feature map:

[0099] X'=X×(X1-out +X 2-out +X 3-out ) (8)

[0100] Where X' represents the output result of three branches X 1-out 、X 2-out 、X 3-out The new feature map after weighted fusion; the Deep Channels Attention Module uses multiple depth-separable convolutions, maximum pooling and average pooling to obtain deeper features of the input channel, which can enhance the channel information of the target object while maintaining a relatively small number of parameters.

[0101] Step 2.3. In the target detection task, extracting different features of the target through different convolution channels will cause too many training resources to be invested in non-target areas, resulting in low network training efficiency. In order to solve this problem, a deep channel attention module is added between the two CBS structures of the Resunit layer, and the improved Resunit layer is named Resunit* layer, and the structure is shown in Figure 2(k); at the same time, the Resunit* layer is introduced into the CSP1_X layer to extract the target, and the improved CSP1_X layer is named CSP1_X* layer, and the structure is shown in Figure 2(l). The CSP1_X* layer is introduced into the Yolox algorithm to obtain an improved Yolox algorithm network model, which can enhance the channel information of the target object while maintaining a relatively small number of parameters, while improving the channel importance level and efficiently detecting bird nests;

[0102] Step 2.4: In the prediction layer, the best performing EIoU loss function is selected to replace the original IoU loss function. The EIOU loss function is calculated as follows:

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] Among them, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, b and b gtare the center points of the predicted box and the real box respectively, c is the diagonal distance of the minimum closed area that can contain both the predicted box and the real box. p is the Euclidean distance between the two center points, α is the weight factor, v is the similarity of the aspect ratio, w gt Indicates the width of the real box, h gt Indicates the height of the real frame, w indicates the width of the predicted frame, h indicates the height of the predicted frame, Loss CIoU Represents the loss of CIoU function, Loss EIoU Represents the loss of the EIoU function; Loss EIoU The loss function consists of three parts: overlap loss 1-IoU, center distance loss Width and height loss The first two parts continue the method in CIOU, but Loss EIoU The width and height loss of the loss function directly minimizes the difference between the width and height of the target box and the predicted box, making the convergence faster;

[0110] Considering the problem of imbalanced training samples in the regression between target boxes and prediction boxes, that is, the number of high-quality anchor boxes with small regression errors in an image is far less than the number of low-quality samples with large errors, the poor-quality samples will produce excessively large gradients that affect the training process. Based on EIOU and combined with the loss function Focal Loss, a Focal EIOU Loss is proposed. Starting from the perspective of gradient, it separates high-quality anchor boxes from low-quality anchor boxes. The formula is as follows:

[0111] L Focal-EIoU =IoU γ Loss EIoU (15)

[0112] Among them, y is a parameter that controls the degree of outlier suppression, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, and L Focal-EIoU The proposed Focal EIOU Loss differs from the traditional Focal Loss in that the more difficult the sample, the greater the loss. This helps identify difficult samples. However, according to the formula above, higher IOUs result in greater losses, which acts as a weighted effect. This gives better target and prediction box regression targets a larger loss, accelerating convergence and improving regression accuracy. The introduction of Focal EIOU Loss optimizes the sample imbalance problem in the target and prediction box regression task. Specifically, it reduces the number of anchor boxes that overlap less with the target box, allowing the regression process to focus on high-quality anchor boxes, resulting in an improved YOLOx algorithm network model.

[0113] Step 3: Input the images in the training set into the improved Yolox algorithm network model for training, and obtain the optimized Yolox algorithm network model through training;

[0114] Step 3 is implemented as follows:

[0115] Step 3.1: Input the training set images divided in step 1.4 into the improved YOLOX algorithm network model obtained in step 3 for training, set batch_size to 64, use stochastic gradient descent (SGD) with momentum of 0.9, train for 300 rounds, and obtain the training results;

[0116] Step 3.2. Use the validation set to validate the improved Yolox algorithm network model, obtain the validation results, and analyze the differences between the training results and the validation results, such as whether they are overfitting or underfitting. If overfitting occurs, it means that the model performs very well on the training set, but very poorly on the validation set, that is, the generalization ability of the model is very poor, which needs to be solved by increasing the number of images in the training set, reducing the complexity of the model, and fine-tuning the hyperparameters. If underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which needs to be solved by increasing the complexity of the model, fine-tuning the hyperparameters, and adding more target features.

[0117] Step 3.3. Fine-tune the hyperparameters of the Yolox algorithm network model on the validation set, fine-tune the learning rate to 0.003 for better robustness, set the batch_size to 32, and obtain the optimized Yolox model.

[0118] Step 4: Input the images in the test set into the optimized Yolox model for detection to obtain the bird's nest detection result image.

[0119] Step 4 is implemented as follows:

[0120] Step 4.1: Input the test set into the optimized YOLOX model obtained in Step 3.3. The optimization process is as follows: YOLOX hyperparameter settings: input image pixel size 640*640, frozen training for 100 epochs with a batch size of 32, unfrozen training for 200 epochs with a batch size of 4, num_workers 2, Adam optimizer, weight decay coefficient 5*10⁻4, initial learning rate 1*10⁻5, and an IoU threshold of 0.5 for testing on the test set. During training, transfer learning is employed. Because the features extracted by the backbone network of the improved YOLOX algorithm are universal, freezing training can speed up training and prevent weight corruption. During the freezing phase, the backbone weights of the model are frozen, the feature extraction network remains unchanged, and the video memory usage is small. Only fine-tuning of the network is performed. During the unfreezing phase, the backbone weights of the model are unfrozen, but the feature extraction network is modified. This requires more video memory, and all network parameters are modified. During validation, the learning rate was fine-tuned to 0.003 for better robustness, and the batch_size was set to 32. The first 100 rounds of training were frozen, and the loss decreased rapidly. The training was unfrozen for the next 200 rounds, and the network was continuously fine-tuned. After 150 rounds out of the total 300 rounds, the loss change of the validation set gradually decreased, and an optimized yolox model was obtained.

[0121] Step 4.2: Use the test set to test the performance of the optimized Yolox model and obtain the detection results: input the transmission line bird nest test image in a residential environment into the optimized Yolox model, and obtain the bird nest image detection result shown in Figure 5(a); input the transmission line bird nest test image under a complex tree background into the optimized Yolox model, and obtain the bird nest image detection result shown in Figure 5(b).

Claims

1. A method for detecting bird nests on power transmission lines in complex environments based on an improved yolox algorithm, characterized in that: Please follow the steps below to implement: Step 1: Collect bird nest image data, annotate and divide the dataset into training set, validation set and test set; Step 2: Establish an improved Yolox algorithm network model and use the deep channel attention module and loss function to improve the Yolox target detection algorithm; The step 2 is specifically implemented according to the following steps: Step 2.1: Improve the Yolox algorithm network model and keep the original Yolox network's Focus layer, SPP layer, CBS layer, Neck layer, and prediction layer's combined network unchanged; add a deep channel attention module to the CSP1_X layer in the backbone network; Step 2.2, the structure of the deep channel attention module is as follows: For the input feature map X(X 1in 、X 2in 、X 3in ), X∈R CxWxH , C is the number of channels, W and H are the height and width of the feature map; First, the first branch input feature map X is extracted from two perspectives using maximum pooling and average pooling. 1in Features: X max1 =f maxpool (X 1in ) (1) X avg1 =f avgpool (X 1in ) (2) Among them, X 1in Represents the first branch input feature map, f maxpool Indicates maximum pooling, f avgpool Indicates average pooling, X max1 represents the result of the maximum pooling and X max1 ∈R C×H / 2×W / 2 , X avg1 represents the average result and X avg1 ∈R C ×H / 2×W / 2 ; Use maximum pooling to extract important information of the input feature map, use average pooling to extract global information of the input feature map, stack the results of maximum pooling and average pooling, and output them through the Sigmoid function: A c =σ[Z concat (X max1 +X avgl ] (3) Among them, σ represents the Sigmoid function, Z concat Indicates stacking the results of maximum pooling and average pooling, A c Represents the output result of the Sigmoid function and A c ∈R C×1×1 , A c Each element in represents the importance level of its corresponding channel; Secondly, the input feature map X 1in It is divided into two paths. One path is fused with the output of the Sigmoid function; the other path first performs a 3x3 convolution, then passes through a depth-wise separable convolution and then passes through the ReLU function output: X l-1 =A c *X lin X l-2 =f conv3 (X lin ) X l-3 =Relu[f conv3×3 (X l-2 )+f conv1×1 (X l-2 )] (4) where X 1-1 Represents the input feature map X 1in And the Sigmoid function output A c The fusion result, f conv3 Indicates a 3x3 convolution operation, X 1-2 Represents the result of the 3x3 convolution operation, f conv3x3 represents channel-by-channel convolution, f conv1x1 Represents point-by-point convolution, channel-by-channel convolution f conv3x3 And point-wise convolution f conv1x1 It is collectively called depth-separable convolution, Relu means Relu activation operation, X 1-3 Represents the result of Relu activation operation; Finally, X 1-1 With X 1-3 To perform stack output: X 1-out =X l-1 +X l-3 (5) Among them, X 1-out Represents X 1-1 and X 1-3 The result of stacking is that the above feature extraction process is depth extraction. The second branch of the deep channel attention module inputs the feature map X 2in The width and height are upsampled by 2 times and the third branch input feature map X 3in The width and height are upsampled by 4 times and then input into the depth extraction to extract features; the second branch inputs the feature map X 2in After deep feature extraction, the maximum pooling output is obtained. 2-out And the third branch input feature map X 3in After deep feature extraction, the average pooling output is obtained to obtain X 3-out : X 2-out =f maxpool [F DM (UP2(X 2in ))] (6) X 3-out =f avgpool [F DM (UP4(X 3in ))] (7) where X 2in Represents the second branch input feature map, X 3in Represents the third branch input feature map, UP2 represents the second branch input feature map X 2in The width and height are upsampled by 2 times, UP4 represents the input feature map X of the third branch 3in The width and height are upsampled by 4 times, F DM Indicates that the upsampled feature map is input into the depth extraction to extract features, X 2-out represents the output result of the second branch after the maximum pooling, X 3-out Represents the output result of the third branch after average pooling; The last part of the deep channel attention module is to output the three branches of the input feature map X 1-out 、X 2-out 、X 3-out Stack them, then perform weighted fusion with the input feature map X to obtain the target weight, and output the new feature map: X'=X×(X 1-out +X 2-out +X 3-out ) (8) Where X' represents the output result of three branches X 1-out 、X 2-out 、X 3-out New feature map after weighted fusion; Step 2.3, add a deep channel attention module between the two CBS structures of the Resunit layer, name the improved Resunit layer as the Resunit* layer, and introduce the Resunit* layer into the CSP1_X layer to extract the target, name the improved CSP1_X layer as the CSP1_X* layer, and introduce the CSP1_X* layer into the Yolox algorithm to obtain the improved Yolox algorithm network model. Step 2.4: In the prediction layer, select the EIoU loss function with the best performance at this stage to replace the original IoU loss function. The EIOU loss function calculation formula is as follows: Among them, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, b and b gt are the center points of the predicted box and the real box respectively, c is the diagonal distance of the minimum closed area that can contain both the predicted box and the real box; p is the Euclidean distance between the two center points, α is the weight factor, v is the similarity of the aspect ratio, w gt Indicates the width of the real box, h gt Indicates the height of the real frame, w indicates the width of the predicted frame, h indicates the height of the predicted frame, Loss CIoU Represents the loss of CIoU function, Loss EIoU Represents the loss of the EIoU function; Loss EIoU The loss function consists of three parts: overlap loss 1-IoU, center distance loss Width and height loss The first two parts continue the method in CIOU, but Loss EIoU The width and height loss of the loss function directly minimizes the difference between the width and height of the target box and the predicted box, making the convergence faster; Based on EIOU and combined with the loss function Focal Loss, a Focal EIOU Loss is proposed. From the perspective of gradient, high-quality anchor frames are separated from low-quality anchor frames. The formula is as follows: L Focal-EIoU =IoU γ Loss EIoU (15) Among them, γ is a parameter that controls the degree of outlier suppression, IoU represents the ratio of the intersection and union of the real box A and the predicted box B, and L Focal-EIoU Represents the proposed Focal EIOU Loss, and obtains the improved yolox algorithm network model; Step 3: Input the images in the training set into the improved Yolox algorithm network model for training, and obtain the optimized Yolox algorithm network model through training; Step 4: Input the images in the test set into the optimized Yolox model for detection to obtain the bird's nest detection result image.

2. The method for detecting bird nests on power transmission lines in a complex environment based on the improved yolox algorithm according to claim 1, characterized in that: The step 1 is specifically implemented according to the following steps: Step 1.1: Use the high-definition camera onboard the drone to capture a large number of inspection videos, including images of transmission line bird nests in various complex backgrounds, as well as distant, close-up, and close-up images of transmission line bird nests. The number of images of each type of transmission line bird nest should be consistent. Step 1.2: rotating, flipping, scaling, and arbitrarily cropping the bird's nest image obtained in step 1.1 to obtain an expanded bird's nest image; Step 1.3: The sum of the bird's nest image obtained in step 1.1 and the expanded bird's nest image obtained in step 1.2 is used as the bird's nest sample image library. A corresponding label file is created for each image sample in the bird's nest sample image library. The corresponding label file contains the category label of the bird's nest. The category label of the bird's nest is nest. The label file meets the XML label file standard of the Pascal VOC format. The XML label file includes the image ID, image path, image name, pixel height and width of the image, where the pixel height and width of the image are represented by the four coordinates of a rectangular box, including x min ,y min ,x max ,y max , where (x min ,y min ) is the coordinate of the upper left vertex of the rectangle, (x max ,y max ) is the coordinate of the lower right vertex of the rectangle; Step 1.4: Divide the bird nest sample image library into training set, validation set, and test set in a ratio of 8:1:

1.

3. The power transmission line bird's nest detection method under a complex environment based on the improved yolox algorithm according to claim 2, wherein the step 3 is specifically implemented according to the following steps: Step 3.1: Input the training set images divided in step 1.4 into the improved Yolox algorithm network model obtained in step 3 for training, set batch_size to 64, use stochastic gradient descent with momentum of 0.9, train for 300 rounds, and obtain the training results; Step 3.2: Use the validation set to validate the improved Yolox algorithm network model, obtain the validation results, and analyze the differences between the training results and the validation results, such as whether they are overfitting or underfitting. If overfitting occurs, it means that the model performs well on the training set, but poorly on the validation set, that is, the model's generalization ability is very poor, which needs to be addressed by increasing the number of training set images, reducing the complexity of the model, and fine-tuning hyperparameters. If underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which needs to be addressed by increasing the model complexity, fine-tuning hyperparameters, and adding more target features. Step 3.

3. Fine-tune the hyperparameters of the Yolox algorithm network model on the validation set, fine-tune the learning rate to 0.003 for better robustness, set the batch_size to 32, and obtain the optimized Yolox model.

4. The method for detecting bird nests on power transmission lines in complex environments based on the improved Yolox algorithm according to claim 3, characterized in that: The step 4 is specifically implemented according to the following steps: Step 4.1: Input the test set into the optimized yolox model obtained in step 3.

3. The optimization process is as follows: yolox hyperparameter settings: input image pixel size 640*640, freeze training 100 rounds of batch_size 32, unfreeze training 200 rounds of batch_size 4, num_workers 2, Adam optimizer, decay weight coefficient 5*10 -4 Initial learning rate 1*10 -5 When testing the test set, the IoU threshold is set to 0.5 for experimentation. During verification, the learning rate is fine-tuned to 0.003 to obtain better robustness. The batch_size is set to 32. The first 100 rounds of training are frozen, and the loss decreases rapidly. After 200 rounds of training, the network is continuously fine-tuned. After 150 rounds of the total 300 rounds, the loss change of the verification set gradually decreases, and the optimized yolox model is obtained. Step 4.2: Use the test set to test the performance of the optimized YOLOX model and obtain the detection results.

Images