Fusion Net model-based foreign matter detection method for power transmission line in severe weather

Abstract

The invention discloses a Fusion Net model-based foreign matter detection method for a power transmission line in severe weather, and the method comprises the steps: firstly collecting a foreign matter image on the power transmission line, carrying out the data preprocessing of the collected image, marking, and dividing a data set into a training set, a verification set and a test set; establishing a network model; inputting the images in the training set into the network model for training, and obtaining a Fusion Net model-based power transmission line foreign matter detection network model in severe weather through training; and finally, inputting images in the test set into the trained network model for detection to obtain a foreign matter detection result graph of the power transmission line in severe weather. According to the invention, the problems of low precision and slow speed of foreign matter detection of the power transmission line in severe weather in the prior art are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for detecting foreign objects in power transmission lines under severe weather conditions based on a FusionNet model. Background Art

[0002] With the continuous development of modern cities and the continuous increase in electricity demand, the scale of power transmission lines is also expanding. At the same time, the environment around the lines is becoming more and more complex, and it is easily affected and attached by various foreign objects. If these foreign objects are not cleaned up in time, it will have a serious negative impact on the safe operation of the transmission lines. Among them, some factors such as the complex environment in which the transmission lines are located, which is susceptible to the attachment of various foreign objects, and difficult to detect in complex weather conditions, may lead to failures and safety hazards in power transmission lines. The traditional line inspection method can no longer meet the actual needs of modern power transmission lines, and a more intelligent and efficient detection method is urgently needed. At present, drones and other equipment are widely used to carry out image inspections of transmission lines. Compared with traditional inspection methods, this intelligent method can efficiently detect foreign objects in the line, thereby effectively preventing and avoiding the occurrence of line failures.

[0003] Considering the criticality of foreign body detection in transmission lines, a method for foreign body detection in transmission lines based on the FusionNet model has been developed. The algorithm studies the detection of foreign body targets in transmission lines under the background of severe weather. First, an image acquisition device is installed to obtain the status information of the transmission line in real time. Secondly, the data obtained from the image acquisition device is transmitted to the equipment health monitoring center on the ground through the 5G communication wireless network to achieve remote monitoring and avoid the safety hazards caused by personnel directly climbing the tower for inspection. Finally, the relevant algorithms are used for image analysis, data training and testing to detect whether there are foreign objects in the image and determine their location. Through this process, real-time monitoring and early warning of whether there are foreign objects in the transmission line can be achieved, thereby improving the inspection efficiency, reducing the dangers brought by climbing the tower for inspection, and providing guarantee for the safe and stable operation of the transmission line. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting foreign objects in power transmission lines under severe weather conditions based on a FusionNet model, which solves the problems of low accuracy and slow speed in detecting foreign objects in power transmission lines under severe weather conditions in the prior art.

[0005] The technical solution adopted by the present invention is a method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model, which is characterized by being implemented specifically according to the following steps:

[0006] Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set;

[0007] Step 2: Establish a network model;

[0008] Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training;

[0009] Step 4: Input the images in the test set into the network model trained in step 3 for detection, and obtain the result map of foreign body detection of power transmission lines under severe weather.

[0010] The present invention is also characterized in that:

[0011] Step 1 is implemented as follows:

[0012] Step 1.1: Collect a large number of inspection videos through tower cameras and inspection drones, including images of foreign objects on various transmission lines. Keep the number of foreign object images of each type of transmission line consistent through long-range, close-up and close-up images of foreign objects.

[0013] Step 1.2, perform data enhancement and expansion on the transmission line foreign body images obtained in step 1.1, including rain, fog, snow, strong light, etc., using a single and multiple methods combined to obtain a transmission line foreign body dataset;

[0014] Step 1.3: Combine the transmission line foreign body image obtained in step 1.1 and the image obtained in step 1.2 as a transmission line foreign body sample image library, and create a corresponding label file for each image sample in the transmission line foreign body sample image library. The corresponding label file contains the category label of the foreign body, the label name is FM, and is divided into training set, verification set and test set according to 6:2:2.

[0015] Step 2 is implemented as follows:

[0016] Step 2.1, the FusionNet algorithm network model is divided into three parts: Backbone, Neck, and Head. The SiLU activation function in the CBS module in the original network model is replaced with the Hardswish activation function, and the CA attention mechanism is embedded to redistribute the channel weights to reduce network parameters;

[0017] Step 2.2, FusionBlock module structure is as follows:

[0018] The CA attention mechanism decomposes the original channel attention into two parallel feature encodings of direction and position, and aggregates the spatial coordinate information into a new attention map;

[0019] First, in order to avoid compressing all spatial information into channels and capture long-range spatial interactions with precise location information, the global average pooling is decomposed as follows:

[0020]

[0021] The input feature map Input with a size of C×H×W is pooled in the X direction and Y direction to encode each channel, generating feature maps with sizes of C×H×1 and C×1×W respectively. The generated C×1×W feature map is transformed and then Concat operation is performed. The formula is as follows:

[0022] f=δ(F1([z h ,z w ])) Formula (3)

[0023] The feature maps extracted after Concat are spliced ​​according to the spatial dimension to form The feature map of the shape, where r is used to control the reduction rate of the block, and then the feature map is passed through the F1 convolution transformation function and the nonlinear activation function to generate the f intermediate feature map, and then the f intermediate feature map is split into two tensors f according to the spatial dimension. h and f w , the shapes are and Then perform F h and F w The convolution transformation function and the Sigmoid activation function get g h and g w Coordinate attention, finally g h and g w The coordinate attention is multiplied by the original input to get an output of the same shape as the input:

[0024]

[0025] The output formula of CoordinateAttention is written as:

[0026]

[0027] The Hardswish activation function proposed in MobileNetv3 is used to replace the Silu activation function. The formula is as follows:

[0028]

[0029] Step 2.3, FasterNet contains 4 Stages. The first Stage is preceded by the Embedding layer. The Embedding layer is a regular 4x4 convolution with a step size of 4, which is equivalent to reducing the width and height to 1 / 4. The Merging layer is used before the other Stages. The Merging layer is a regular 2x2 convolution with a step size of 2. Each Stage is stacked by several FasterNet blocks. First, the conventional convolution operation is used to extract spatial features on part of the input channel while keeping the remaining channels unchanged. Then, in the regular memory access, it regards the first or last continuous channel as the representative of the entire feature map for calculation. Assuming that the input and output feature maps have the same number of channels, the FLOPs of PConv are only For a typical case of r = 1 / 4, the FLOPs of PConv is only 1 / 16 of that of conventional Conv. In addition, the memory access of PConv is also smaller, i.e. For r=1 / 4, it is only 1 / 4 of the regular Conv;

[0030] Step 2.4: Replace the original CBS module with the FB module in step 2.2, and then send the concat result to the FSB module in step 2.3, add a residual edge to the right side of SPPF, and enhance the feature extraction ability of the network through the CA attention mechanism to form an FMB module;

[0031] Step 2.5: Combine the CIOU loss function and the EIOU loss function at the prediction layer to form a new ECIOU loss function, as follows:

[0032] CIoU uses IoU, Euclidean distance, corresponding aspect ratio and angle to measure the overlapping area between the target and the real box. The formula is as follows:

[0033]

[0034] Among them, ρ 2 (b,b gt ) represent the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, α is a positive trade-off parameter, v is used to measure the consistency of the aspect ratio, and w p and h p Represents the width and height of the prediction box, w gt and h gt Represent the width and height of the real frame respectively;

[0035] EIOU_Loss decomposes the aspect ratio influencing factors based on CIOU_Loss and calculates the width and height of the predicted frame and the real frame to solve the problems that may arise in CIOU_Loss. The formula is as follows:

[0036]

[0037] where c w and c h It is the width and height of the minimum bounding box covering the two Boxes;

[0038] The ECIOU loss function first adjusts the aspect ratio of the predicted frame through CIOU until it converges to a suitable range. Then, each edge is fine-tuned through EIOU until the correct value is reached; through this improved loss function, the adjustment of the predicted frame is increased and the regression speed of the frame is accelerated. The ECIOU formula is as follows:

[0039]

[0040] Step 2.5: Send the above results to the Head to judge the feature points and determine whether there is an object corresponding to the feature points.

[0041] Step 3 is implemented as follows:

[0042] Step 3.1: Input the training set images divided in step 2 into the FusionNet network model for training, set the parameter learning rate to 0.001, momentum to 0.937, weight decay to 0.0005, epoch to 300 times, BatchSize = 16, image size to 640 × 640, and use the learning rate decay method of cosine annealing to obtain the training results;

[0043] Step 3.2, use the validation set to test the optimized FusionNet network model, obtain the validation results, and analyze the differences between the training results and the validation results. If overfitting occurs, it means that the model performs well on the training set, but poorly on the validation set, that is, the generalization ability of the model is poor, which needs to be solved by increasing the amount of training data, using regularization constraints, adjusting parameters and hyperparameters, and reducing the complexity of the model; if underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which needs to be solved by increasing the complexity of the model, reducing regularization constraints, and adding more features;

[0044] Step 3.3: By fine-tuning the network's hyperparameters on the validation set, fine-tuning the learning rate to 0.002, increasing the learning rate, and setting batch_size to 16, we can obtain the optimized FusionNet network model.

[0045] Step 4 is implemented according to the following steps:

[0046] Step 4.1, input the test set into the optimized FusionNet model obtained in step 3.3. The specific optimization process is as follows: FusionNet model hyperparameter settings: num_workers = 4, Adam optimizer, attenuation weight coefficient 5*10-4, initial learning rate 1*10-5, set the IoU threshold to 0.5 for the test set test, and fine-tune the learning rate to 0.003 during verification. The loss change of the verification set gradually decreases in the total 300 rounds, and the FusionNet foreign body detection model is obtained;

[0047] Step 4.2: Use the test set to test the performance of the transmission line foreign object detection algorithm model under severe weather conditions based on the FusionNet model to obtain the detection results.

[0048] The beneficial effect of the present invention is that the foreign body detection method for transmission lines under bad weather based on the FusionNet model aims at the problems of low accuracy, slow speed and poor reliability of foreign body detection for transmission lines under bad weather in the existing monitoring methods, and proposes a foreign body detection algorithm for transmission lines under bad weather based on the FusionNet model. The FB module in FusionNet can better capture target features and improve detection accuracy; the FSB module helps to better process spatial information and improve the positioning accuracy of the target; the FMB module helps to dynamically adjust feature mapping and enhance the adaptability of the network to different targets; the ECIOU loss function reduces the number of model missed detections and false detections by better weighing the accuracy and recall rate in target detection, making the detection results more reliable. Experiments have proved that the FusionNet model can efficiently detect and identify various foreign bodies, especially under adverse meteorological conditions such as strong winds, rain and snow, or low visibility. At the same time, the improved model balances the relationship between model detection accuracy and model size, and has broad practical application prospects. In addition, the present invention also provides an advanced solution for the power industry, which is expected to provide more reliable transmission line monitoring under different meteorological conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a model overall deployment structure diagram of the transmission line foreign body detection algorithm under severe weather conditions based on the FusionNet model of the present invention;

[0050] Figure 2 The algorithm model design diagram of the transmission line foreign body detection algorithm under severe weather conditions based on the FusionNet model of the present invention;

[0051] Figure 3This is a CA attention structure diagram used by the FusionBlock of the power transmission line foreign body detection algorithm under severe weather conditions based on the FusionNet model of the present invention;

[0052] Figure 4 It is a structural diagram of the Fusion SpeedBlock of the foreign body detection algorithm for power transmission lines under severe weather conditions based on the FusionNet model of the present invention;

[0053] Figure 5 It is a FusionMemoryBlock structure diagram of the foreign body detection algorithm for power transmission lines under severe weather conditions based on the FusionNet model of the present invention;

[0054] Figure 6 It is a schematic diagram of adding rain, fog, snow and other conditions to a transmission line foreign object image database sample in a transmission line foreign object detection method under bad weather based on the FusionNet model of the present invention to achieve capacity expansion;

[0055] Figure 7 This is a diagram of foreign object detection results in the method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model of the present invention. DETAILED DESCRIPTION

[0056] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] The invention is based on the FusionNet model to detect foreign objects in power transmission lines under severe weather conditions. Figure 1 , Figure 2 , follow the steps below:

[0058] Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set;

[0059] Step 1 is implemented as follows:

[0060] Step 1.1: Collect a large number of inspection videos through tower cameras and inspection drones, including images of foreign objects on various transmission lines. Keep the number of foreign object images of each type of transmission line consistent through long-range, close-up and close-up images of foreign objects.

[0061] Step 1.2, perform data enhancement and expansion on the transmission line foreign body images obtained in step 1.1, including rain, fog, snow, strong light, etc., using a single and multiple methods combined to obtain a transmission line foreign body dataset;

[0062] Step 1.3: Combine the transmission line foreign object image obtained in step 1.1 and the image obtained in step 1.2 as a transmission line foreign object sample image library, and create a corresponding label file for each image sample in the transmission line foreign object sample image library. The corresponding label file contains the category label of the foreign object, and the label is named FM (Foreign Matter). It is divided into training set, verification set and test set according to 6:2:2.

[0063] Combination Figure 3 , Figure 4 , Step 2, establish the network model. First, replace the SiLU activation function in the original CBS module with the Hardswish activation function to reduce the number of memory accesses. At the same time, embed the CA attention mechanism to redistribute the weights of the channels, so that the network adds parameter learning of position information on the basis of paying attention to the overall classification information, thereby reducing the network parameters. Then, based on FasterNet, PConv is introduced to propose the Fusion SpeedBlock, which can extract spatial features more effectively while reducing redundant calculations, achieving a much faster running speed than other networks without affecting accuracy. Secondly, the FusionMemoryBlock is proposed, adding a residual edge and focusing on the key information in the input sequence through the CA attention mechanism, improving the generalization ability of the model and the accuracy of processing sequence tasks. Finally, EfficiCIoU is used as the loss function to accelerate the convergence speed of the model and improve the detection accuracy of the model.

[0064] Combination Figure 3 to Figure 6 , step 2 is implemented according to the following steps:

[0065] Step 2.1, the FusionNet algorithm network model is divided into three parts: Backbone, Neck, and Head. The SiLU activation function in the CBS module in the original network model is replaced with the Hardswish activation function, and the CA attention mechanism is embedded to redistribute the channel weights to reduce network parameters;

[0066] Step 2.2, FusionBlock module structure is as follows:

[0067] The CA attention mechanism decomposes the original channel attention into two parallel feature codes of direction and position, aggregates the spatial coordinate information into a new attention map, and strengthens the precise positioning of space, which means that the model can more effectively identify and extract the features of foreign objects at different angles and positions on the transmission line, thereby improving the model's feature extraction capabilities and the accuracy of foreign object detection.

[0068] First, in order to avoid compressing all spatial information into channels and capture long-range spatial interactions with precise location information, the global average pooling is decomposed as follows:

[0069]

[0070] The input feature map Input with a size of C×H×W is pooled in the X direction and Y direction to encode each channel, generating feature maps with sizes of C×H×1 and C×1×W respectively. The generated C×1×W feature map is transformed and then Concat operation is performed. The formula is as follows:

[0071] f=δ(F1([z h ,z w ])) Formula (3)

[0072] The feature maps extracted after Concat are spliced ​​according to the spatial dimension to form The feature map of the shape, where r is used to control the reduction rate of the block, and then the feature map is passed through the F1 convolution transformation function (1x1 convolution) and the nonlinear activation function to generate the f intermediate feature map, and then the f intermediate feature map is split into two tensors f according to the spatial dimension h and f w , the shapes are and Then perform F h and F w The convolution transformation function (1x1 convolution) and the Sigmoid activation function get g h and g w Coordinate attention, finally g h and g w The coordinate attention is multiplied by the original input to get an output of the same shape as the input:

[0073]

[0074] The output formula of CoordinateAttention is written as:

[0075]

[0076] Detecting foreign objects on the line in complex weather conditions can improve the safety and reliability of the power grid, reduce maintenance costs, and have a positive impact on the environment. This enables power companies to provide a stable and reliable power supply to meet electricity demand. Therefore, in order to better apply the algorithm model to practice, the present invention uses the Hardswish activation function proposed in MobileNetv3 to replace the Silu activation function. The formula is as follows:

[0077]

[0078] Compared to the original activation function, the Hardswish activation function only involves some simple multiplication and addition operations, which can enable faster training and reasoning of neural networks with the same hardware resources. At the same time, the Hardswish activation function outputs 0 when the input is negative, which means it has better sparsity and helps reduce the complexity and computational load of the model. In contrast, Silu produces non-zero outputs for all inputs. In the task of foreign body detection on power transmission lines, the characteristics of Hardswish make it more robust to noise and outliers, which helps to improve the robustness and adaptability of the model.

[0079] Step 2.3, drawing on the core idea of ​​Fasternet, Pconv is introduced into the C3 module, which reduces unnecessary calculations and memory accesses, and improves the efficiency of spatial feature extraction. FasterNet contains 4 Stages. The first Stage is preceded by the Embedding layer, which is a regular 4x4 convolution with a step size of 4, equivalent to reducing the width and height to 1 / 4. The other Stages are preceded by the Merging layer, which is a regular 2x2 convolution with a step size of 2, and also plays a downsampling role. Each Stage is composed of several FasterNet blocks stacked together. Figure 5 The left side of the figure shows how PConv works. First, it uses regular convolution operations to extract spatial features on a portion of the input channels while keeping the rest of the channels unchanged. Then, in regular memory access, it treats the first or last consecutive channel as a representative of the entire feature map for calculation, assuming that the input and output feature maps have the same number of channels, so that it can be processed without loss of generality. Without loss of generality, assuming that the input and output feature maps have the same number of channels, the FLOPs of PConv are only For a typical case of r = 1 / 4, the FLOPs of PConv is only 1 / 16 of that of conventional Conv. In addition, the memory access of PConv is also smaller, i.e. For r=1 / 4, it is only 1 / 4 of the regular Conv;

[0080] Step 2.4: Replace the original CBS module with the FB module in step 2.2, and then send the result after Concat to the FSB module in step 2.3 to reduce redundant calculations and extract space more effectively. At the same time, drawing on the idea of ​​the Resnet network, a residual edge is added to the right side of the SPPF, and the feature extraction ability of the network is enhanced through the CA attention mechanism to form an FMB module, which not only considers channel information, but also incorporates direction-related position information, and can effectively use the previously accumulated information to extract richer features for the back propagation of the network.

[0081] Step 2.5: Combine the CIOU loss function and the EIOU loss function at the prediction layer to form a new ECIOU loss function, as follows:

[0082] CIoU loss adds the aspect ratio v of the predicted box and the GT box to the DIoU loss. It takes into account three important geometric factors: overlapping area, center point distance, and aspect ratio. CIoU uses IoU, Euclidean distance, corresponding aspect ratio, and angle to measure the overlapping area between the target and the real box. The formula is as follows:

[0083]

[0084] Among them, ρ 2 (b,b gt ) represent the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, α is a positive trade-off parameter, v is used to measure the consistency of the aspect ratio, and w p and h p Represents the width and height of the prediction box, w gt and h gt Represent the width and height of the real frame respectively;

[0085] EIOU_Loss decomposes the aspect ratio influencing factors based on CIOU_Loss and calculates the width and height of the predicted frame and the real frame to solve the problems that may arise in CIOU_Loss. The formula is as follows:

[0086]

[0087] where c w and c h It is the width and height of the minimum bounding box covering the two Boxes;

[0088] The ECIOU loss function first adjusts the aspect ratio of the predicted frame through CIOU until it converges to a suitable range. Then, each edge is fine-tuned through EIOU until the correct value is reached; through this improved loss function, the adjustment of the predicted frame is increased and the regression speed of the frame is accelerated. The ECIOU formula is as follows:

[0089]

[0090] Step 2.5: Send the above results to the Head to judge the feature points and determine whether there is an object corresponding to the feature points.

[0091] Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training;

[0092] Step 3 is implemented as follows:

[0093] Step 3.1: Input the training set images divided in step 2 into the FusionNet network model for training, set the parameter learning rate to 0.001, momentum to 0.937, weight decay to 0.0005, epoch to 300 times, BatchSize = 16, image size to 640 × 640, and use the learning rate decay method of cosine annealing to obtain the training results;

[0094] Step 3.2, use the validation set to test the optimized FusionNet network model, obtain the validation results, and analyze the differences between the training results and the validation results, such as whether they are overfitting or underfitting; if overfitting occurs, it means that the model performs very well on the training set, but very poorly on the validation set, that is, the generalization ability of the model is very poor, which needs to be solved by increasing the number of training data, using regularization constraints, adjusting parameters and hyperparameters, and reducing the complexity of the model; if underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which is usually caused by low model complexity and too few features. It needs to be solved by increasing model complexity, reducing regularization constraints, adding more features, making the input data more expressive, adjusting parameters and hyperparameters, etc.

[0095] Step 3.3: Fine-tune the network hyperparameters on the validation set, fine-tune the learning rate to 0.002, increase the learning rate to obtain better robustness, set batch_size to 16, and obtain the optimized FusionNet network model.

[0096] Combination Figure 7 , step 4, input the images in the test set into the network model trained in step 3 for detection, and obtain the foreign body detection result map of the power transmission line under severe weather.

[0097] Step 4 is implemented according to the following steps:

[0098] Step 4.1, input the test set into the optimized FusionNet model obtained in step 3.3. The specific optimization process is as follows: FusionNet model hyperparameter settings: num_workers = 4, Adam optimizer, attenuation weight coefficient 5*10-4, initial learning rate 1*10-5, set the IoU threshold to 0.5 for the test set test, and fine-tune the learning rate to 0.003 during verification to obtain better robustness. The loss change of the verification set gradually decreases in the total 300 rounds, and the FusionNet foreign body detection model is obtained;

[0099] Step 4.2: Use the test set to test the performance of the transmission line foreign object detection algorithm model under severe weather conditions based on the FusionNet model to obtain the detection results.

[0100] Example 1

[0101] The invention is based on the FusionNet model to detect foreign objects in power transmission lines under severe weather conditions. Figure 1 , Figure 2 , follow the steps below:

[0102] Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set;

[0103] Step 2: Establish a network model;

[0104] Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training;

[0105] Step 4: Input the images in the test set into the network model trained in step 3 for detection, and obtain the result map of foreign body detection of power transmission lines under severe weather.

[0106] Example 2

[0107] The invention is based on the FusionNet model to detect foreign objects in power transmission lines under severe weather conditions. Figure 1 , Figure 2 , follow the steps below:

[0108] Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set;

[0109] Step 1 is implemented according to the following steps:

[0110] Step 1.1: Collect a large number of inspection videos through tower cameras and inspection drones, including images of foreign objects on various transmission lines. Keep the number of foreign object images of each type of transmission line consistent through long-range, close-up and close-up images of foreign objects.

[0111] Step 1.2, perform data enhancement and expansion on the transmission line foreign body images obtained in step 1.1, including rain, fog, snow, strong light, etc., using a single and multiple methods combined to obtain a transmission line foreign body dataset;

[0112] Step 1.3: Combine the transmission line foreign object image obtained in step 1.1 and the image obtained in step 1.2 as a transmission line foreign object sample image library, and create a corresponding label file for each image sample in the transmission line foreign object sample image library. The corresponding label file contains the category label of the foreign object, and the label is named FM (Foreign Matter). It is divided into training set, verification set and test set according to 6:2:2.

[0113] Step 2: Establish a network model. Specific steps are as follows:

[0114] Step 2.1, the FusionNet algorithm network model is divided into three parts: Backbone, Neck, and Head. The SiLU activation function in the CBS module in the original network model is replaced with the Hardswish activation function, and the CA attention mechanism is embedded to redistribute the channel weights to reduce network parameters;

[0115] Step 2.2, FusionBlock module structure is as follows:

[0116] The CA attention mechanism decomposes the original channel attention into two parallel feature codes of direction and position, aggregates the spatial coordinate information into a new attention map, and strengthens the precise positioning of space, which means that the model can more effectively identify and extract the features of foreign objects at different angles and positions on the transmission line, thereby improving the model's feature extraction capabilities and the accuracy of foreign object detection.

[0117] First, in order to avoid compressing all spatial information into channels and capture long-range spatial interactions with precise location information, the global average pooling is decomposed as follows:

[0118]

[0119] The input feature map Input with a size of C×H×W is pooled in the X direction and Y direction to encode each channel, generating feature maps with sizes of C×H×1 and C×1×W respectively. The generated C×1×W feature map is transformed and then Concat operation is performed. The formula is as follows:

[0120] f=δ(F1([z h ,z w ])) Formula (3)

[0121] The feature maps extracted after Concat are spliced ​​according to the spatial dimension to form The feature map of the shape, where r is used to control the reduction rate of the block, and then the feature map is passed through the F1 convolution transformation function (1x1 convolution) and the nonlinear activation function to generate the f intermediate feature map, and then the f intermediate feature map is split into two tensors f according to the spatial dimension h and f w , the shapes are and Then perform F h and F w The convolution transformation function (1x1 convolution) and the Sigmoid activation function get g h and g w Coordinate attention, finally g h and g w The coordinate attention is multiplied by the original input to get an output of the same shape as the input:

[0122]

[0123] The output formula of CoordinateAttention is written as:

[0124]

[0125] Detecting foreign objects on the line in complex weather conditions can improve the safety and reliability of the power grid, reduce maintenance costs, and have a positive impact on the environment. This enables power companies to provide a stable and reliable power supply to meet electricity demand. Therefore, in order to better apply the algorithm model to practice, the present invention uses the Hardswish activation function proposed in MobileNetv3 to replace the Silu activation function. The formula is as follows:

[0126]

[0127] Compared to the original activation function, the Hardswish activation function only involves some simple multiplication and addition operations, which can enable faster training and reasoning of neural networks with the same hardware resources. At the same time, the Hardswish activation function outputs 0 when the input is negative, which means it has better sparsity and helps reduce the complexity and computational load of the model. In contrast, Silu produces non-zero outputs for all inputs. In the task of foreign body detection on power transmission lines, the characteristics of Hardswish make it more robust to noise and outliers, which helps to improve the robustness and adaptability of the model.

[0128] Step 2.3, drawing on the core idea of ​​Fasternet, Pconv is introduced into the C3 module, which reduces unnecessary calculations and memory accesses, and improves the efficiency of spatial feature extraction. FasterNet contains 4 Stages. The first Stage is preceded by the Embedding layer, which is a regular 4x4 convolution with a step size of 4, equivalent to reducing the width and height to 1 / 4. The other Stages are preceded by the Merging layer, which is a regular 2x2 convolution with a step size of 2, and also plays a downsampling role. Each Stage is composed of several FasterNet blocks stacked together. Figure 5 The left side of the figure shows how PConv works. First, it uses regular convolution operations to extract spatial features on a portion of the input channels while keeping the rest of the channels unchanged. Then, in regular memory access, it treats the first or last consecutive channel as a representative of the entire feature map for calculation, assuming that the input and output feature maps have the same number of channels, so that it can be processed without loss of generality. Without loss of generality, assuming that the input and output feature maps have the same number of channels, the FLOPs of PConv are only For a typical case of r = 1 / 4, the FLOPs of PConv is only 1 / 16 of that of conventional Conv. In addition, the memory access of PConv is also smaller, i.e. For r=1 / 4, it is only 1 / 4 of the regular Conv;

[0129] Step 2.4: Replace the original CBS module with the FB module in step 2.2, and then send the result after Concat to the FSB module in step 2.3 to reduce redundant calculations and extract space more effectively. At the same time, drawing on the idea of ​​the Resnet network, a residual edge is added to the right side of the SPPF, and the feature extraction ability of the network is enhanced through the CA attention mechanism to form an FMB module, which not only considers channel information, but also incorporates direction-related position information, and can effectively use the previously accumulated information to extract richer features for the back propagation of the network.

[0130] Step 2.5: Combine the CIOU loss function and the EIOU loss function at the prediction layer to form a new ECIOU loss function, as follows:

[0131] CIoU loss adds the aspect ratio v of the predicted box and the GT box to the DIoU loss. It takes into account three important geometric factors: overlapping area, center point distance, and aspect ratio. CIoU uses IoU, Euclidean distance, corresponding aspect ratio, and angle to measure the overlapping area between the target and the real box. The formula is as follows:

[0132]

[0133]

[0134] Among them, ρ 2 (b,b gt ) represent the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, α is a positive trade-off parameter, v is used to measure the consistency of the aspect ratio, and w p and h p Represents the width and height of the prediction box, w gt and h gt Represent the width and height of the real frame respectively;

[0135] EIOU_Loss decomposes the aspect ratio influencing factors based on CIOU_Loss and calculates the width and height of the predicted frame and the real frame to solve the problems that may arise in CIOU_Loss. The formula is as follows:

[0136]

[0137] where c w and c h It is the width and height of the minimum bounding box covering the two Boxes;

[0138] The ECIOU loss function first adjusts the aspect ratio of the predicted frame through CIOU until it converges to a suitable range. Then, each edge is fine-tuned through EIOU until the correct value is reached; through this improved loss function, the adjustment of the predicted frame is increased and the regression speed of the frame is accelerated. The ECIOU formula is as follows:

[0139]

[0140] Step 2.5: Send the above results to the Head to judge the feature points and determine whether there is an object corresponding to the feature points.

[0141] Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training;

[0142] Step 4: Input the images in the test set into the network model trained in step 3 for detection, and obtain the result map of foreign body detection of power transmission lines under severe weather.

[0143] Step 4 is implemented according to the following steps:

[0144] Step 4.1, input the test set into the optimized FusionNet model obtained in step 3.3. The specific optimization process is as follows: FusionNet model hyperparameter settings: num_workers = 4, Adam optimizer, attenuation weight coefficient 5*10-4, initial learning rate 1*10-5, set the IoU threshold to 0.5 for the test set test, and fine-tune the learning rate to 0.003 during verification to obtain better robustness. The loss change of the verification set gradually decreases in the total 300 rounds, and the FusionNet foreign body detection model is obtained;

[0145] Step 4.2: Use the test set to test the performance of the transmission line foreign object detection algorithm model under severe weather conditions based on the FusionNet model to obtain the detection results.

[0146] Example 3

[0147] The invention is based on the FusionNet model to detect foreign objects in power transmission lines under severe weather conditions. Figure 1 , Figure 2 , follow the steps below:

[0148] Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set;

[0149] Step 1 is implemented as follows:

[0150] Step 1.1: Collect a large number of inspection videos through tower cameras and inspection drones, including images of foreign objects on various transmission lines. Keep the number of foreign object images of each type of transmission line consistent through long-range, close-up and close-up images of foreign objects.

[0151] Step 1.2, perform data enhancement and expansion on the transmission line foreign body images obtained in step 1.1, including rain, fog, snow, strong light, etc., using a single and multiple methods combined to obtain a transmission line foreign body dataset;

[0152] Step 1.3: Combine the transmission line foreign object image obtained in step 1.1 and the image obtained in step 1.2 as a transmission line foreign object sample image library, and create a corresponding label file for each image sample in the transmission line foreign object sample image library. The corresponding label file contains the category label of the foreign object, and the label is named FM (Foreign Matter). It is divided into training set, verification set and test set according to 6:2:2.

[0153] Step 2: Establish the network model. First, replace the SiLU activation function in the original CBS module with the Hardswish activation function to reduce the number of memory accesses. At the same time, embed the CA attention mechanism to redistribute the weights of the channels, so that the network adds parameter learning of position information on the basis of paying attention to the overall classification information, thereby reducing the network parameters. Then, based on FasterNet, PConv is introduced to propose the Fusion SpeedBlock, which can extract spatial features more effectively while reducing redundant calculations, achieving a much faster running speed than other networks without affecting accuracy. Secondly, the FusionMemoryBlock is proposed, adding a residual edge and focusing on the key information in the input sequence through the CA attention mechanism, improving the generalization ability of the model and the accuracy of processing sequence tasks. Finally, EfficiCIoU is used as the loss function to accelerate the convergence speed of the model and improve the detection accuracy of the model.

[0154] Step 2 is implemented as follows:

[0155] Step 2.1, the FusionNet algorithm network model is divided into three parts: Backbone, Neck, and Head. The SiLU activation function in the CBS module in the original network model is replaced with the Hardswish activation function, and the CA attention mechanism is embedded to redistribute the channel weights to reduce network parameters;

[0156] Step 2.2, FusionBlock module structure is as follows:

[0157] The CA attention mechanism decomposes the original channel attention into two parallel feature codes of direction and position, aggregates the spatial coordinate information into a new attention map, and strengthens the precise positioning of space, which means that the model can more effectively identify and extract the features of foreign objects at different angles and positions on the transmission line, thereby improving the model's feature extraction capabilities and the accuracy of foreign object detection.

[0158] First, in order to avoid compressing all spatial information into channels and capture long-range spatial interactions with precise location information, the global average pooling is decomposed as follows:

[0159]

[0160] The input feature map Input with a size of C×H×W is pooled in the X direction and Y direction to encode each channel, generating feature maps with sizes of C×H×1 and C×1×W respectively. The generated C×1×W feature map is transformed and then Concat operation is performed. The formula is as follows:

[0161] f=δ(F1([z h ,z w ])) Formula (3)

[0162] The feature maps extracted after Concat are spliced ​​according to the spatial dimension to form The feature map of the shape, where r is used to control the reduction rate of the block, and then the feature map is passed through the F1 convolution transformation function (1x1 convolution) and the nonlinear activation function to generate the f intermediate feature map, and then the f intermediate feature map is split into two tensors f according to the spatial dimension h and f w , the shapes are and Then perform F h and F w The convolution transformation function (1x1 convolution) and the Sigmoid activation function get g h and g w Coordinate attention, finally g h and g w The coordinate attention is multiplied by the original input to get an output of the same shape as the input:

[0163]

[0164] The output formula of CoordinateAttention is written as:

[0165]

[0166] Detecting foreign objects on the line in complex weather conditions can improve the safety and reliability of the power grid, reduce maintenance costs, and have a positive impact on the environment. This enables power companies to provide a stable and reliable power supply to meet electricity demand. Therefore, in order to better apply the algorithm model to practice, the present invention uses the Hardswish activation function proposed in MobileNetv3 to replace the Silu activation function. The formula is as follows:

[0167]

[0168] Compared to the original activation function, the Hardswish activation function only involves some simple multiplication and addition operations, which can enable faster training and reasoning of neural networks with the same hardware resources. At the same time, the Hardswish activation function outputs 0 when the input is negative, which means it has better sparsity and helps reduce the complexity and computational load of the model. In contrast, Silu produces non-zero outputs for all inputs. In the task of foreign body detection on power transmission lines, the characteristics of Hardswish make it more robust to noise and outliers, which helps to improve the robustness and adaptability of the model.

[0169] Step 2.3, drawing on the core idea of ​​Fasternet, Pconv is introduced into the C3 module, which reduces unnecessary calculations and memory accesses, and improves the efficiency of spatial feature extraction. FasterNet contains 4 Stages. The first Stage is preceded by the Embedding layer, which is a regular 4x4 convolution with a step size of 4, equivalent to reducing the width and height to 1 / 4. The other Stages are preceded by the Merging layer, which is a regular 2x2 convolution with a step size of 2, and also plays a downsampling role. Each Stage is composed of several FasterNet blocks stacked together. Figure 5 The left side of the figure shows how PConv works. First, it uses regular convolution operations to extract spatial features on a portion of the input channels while keeping the rest of the channels unchanged. Then, in regular memory access, it treats the first or last consecutive channel as a representative of the entire feature map for calculation, assuming that the input and output feature maps have the same number of channels, so that it can be processed without loss of generality. Without loss of generality, assuming that the input and output feature maps have the same number of channels, the FLOPs of PConv are only For a typical case of r = 1 / 4, the FLOPs of PConv is only 1 / 16 of that of conventional Conv. In addition, the memory access of PConv is also smaller, i.e. For r=1 / 4, it is only 1 / 4 of the regular Conv;

[0170] Step 2.4: Replace the original CBS module with the FB module in step 2.2, and then send the result after Concat to the FSB module in step 2.3 to reduce redundant calculations and extract space more effectively. At the same time, drawing on the idea of ​​the Resnet network, a residual edge is added to the right side of the SPPF, and the feature extraction ability of the network is enhanced through the CA attention mechanism to form an FMB module, which not only considers channel information, but also incorporates direction-related position information, and can effectively use the previously accumulated information to extract richer features for the back propagation of the network.

[0171] Step 2.5: Combine the CIOU loss function and the EIOU loss function at the prediction layer to form a new ECIOU loss function, as follows:

[0172] CIoU loss adds the aspect ratio v of the predicted box and the GT box to the DIoU loss. It takes into account three important geometric factors: overlapping area, center point distance, and aspect ratio. CIoU uses IoU, Euclidean distance, corresponding aspect ratio, and angle to measure the overlapping area between the target and the real box. The formula is as follows:

[0173]

[0174] Among them, ρ 2 (b,b gt ) represent the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, α is a positive trade-off parameter, v is used to measure the consistency of the aspect ratio, and w p and h p Represents the width and height of the prediction box, w gt and h gt Represent the width and height of the real frame respectively;

[0175] EIOU_Loss decomposes the aspect ratio influencing factors based on CIOU_Loss and calculates the width and height of the predicted frame and the real frame to solve the problems that may arise in CIOU_Loss. The formula is as follows:

[0176]

[0177] where c w and c h It is the width and height of the minimum bounding box covering the two Boxes;

[0178] The ECIOU loss function first adjusts the aspect ratio of the predicted frame through CIOU until it converges to a suitable range. Then, each edge is fine-tuned through EIOU until the correct value is reached; through this improved loss function, the adjustment of the predicted frame is increased and the regression speed of the frame is accelerated. The ECIOU formula is as follows:

[0179]

[0180] Step 2.5: Send the above results to the Head to judge the feature points and determine whether there is an object corresponding to the feature points.

[0181] Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training;

[0182] Step 3 is implemented as follows:

[0183] Step 3.1: Input the training set images divided in step 2 into the FusionNet network model for training, set the parameter learning rate to 0.001, momentum to 0.937, weight decay to 0.0005, epoch to 300 times, BatchSize = 16, image size to 640 × 640, and use the learning rate decay method of cosine annealing to obtain the training results;

[0184] Step 3.2, use the validation set to test the optimized FusionNet network model, obtain the validation results, and analyze the differences between the training results and the validation results, such as whether they are overfitting or underfitting; if overfitting occurs, it means that the model performs very well on the training set, but very poorly on the validation set, that is, the generalization ability of the model is very poor, which needs to be solved by increasing the number of training data, using regularization constraints, adjusting parameters and hyperparameters, and reducing the complexity of the model; if underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which is usually caused by low model complexity and too few features. It needs to be solved by increasing model complexity, reducing regularization constraints, adding more features, making the input data more expressive, adjusting parameters and hyperparameters, etc.

[0185] Step 3.3: Fine-tune the network hyperparameters on the validation set, fine-tune the learning rate to 0.002, increase the learning rate to obtain better robustness, set batch_size to 16, and obtain the optimized FusionNet network model.

[0186] Step 4: Input the images in the test set into the network model trained in step 3 for detection, and obtain the result map of foreign body detection of power transmission lines under severe weather.

Claims

1. Foreign object detection method for power transmission lines in severe weather based on FusionNet model, It is characterized in that Follow the steps below to implement it: Step 1: Collect images of foreign objects on the transmission line, perform data preprocessing on the collected images, annotate and divide the data set into training set, verification set and test set; Step 2: Establish a network model; Step 3: input the images in the training set into the network model established in step 2 for training, and obtain a network model for foreign body detection of power transmission lines under severe weather conditions based on the FusionNet model through training; Step 4: Input the images in the test set into the network model trained in step 3 for detection, and obtain the result map of foreign body detection of power transmission lines under severe weather.

2. According to claim 1, the method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model, It is characterized in that The step 1 is specifically implemented according to the following steps: Step 1.1: Collect a large number of inspection videos through tower cameras and inspection drones, including images of foreign objects on various transmission lines. Keep the number of foreign object images of each type of transmission line consistent through long-range, close-up and close-up images of foreign objects. Step 1.2, perform data enhancement and expansion on the transmission line foreign body images obtained in step 1.1, including rain, fog, snow, strong light, etc., using a single and multiple methods combined to obtain a transmission line foreign body dataset; Step 1.3: Combine the transmission line foreign body image obtained in step 1.1 and the image obtained in step 1.2 as a transmission line foreign body sample image library, and create a corresponding label file for each image sample in the transmission line foreign body sample image library. The corresponding label file contains the category label of the foreign body, the label name is FM, and is divided into training set, verification set and test set according to 6:2:

2.

3. According to claim 2, the method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model, It is characterized in that The step 2 is specifically implemented according to the following steps: Step 2.1, the FusionNet algorithm network model is divided into three parts: Backbone, Neck, and Head. The SiLU activation function in the CBS module in the original network model is replaced with the Hardswish activation function, and the CA attention mechanism is embedded to redistribute the channel weights to reduce network parameters; Step 2.2, FusionBlock module structure is as follows: The CA attention mechanism decomposes the original channel attention into two parallel feature encodings of direction and position, and aggregates the spatial coordinate information into a new attention map; First, in order to avoid compressing all spatial information into channels and capture long-range spatial interactions with precise location information, the global average pooling is decomposed as follows: The input feature map Input with a size of C×H×W is pooled in the X direction and Y direction to encode each channel, generating feature maps with sizes of C×H×1 and C×1×W respectively. The generated C×1×W feature map is transformed and then Concat operation is performed. The formula is as follows: f = δ(F 1 ([z h , z w )) Equation (3) The feature maps extracted after Concat are spliced ​​according to the spatial dimension to form The feature map of the shape, where r is used to control the reduction rate of the block, and then the feature map is passed through the F1 convolution transformation function and the nonlinear activation function to generate the f intermediate feature map, and then the f intermediate feature map is split into two tensors f according to the spatial dimension. h and fw, the shapes are and Then perform F h And Fw convolution transformation function and Sigmoid activation function to get g h and g w Coordinate attention, finally g h and g w The coordinate attention is multiplied by the original input to get an output of the same shape as the input: The output formula of CoordinateAttention is written as: The Hardswish activation function proposed in MobileNetv3 is used to replace the Silu activation function. The formula is as follows: Step 2.3, FasterNet contains 4 Stages. The first Stage is preceded by the Embedding layer. The Embedding layer is a regular 4x4 convolution with a step size of 4, which is equivalent to reducing the width and height to 1 / 4. The Merging layer is used before the other Stages. The Merging layer is a regular 2x2 convolution with a step size of 2. Each Stage is stacked by several FasterNet blocks. First, the conventional convolution operation is used to extract spatial features on part of the input channel while keeping the remaining channels unchanged. Then, in the regular memory access, it regards the first or last continuous channel as the representative of the entire feature map for calculation. Assuming that the input and output feature maps have the same number of channels, the FLOPs of PConv are only For a typical case of r = 1 / 4, the FLOPs of PConv is only 1 / 16 of that of conventional Conv. In addition, the memory access of PConv is also smaller, i.e. For r=1 / 4, it is only 1 / 4 of the regular Conv; Step 2.4: Replace the original CBS module with the FB module in step 2.2, and then send the concat result to the FSB module in step 2.3, add a residual edge to the right side of SPPF, and enhance the feature extraction ability of the network through the CA attention mechanism to form an FMB module; Step 2.5: Combine the CIOU loss function and the EIOU loss function at the prediction layer to form a new ECIOU loss function, as follows: CIoU uses IoU, Euclidean distance, corresponding aspect ratio and angle to measure the overlapping area between the target and the real box. The formula is as follows: Among them, ρ 2 (b,b gt ) represent the Euclidean distance between the center points of the predicted box and the true box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the true box, α is a positive trade-off parameter, v is used to measure the consistency of the aspect ratio, and w p and h p Represent the width and height of the prediction box respectively, w gt and h gt Represent the width and height of the real frame respectively; EIOU_Loss decomposes the aspect ratio influencing factors based on CIOU_Loss and calculates the width and height of the predicted frame and the real frame to solve the problems that may arise in CIOU_Loss. The formula is as follows: where c w and c h It is the width and height of the minimum bounding box covering the two Boxes; The ECIOU loss function first adjusts the aspect ratio of the predicted frame through CIOU until it converges to a suitable range, and then each edge is fine-tuned through EIOU until the correct value is reached; through this improved loss function, the adjustment of the predicted frame is increased and the regression speed of the frame is accelerated. The ECIOU formula is as follows: Step 2.5: Send the above results to the Head to judge the feature points and determine whether there is an object corresponding to the feature points.

4. According to claim 3, the method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model, It is characterized in that The step 3 is specifically implemented according to the following steps: Step 3.1: Input the training set images divided in step 2 into the FusionNet network model for training, set the parameter learning rate to 0.001, momentum to 0.937, weight decay to 0.0005, epoch to 300 times, BatchSize = 16, image size to 640 × 640, and use the learning rate decay method of cosine annealing to obtain the training results; Step 3.2, use the validation set to test the optimized FusionNet network model, obtain the validation results, and analyze the differences between the training results and the validation results. If overfitting occurs, it means that the model performs well on the training set, but poorly on the validation set, that is, the generalization ability of the model is poor, which needs to be solved by increasing the amount of training data, using regularization constraints, adjusting parameters and hyperparameters, and reducing the complexity of the model; if underfitting occurs, it means that the model performs poorly on both the training set and the validation set, which needs to be solved by increasing the complexity of the model, reducing regularization constraints, and adding more features; Step 3.3: By fine-tuning the network's hyperparameters on the validation set, fine-tuning the learning rate to 0.002, increasing the learning rate, and setting batch_size to 16, we can obtain the optimized FusionNet network model.

5. According to claim 4, the method for detecting foreign objects in power transmission lines under severe weather conditions based on the FusionNet model, It is characterized in that The step 4 is specifically implemented according to the following steps: Step 4.1, input the test set into the optimized FusionNet model obtained in step 3.

3. The specific optimization process is as follows: FusionNet model hyperparameter settings: num_workers = 4, Adam optimizer, attenuation weight coefficient 5*10-4, initial learning rate 1*10-5, set the IoU threshold to 0.5 for the test set test, and fine-tune the learning rate to 0.003 during verification. The loss change of the verification set gradually decreases in the total 300 rounds, and the FusionNet foreign body detection model is obtained; Step 4.2: Use the test set to test the performance of the transmission line foreign object detection algorithm model under severe weather conditions based on the FusionNet model to obtain the detection results.

Images