Lightweight power transmission line foreign matter detection method and device, medium and product

Abstract

The invention discloses a lightweight power transmission line foreign matter detection method and device, a medium and a product, and relates to the field of power transmission line foreign matter target detection, and the method comprises the steps: taking a YOLOv8n network as a basis, replacing the last three convolution blocks in a backbone network of the network and a first convolution block in a neck network with a lightweight adaptive weight down-sampling module, replacing a C2f module of a backbone network of the network with an efficient multi-scale fusion module, and after a connection layer of a neck network of the network, introducing a space and channel reconstruction convolution module to combine the space and channel reconstruction convolution module with the C2f module of the neck network of the network to obtain a lightweight transmission line foreign matter detection model; and obtaining the position of the foreign matter and the type of the foreign matter by using the trained and optimized model. The method can significantly improve the detection precision of a small target under a complex background, enables the model to be lighter, and can be better suitable for the actual foreign matter detection of a power transmission line.

Description

Technical Field

[0001] The present application relates to the technical field of foreign object detection for power transmission lines, and in particular to a lightweight foreign object detection method, equipment, medium and product for power transmission line. Background Art

[0002] Transmission lines are key infrastructure to ensure the continuous supply of electricity, and their safe and stable operation is directly related to the stable development of the national economy and people's livelihood. As the main channel for power transmission, transmission lines are responsible for transmitting the electricity generated by power plants from remote areas to peak power consumption areas such as load centers and cities. By connecting multiple independent power grids, the allocation of power resources is optimized, the supply and demand differences in different regions are balanced, and the stable operation of the power system is ensured. However, due to the wide coverage and complex environment of transmission lines, their operation safety is affected by many factors. First, because the lines are exposed to the wild for a long time, they are very susceptible to failures caused by factors such as bad weather, natural disasters, and interference from animals and plants; secondly, transmission lines are often invaded by various foreign objects, including bird nests, kites, balloons, garbage, etc. Once these foreign objects are entangled or attached to the lines, they will not only increase the load on the lines, but also cause uneven force and wear on the wires, causing arc discharge or short circuit failures. In severe cases, they may even cause large-scale power outages, affecting people's normal lives, and at the same time increase the difficulty and cost of line maintenance and cleaning. Therefore, timely detection and removal of foreign objects on the lines is crucial to ensure the safe and stable operation of transmission lines.

[0003] Traditional power transmission line inspections can no longer meet the actual needs of modern line detection. These methods usually rely on manual inspections, which are cumbersome and time-consuming, and it is difficult to detect potential problems in a timely manner. In addition, manual inspections have many limitations in terms of wide coverage and complex environmental conditions, and are easily affected by factors such as weather and terrain, resulting in low detection efficiency and insufficient accuracy. At present, equipment such as drones are widely used in image transmission and inspection of power transmission lines. Compared with manual inspections, drone inspections have the advantages of low cost, high efficiency, and enhanced safety measures. However, the images collected by drone inspections usually have complex backgrounds, which significantly increase the difficulty of subsequent image processing and analysis, resulting in uncertainty in the detection results. In addition, the shooting distance and shooting angle problems during the inspection will cause the collected foreign objects to be small and easily blocked. Therefore, in order to give full play to the advantages of drone inspections, it is also necessary to continuously optimize shooting technology and image processing algorithms to cope with these complex situations.

[0004] Traditional target detection algorithms generally include two relatively independent processes: feature extraction and feature recognition. In the feature extraction process, the algorithm mainly relies on manually extracted features, which include the size, shape, texture, etc. of the target. Common feature extraction methods include edge detection, scale-invariant feature transform (SIFT), etc. Feature recognition methods mainly include histogram of oriented gradients (HOG), support vector machine (SVM), random forest, etc. Through these methods, the algorithm can accurately classify and locate the target based on feature data. For example: two common Hough transform techniques and line tracing algorithms are used to realize power line detection. Transmission line noise fault detection based on random matrix theory. However, traditional methods have low computational complexity, slow processing speed, and the detection results are easily affected by complex background and lighting conditions.

[0005] In recent years, deep learning has become a fast and powerful tool in the field of image classification and target detection. Compared with traditional image processing methods, target detection technology based on deep learning has stronger fault tolerance and robustness, as well as more stable recognition accuracy, which also has important application value in the field of foreign body target detection on power transmission lines. The current mainstream target detection algorithms based on deep learning can be roughly divided into two categories: two-stage detection algorithms based on candidate regions and single-stage detection algorithms without candidate region generation. The two-stage detection network first obtains the target candidate region, then extracts target features, classifies, and regresses the bounding box. This type of algorithm has a high accuracy, but the detection speed is unsatisfactory. Typical algorithms include Faster R-CNN, Mask R-CNN, Cascade R-CNN, etc. The single-stage detection network directly performs target classification and bounding box regression after extracting features, and outputs the predicted target position and category, but the accuracy will drop significantly when dealing with small targets and complex backgrounds. Typical algorithms include YOLO series, SSD, etc. In recent years, scholars have been continuously conducting research in the field of power transmission line detection. Zhu Jinguo et al. designed a network with directional bounding boxes based on regression strategy, and proposed a simple and effective scale histogram matching method for detecting foreign objects on transmission lines, which can improve the detection performance even with limited annotated samples. Hao Qiang et al. used triplet attention (TA) and improved bidirectional feature pyramid network (BiFPN) to enhance the ability to extract distinguishing features, thereby obtaining higher semantic information after cross-layer fusion. Wang Yaru et al. constructed a dual-branch downsampling module in the YOLOv8 neck network and inserted a hybrid enhanced attention module in the backbone network to achieve efficient fusion of semantic information and detail information, and enhance the network's ability to capture key features of the target. Although the above algorithms have improved the detection effect of foreign objects on transmission lines, there is still a lot of room for improvement in the detection accuracy and model lightweight of small targets under complex backgrounds, and they cannot be better applied to actual foreign object detection on transmission lines. Summary of the invention

[0006] The purpose of this application is to provide a lightweight transmission line foreign body detection method, equipment, medium and product, which can significantly improve the detection accuracy of small targets under complex backgrounds and make the model more lightweight, so that it can be better applied to actual transmission line foreign body detection.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides a method for detecting foreign objects in a lightweight power transmission line, the method comprising:

[0009] Acquire a training data set; the training data set includes a plurality of power transmission line image samples marked with the location and type of foreign objects;

[0010] Based on the YOLOv8n network, the last three convolution blocks in the backbone network of the YOLOv8n network and the first convolution block in the neck network are replaced with a lightweight adaptive weight downsampling module, the C2f module of the backbone network of the YOLOv8n network is replaced with an efficient multi-scale fusion module, and after the connection layer of the neck network of the YOLOv8n network, a space and channel reconstruction convolution module is introduced to combine the space and channel reconstruction convolution module with the C2f module of the neck network of the YOLOv8n network to obtain a lightweight transmission line foreign body detection model;

[0011] Using the training data set to train and optimize the lightweight power transmission line foreign object detection model, to obtain an optimized lightweight power transmission line foreign object detection model;

[0012] The image of the power transmission line to be detected is input into the optimized lightweight power transmission line foreign object detection model, and the location and type of the foreign object in the image of the power transmission line to be detected are obtained using the optimized lightweight power transmission line foreign object detection model.

[0013] Optionally, the lightweight adaptive weight downsampling module comprises two branches, wherein one branch extracts weight information through average pooling, 1×1 convolution, array dimension transformation operation and Softmax activation function, and the other branch extracts weight information through depthwise separable convolution with a step size of 2, and the weight information extracted by the two branches is obtained through a weighted sum operation to obtain the features extracted by the lightweight adaptive weight downsampling module.

[0014] Optionally, the calculation of the lightweight adaptive weight downsampling module is expressed as: LWM = Sotmax(c 1×1 (AvgPool(X)))×ReLU(Norm(d k×k (X)));

[0015] Wherein, LWM represents the features extracted by the lightweight adaptive weight downsampling module, c 1×1 represents a convolution of size 1×1, d k×k represents a depthwise separable convolution of size k×k, X represents the input feature map, AvgPool represents average pooling, Norm represents normalization, Softmax represents the Softmax activation function, ReLU represents the ReLU activation function, and k represents the size of the convolution kernel.

[0016] Optionally, the efficient multi-scale fusion module is obtained by introducing an EMA attention mechanism in a Fasterblock module.

[0017] Optionally, the space and channel reconstruction convolution module is combined with the C2f module of the neck network of the YOLOv8n network to obtain a C2f-SCConv module; the C2f-SCConv module is improved by introducing the space and channel reconstruction convolution module to form a new SCBlock module, and the bottleneck module in the C2f module of the neck network of the YOLOv8n network is replaced by the new SCBlock module.

[0018] Optionally, the spatial and channel reconstruction convolution module consists of a spatial reconstruction unit and a channel reconstruction unit.

[0019] Optionally, a stochastic gradient descent optimizer is used to perform optimization during the process of training and optimizing the lightweight power transmission line foreign object detection model using the training data set.

[0020] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described lightweight transmission line foreign object detection methods.

[0021] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described lightweight transmission line foreign object detection methods.

[0022] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned lightweight transmission line foreign object detection methods.

[0023] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0024] The present application provides a lightweight power transmission line foreign body detection method, device, medium and product. By replacing the last three convolution blocks in the backbone network of the YOLOv8n network and the first convolution block in the neck network with a lightweight adaptive weight downsampling module, the lightweight adaptive weight downsampling module can generate deeper feature information for the foreign body target of the power transmission line, enhance the feature extraction capability of small targets, and achieve a lighter effect at the same time. By replacing the C2f module of the backbone network of the YOLOv8n network with an efficient multi-scale fusion module, the efficient multi-scale fusion module can be used to fuse features of different scales, thereby capturing more levels of information, effectively integrating features from different resolutions, and improving the ability to understand small targets in complex scenes. While ensuring lightweight, it can capture more scale features and improve the performance and efficiency of the overall model. By introducing the spatial and channel reconstruction convolution module and combining it with the C2f module of the neck network of the YOLOv8n network, the C2f module combined with the spatial and channel reconstruction convolution module can reduce the number of model parameters and calculations while ensuring model accuracy. Based on the lightweight adaptive weight downsampling module, the efficient multi-scale fusion module and the C2f module combined with the spatial and channel reconstruction convolution module, the problem that the existing transmission line foreign object detection algorithm is easily disturbed by the complex background environment and difficult to detect tiny targets is solved. The detection accuracy of small targets under complex backgrounds is significantly improved, and the model is made more lightweight, so that it can be better applied to actual transmission line foreign object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a lightweight transmission line foreign body detection method provided in one embodiment of the present application;

[0027] Figure 2 This is the network structure diagram of the YOLOv8n-LAF algorithm;

[0028] Figure 3 This is the LWM module structure diagram;

[0029] Figure 4 This is the module structure diagram of Fasterblock-EMA;

[0030] Figure 5 This is the C2f-SCConv module structure diagram;

[0031] Figure 6 This is the structure diagram of the SCConv module;

[0032] Figure 7 This is the visualization result of the original image;

[0033] Figure 8 This is the visualization result after FasterR-CNN algorithm detects the original image;

[0034] Fig. 9 This is the visualization result after the YOLOv3 algorithm detects the original image;

[0035] Fig.10 This is the visualization result after the YOLOv5s algorithm detects the original image;

[0036] Fig.11 This is the visualization result after the YOLOv7-tiny algorithm detects the original image;

[0037] Fig.12 This is the visualization result after the YOLOv8n algorithm detects the original image;

[0038] Fig.13 This is the visualization result after the YOLO-LAF algorithm detects the original image;

[0039] Fig.14 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] The purpose of this application is to provide a lightweight transmission line foreign body detection method, equipment, medium and product, which can significantly improve the detection accuracy of small targets under complex backgrounds and make the model more lightweight, so that it can be better applied to actual transmission line foreign body detection.

[0042] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0043] like Figure 1 As shown, the present application provides a lightweight transmission line foreign body detection method, comprising:

[0044] Step 101: Obtain a training data set; the training data set includes a plurality of power transmission line image samples marked with the location and type of foreign objects.

[0045] Step 102: Based on the YOLOv8n network, the last three convolution blocks in the backbone network of the YOLOv8n network and the first convolution block in the neck network are replaced with a lightweight adaptive weight downsampling module, and the C2f module of the backbone network of the YOLOv8n network is replaced with an efficient multi-scale fusion module. After the connection layer of the neck network of the YOLOv8n network, a spatial and channel reconstruction convolution module is introduced to combine the spatial and channel reconstruction convolution module with the C2f module of the neck network of the YOLOv8n network to obtain a lightweight transmission line foreign body detection model.

[0046] In step 102, the lightweight adaptive weight downsampling module includes two branches, wherein one branch extracts weight information through average pooling, 1×1 convolution, array dimension transformation operation and Softmax activation function, and the other branch extracts weight information through depthwise separable convolution with a step size of 2. The weight information extracted by the two branches is subjected to a weighted sum operation to obtain the features extracted by the lightweight adaptive weight downsampling module.

[0047] The calculation of the lightweight adaptive weight downsampling module is expressed as: LWM = Softmax (c 1×1 (AvgPool(X)))×ReLU(Norm(d k×k (X))); where LWM represents the features extracted by the lightweight adaptive weight downsampling module, c 1×1 represents a convolution of size 1×1, d k×k represents a depthwise separable convolution of size k×k, X represents the input feature map, AvgPool represents average pooling, Norm represents normalization, Softmax represents the Softmax activation function, ReLU represents the ReLU activation function, and k represents the size of the convolution kernel.

[0048] In step 102, the efficient multi-scale fusion module is obtained by introducing the EMA attention mechanism into the Fasterblock module.

[0049] In step 102, the spatial and channel reconstruction convolution module is combined with the C2f module of the neck network of the YOLOv8n network to obtain a C2f-SCConv module; the C2f-SCConv module is obtained by improving the bottleneck module in the C2f module of the neck network of the YOLOv8n network by introducing the spatial and channel reconstruction convolution module to form a new SCBlock module, and replacing the bottleneck module in the C2f module of the neck network of the YOLOv8n network with the new SCBlock module.

[0050] Specifically, the spatial and channel reconstruction convolution module consists of a spatial reconstruction unit and a channel reconstruction unit.

[0051] Step 103: Use the training data set to train and optimize the lightweight transmission line foreign object detection model to obtain an optimized lightweight transmission line foreign object detection model.

[0052] In step 103, a stochastic gradient descent optimizer is used to optimize the lightweight power transmission line foreign object detection model during training and optimization using the training data set.

[0053] Step 104: input the transmission line image to be detected into the optimized lightweight transmission line foreign object detection model, and use the optimized lightweight transmission line foreign object detection model to obtain the location and type of the foreign object in the transmission line image to be detected.

[0054] The technical solution of this application is described below with a specific embodiment:

[0055] In order to improve the robustness of the foreign body detection algorithm for power transmission lines under different environmental conditions, and to address the problem that the existing algorithms are easily disturbed by the background environment and difficult to detect small targets, this application proposes a lightweight foreign body detection method for power transmission lines, which is a lightweight foreign body detection algorithm for power transmission lines combined with adaptive weight downsampling (named YOLO-LAF algorithm in this application). First, this application designs a lightweight adaptive weight downsampling module (named LWM module in this application) to generate deeper feature information for foreign body targets in power transmission lines and enhance the feature extraction capability for small targets; secondly, combining the Fasterblock (i.e., FasterBlock) module with the EMA attention mechanism, an efficient multi-scale fusion module (named Fasterblock-EMA module in this application, referred to as FEA module) is proposed to achieve cross-channel interaction and speed up the real-time detection speed of the algorithm; finally, the spatial and channel reconstruction convolution (SCConv) is introduced and fused with the C2f module, which significantly reduces the complexity and computational cost of the model, effectively reduces redundant features, and achieves better performance. The experimental results show that the average precision (mAP@0.5) of the improved algorithm in this application for foreign object detection in the transmission line scenario reaches 91.2%. Compared with the YOLOv8n algorithm, the accuracy is improved by 2.6%, and the model parameters and calculation amount are reduced by 7.1×10 6 M and 1.2GFLOPS. Compared with the existing models, the YOLO-LAF model has a smaller size and higher detection accuracy, which significantly improves the performance and efficiency of foreign object detection on transmission lines.

[0056] The following is a detailed introduction to the lightweight transmission line foreign object detection algorithm combined with adaptive weight downsampling proposed in this application:

[0057] In order to improve the robustness of the target detection network, considering the limited computing resources of the edge detection equipment, this application takes the YOLOv8n algorithm as the basic model, makes targeted improvements to its backbone network and neck network, and proposes a lightweight transmission line foreign object detection algorithm combined with adaptive weight downsampling, which is a lightweight multi-scale transmission line foreign object detection algorithm fused with adaptive weight downsampling (named YOLOv8n-LAF algorithm in this application). The network structure diagram of the YOLOv8n-LAF algorithm is shown in the figure. Figure 2 As shown, first, the last three convolution blocks in the original backbone network and the first convolution block in the neck network are replaced with the lightweight adaptive weight downsampling module proposed in this application, and the weights are dynamically adjusted according to the different input features, so as to maintain good generalization in different scenarios. Secondly, the C2f module of the backbone network is replaced by an efficient multi-scale fusion module (i.e., an efficient multi-scale fusion module), which effectively integrates features from different resolutions and improves the ability to understand small targets in complex scenes. Finally, after the neck network connection layer (i.e., the neck network Concat connection layer), the spatial and channel reconstruction convolution module is introduced to combine it with the original C2f module, which significantly reduces the complexity and computational cost of the model, effectively reduces redundant features, and thus achieves better performance.

[0058] The YOLOv8 model is introduced as follows:

[0059] YOLOv8 is a new generation of YOLO series real-time object detection models launched by Ultralytics. It inherits the success of the previous YOLOv5 version and introduces new features and improvements. These improvements not only significantly improve the performance of the model, but also enhance its flexibility and adaptability in a variety of application scenarios. Specifically, the YOLOv8 algorithm combines the design ideas of YOLOv7ELAN and adopts the C2f structure of YOLOv5 in the backbone and neck networks to obtain richer gradient flows. Secondly, YOLOv8 uses feature pyramid network (FPN) and path aggregation network (PAN) to fuse multi-scale features to enhance the feature extraction ability of the model. In addition, YOLOv8 uses more efficient activation functions such as SiLU or Mish to improve the nonlinear expression ability of the model. It also uses an improved loss function that combines classification loss, bounding box regression loss and confidence loss to ensure the accuracy and robustness of the detection results. Through these improvements, the speed and accuracy of YOLOv8 in real-time object detection have been significantly improved.

[0060] The following is a detailed introduction to the network improvements of this application:

[0061] 1. Lightweight adaptive weight downsampling module

[0062] Aiming at the problem of unbalanced feature extraction of foreign object detection images in complex environments of power transmission lines, that is, small targets may be ignored during the downsampling process, resulting in loss of details, as well as limited computing power and memory of model equipment. This application designs a lightweight adaptive weight downsampling module (i.e., lightweight adaptive weight downsampling module), which contains two branches. The first branch generates a weight map through average pooling and 1×1 convolution, and then calculates the importance of each position of the attention weight map through array dimension transformation operations, trying to retain key information and features, thereby realizing the adaptive downsampling process. Finally, the generated weights are normalized into a probability distribution using the Softmax activation function to ensure that the sum of the weights of each area is 1. The second branch draws on the idea of ​​Focus slicing operation and saves the pixels on the feature map into the channel. Since the slicing operation is time-consuming and labor-intensive, the use of a depth-separable convolution with a step size of 2 makes the model more lightweight (i.e., lightweight). The depth-separable convolution performs convolution operations separately on each channel, significantly reducing the amount of calculation and the amount of parameters, significantly improving the inference speed of the model, and enabling the model to predict more quickly. Finally, the weight information extracted by the two branches is weighted summed to ensure that the model maintains the diversity of feature information and improves the overall calculation speed and detection performance. Figure 3 In summary, the calculation of the LWM module can be expressed as:

[0063] LWM=Softmax(c 1×1 (AvgPool(X)))×ReLU(Norm(d k×k (X)))

[0064] Among them, c 1×1 represents a convolution of size 1×1, d k×k represents a depthwise separable convolution of size k×k, Norm represents normalization, X represents the input feature map, AvgPool represents average pooling, and k×k represents the size of the convolution kernel. In order to minimize the computational overhead and the number of parameters, AvgPool is used to aggregate the global information of each receptive field feature, and then 1×1 convolution is used for information interaction. Finally, Softmax is used to emphasize the importance of each feature in the receptive field feature. The array dimension transformation formula is as follows:

[0065]

[0066] Among them, DIM represents the dimension size of the weight array, bs represents the size of the batch size, ch represents the number of input channels of the feature map, h and w represent the height and width of the feature map respectively, and S represents the weight information, whose size is 4. It can be seen from the above formula that after the downsampling operation, the height and width of the original feature map are reduced to half of the original, the number of channels remains unchanged, and the feature information of the original image is retained in the weight channel S, so that the network can better focus on the details and effectively capture key features even in complex environments. Depthwise separable convolution splits the standard convolution into depthwise convolution and pointwise convolution, where the depthwise convolution performs convolution operations on each channel of the input separately, that is, each input channel is convolved with a separate convolution kernel. Pointwise convolution uses a 1×1 convolution kernel to further process the output of the depthwise convolution. This operation can reduce the computational overhead of the model in the training and inference stages, and at the same time help alleviate the redundancy problem in the channel dimension.

[0067] 2. Fasterblock-EMA module

[0068] The foreign object detection model for transmission lines requires a large amount of data for training in order to accurately identify and detect foreign objects on the transmission lines and ensure the safe and stable operation of the power system. Since these models contain a large number of parameters, they usually rely on high-performance computing equipment and advanced algorithm optimization techniques. However, given the limited equipment resources, the efficiency of the model must be improved by improving the algorithm computing power. In view of the above reasons, this application introduces the Fasterblock module in the YOLOv8n algorithm to reduce redundant convolution calculations and memory accesses, thereby improving the running speed and resource utilization efficiency of the model. At the same time, the EMA attention mechanism is introduced in the Fasterblock module to construct an efficient multi-scale fusion module FEA, which improves the performance and efficiency of the model in different scenarios. The Fasterblock module consists of four parts: Partial Convolution (PConv), Convolution (Conv), Batch Normalization (BN), and Rectified Linear Unit (ReLU). Among them, PConv applies regular Conv on some input channels for spatial feature extraction, while the remaining channels remain unchanged. In addition, for continuous or regular memory access, PConv regards the first or last continuous channel as the representative of the entire feature map for calculation, so that the feature maps of the algorithm input and output have the same number of channels. Therefore, the FLOP formula of PConv is as follows:

[0069] FLOPs = h × w × k 2 ×c 2 p

[0070] Among them, PLOPs represents the number of floating-point operations per second, h and w are the height and width of the feature map, k is the convolution kernel size, and c is the number of floating-point operations per second. p is the number of channels for conventional convolution. In actual implementation, Therefore, the FLOPs of PConv is only 1 / 16 of that of regular convolution, where r represents the ratio of the number of channels of regular convolution and P convolution (i.e. PConv), and c represents the number of channels affected by P convolution. In addition, the memory access amount of PConv is shown in the following formula:

[0071] AMC=h×w×2c p +k 2 ×c 2 p

[0072] The number of memory accesses of PConv is only 1 / 4 of that of regular convolution, and the rest (cc p ) channels do not participate in the calculation, so there is no need to access memory.

[0073] Modeling cross-channel relationships through channel dimensionality reduction may have side effects on extracting deep vision. Therefore, this application introduces an efficient multi-scale attention module (EMA) that does not require dimensionality reduction, focusing on retaining the information of each channel and reducing computational overhead. It reshapes some channels into batch dimensions and groups the channel dimensions into multiple sub-features so that spatial semantic features are evenly distributed in each feature group. In addition, EMA also provides a method for aggregating cross-spatial information in different spatial dimensional directions to achieve richer feature aggregation. By combining the EMA module with the Fasterblock module, its memory access, parameter amount, and computational amount are reduced to a certain extent. While ensuring lightweight, it can capture more scale features and improve the performance of the model. The Fasterblock-EMA module structure diagram is shown below. Figure 4 shown.

[0074] 3. C2f-SCConv module

[0075] In the process of target detection, it is a common problem that the convolutional layer extracts redundant features. These redundant features not only increase the computational burden and memory usage of the model, but may also cause the model to perform poorly when dealing with complex scenes and small-sized targets, increase the complexity of the model, and make the model more difficult to optimize and debug. Therefore, reducing and effectively managing the redundant features extracted by the convolutional layer is crucial to optimizing the key performance of the target detection model. This application improves the Bottleneck module in the original C2f structure by introducing spatial and channel reconstruction convolution (SCConv), forming a new SCBlock module, and replacing the original Bottleneck module with the SCBlock module to obtain the C2f-SCConv module, whose structure is as follows: Figure 5 As shown, h and w are the height and width of the feature map, c is the number of channels of the feature map, and n is the number of layers of the feature map.

[0076] The structure of the SCConv module is as follows Figure 6 As shown in the figure, the SCConv module consists of two units, namely the spatial reconstruction unit (SRU) and the channel reconstruction unit (CRU). The SRU separates the redundant features according to the weights and reconstructs them to suppress the redundancy in the spatial dimension and enhance the representation of the features. The CRU adopts a split-transform-fusion strategy to reduce channel redundancy as well as computational cost and storage. Through the spatial reconstruction unit and the channel reconstruction unit (referred to as the spatial and reconstruction units), the SCConv module is able to better capture the complex relationship of the input features, aiming to effectively limit feature redundancy, which not only reduces the number of model parameters and FLOPs, but also enhances the ability of feature representation.

[0077] The following is a detailed introduction to the experimental results and analysis of this application:

[0078] 1. Experimental environment and parameter configuration

[0079] This application uses the Stochastic Gradient Descent (SGD) optimizer during model training to reduce the possibility of the model falling into the local optimum. The training environment is Pytorch 2.0.1, CUDA 11.3 and Python 3.9.0, and the detailed server configuration is shown in Table 1. The input training sample is a 640×640 three-channel image, the initial learning rate is 0.01, the momentum factor is 0.937, the hyp hyperparameter is used, the batch size is set to 16, and a total of 300 rounds of training are performed.

[0080] Table 1 Server configuration

[0081] Configuration Name Version / Parameters operating system Ubuntu 20.04LTS GPU RTX4090ti*2 RAM 24GB Memory 2TB SATA

[0082] 2. Experimental Dataset

[0083] In order to evaluate the superiority of the target detection algorithm proposed in this application, this application uses the tower foreign object detection image dataset disclosed by the Southern Power Grid. This dataset contains 2,400 images, covering four different types of foreign objects such as bird nests, kites, balloons, and garbage. Among them, this application uses 1,700 images as a validation set, and the remaining 700 images are expanded to 1,300 after data enhancement processing such as flipping, scaling, and cropping, and re-annotated in YOLO format using labelme software. The dataset enhancement sample is as follows: Figures 7 to 13 shown.

[0084] The YOLO-LAF model is trained using the China Southern Power Grid public dataset, which has been annotated using the labelme software. The model input is the original 1,680 validation set images and 1,300 images that have been data augmented. A total of 3,000 images are used as model input, and the output is the trained best.pt file used to predict the model.

[0085] 3. Evaluation indicators

[0086] When evaluating the target detection algorithm, it is necessary to consider indicators such as the model's detection accuracy, detection speed, and running memory size. Therefore, this application uses precision, recall, multi-category average precision (mAP), number of floating-point operations (GFLOPs), and parameter count (Params) as evaluation indicators. The specific calculation method is as follows:

[0087]

[0088] Among them, TP is the number of positive samples that are predicted as positive, FP is the number of negative samples that are predicted as positive, FN is the number of positive samples that are not detected as negative, AP is the average precision of each category, mAP is the average of AP of all categories, t represents a symbol between 0 and 1, which has no actual meaning, and N represents a range from 1 to N. T and F in TP, FP, and FN represent the positive and negative nature of the actual samples, and P and N represent the positive and negative nature of the predicted results (where N represents the predicted result is negative).

[0089] 4. Experimental results and analysis

[0090] 4.1 Experimental analysis of lightweight adaptive weight downsampling module

[0091] In order to verify the effectiveness of the lightweight adaptive weight downsampling module proposed in this application in the model and explore its specific effects in different positions, this application uses the original YOLOv8n as the benchmark model, and replaces the 3×3 standard convolution with a step size of 2 in the backbone network and the neck network with the lightweight adaptive weight downsampling module. To ensure that the downsampling process retains as much key information as possible, this application selects the last three convolutional layers in the backbone network and the two convolutional layers in the neck network for replacement. Next, this application discusses the impact of the number and combination of downsampling modules on the overall model performance, and conducts comparative experiments on the public dataset of the Southern Power Grid. The experimental results are shown in Table 2.

[0092] In Table 2, YOLOv8n-LWM-i means that the i-th 3×3 standard convolution in the original backbone network and neck network is replaced by the LWM module. The results show that the detection accuracy of the model is improved to varying degrees by using the LWM module at different positions in the backbone network and neck network, while significantly reducing the number of parameters and the amount of calculation. This proves the effectiveness of the LWM module, indicating that it can not only achieve lightweight, but also improve the detection effect of the model. Among them, the YOLOv8n-LWM-3,4,5,6 model has the best detection effect, with mAP@0.5 reaching 90.7%, which is 2.5% higher than the original YOLOv8n. At the same time, the number of model parameters is reduced by 0.87M, and the amount of calculation has also decreased significantly.

[0093] However, it is worth noting that the more downsampling modules are used, the better. Too many downsampling modules will cause the loss of detailed information in the input data, making it difficult for the model to capture key features, thus affecting the final prediction results. Therefore, it is necessary to arrange the use of downsampling modules reasonably to balance computational efficiency and model accuracy to ensure the optimal performance of the model.

[0094] Table 2 Verification experimental results of lightweight adaptive weight downsampling module

[0095]

[0096] 4.2 Attention Mechanism Selection Experiment

[0097] In order to evaluate the effectiveness of the EMA attention mechanism in the efficient multi-scale fusion module, this application compares the performance of the EMA attention mechanism with the SE, CBAM, ECA, CA, and SimAM attention mechanisms under the same experimental environment and parameter settings, and analyzes the impact of different attention mechanisms on the model detection effect. According to the data in Table 3, the model detection accuracy of the EMA attention mechanism is the highest, the precision rate is improved by 1.3%, the recall rate and the number of parameters do not change much, the calculation amount is reduced by 1GFLOPs, and the mAP is improved by 1.4%. In summary, the EMA attention mechanism performs best among all the tested attention mechanisms, not only significantly improving the detection accuracy of the model, but also reducing the calculation cost and improving the calculation efficiency while maintaining high performance, demonstrating its significant advantages in the efficient multi-scale fusion module.

[0098] Table 3 Comparative experimental results of attention mechanism

[0099]

[0100]

[0101] 4.3 Ablation Experiment

[0102] By adding three modules, LWM, Fasterblock-EMA and C2f-SCConv, to the YOLOv8n algorithm, the impact of these modules on the algorithm performance was discussed in detail. The experimental results show that each module has shown significant effects in improving detection accuracy and optimizing network complexity. The experimental results are shown in Table 4. After replacing the standard convolution in the original network with the LWM module in Group B, the amount of parameters and the amount of calculation were effectively reduced. Specifically, the number of parameters was reduced from 3.0M to 2.7M, and the amount of calculation was reduced from 8.1GFLOPs to 7.9GFLOPs; Group C introduced the Fasterblock-EMA module to reduce the number of parameters and the amount of calculation of the algorithm to 2.7M and 7.9GFLOPs respectively, which shows that the Fasterblock-EMA module helps to improve the operating efficiency of the model in a resource-constrained environment; Group D replaced the C2f module in the original network with the C2f-SCConv module, which not only reduced the parameters and the amount of calculation, but also further improved the detection accuracy. This shows that the C2f-SCConv module can achieve more while maintaining high performance. Lightweight model design; after adding two improved modules on the basis of the original YOLOV8n network structure, Group E, Group F and Group G achieved different degrees of improvement in parameters, computational complexity and detection accuracy; Group H is the final improvement effect of this experiment. The results show that compared with the original YOLOv8n model, the model parameters and floating-point operations are reduced by 0.59M and 1.3GFLOPs respectively, and the mAP value is increased by 2.6%, which shows that by adding a lightweight adaptive weight downsampling module and introducing spatial and channel reconstruction convolutions, the image details can be better captured, thereby improving the accuracy and robustness of the model. The use of the Fasterblock-EMA module can significantly speed up the training speed, reduce the consumption of computing resources, and meet the expected real-time detection needs.

[0103] Table 4 Ablation experiment results

[0104]

[0105]

[0106] In Table 4, × indicates that the module is not added, and √ indicates that the module is added.

[0107] 4.4 Comparative Experiment

[0108] In order to further verify the performance of the YOLO-LAF detection model, the improved algorithm of this application is compared with the current mainstream target detection algorithms such as Faster-RCNN, YOLOv3, YOLOv5s, YOLOX, YOLOv7-tiny, and YOLOv8s using the same data set and in the same experimental environment. The experimental results are shown in Table 5.

[0109] Table 5 Comparative experimental results

[0110]

[0111] As shown in Table 5, Faster R-CNN is an early developed target detection algorithm. Its two-stage detection characteristics lead to relatively large parameters and calculations. This multi-step processing process not only increases the complexity of the model, but also makes the detection speed relatively slow. For this experiment with the goal of lightweight, the Faster R-CNN algorithm is not applicable. YOLOv3, YOLOv5s, YOLOX, and YOLOv7 are classic representative algorithms of the YOLO series. With the update of the version, their calculation amount and parameter amount are gradually reduced, and the detection accuracy and speed are also continuously improved. Compared with the Faster R-CNN algorithm, there has been a significant improvement, but there is still a lot of room for improvement in lightweight and detection accuracy. As the current advanced target detection algorithm, the YOLOv8 series not only further reduces the amount of calculation and parameters, but also significantly improves the detection accuracy and speed by optimizing the network structure, but these algorithms still have room for improvement. The YOLO-LAF algorithm proposed in this application reduces the model parameters and calculation amount to 2.35×10 6 The model accuracy is as high as 91.2%, and the overall performance is excellent. It not only has a significant improvement in accuracy, but also is more efficient in resource utilization. It meets the needs of modern power transmission line inspection for intelligence and efficiency, and provides strong technical support for improving the accuracy and real-time performance of inspections.

[0112] In order to more intuitively compare the effects of the transmission line foreign body detection model before and after the improvement, this application selects representative images for visual analysis. Some foreign body detection effects are as follows: Figures 7 to 13 As shown. Among them, Figure 7 This is the visualization result of the original image. Figure 7 Part (a) in the middle is the visualization result of the original image containing two nests. Figure 7 Part (b) shows the visualization result of the original image containing a bird’s nest. Figure 7 Part (c) is the visualization result of the original image containing trash; Figure 8 This is the visualization result after Faster R-CNN algorithm detects the original image. Figure 8 Part (a) shows the visualization result of the Faster R-CNN algorithm detecting the original image containing two bird nests. Figure 8 Part (b) shows the visualization result of the Faster R-CNN algorithm detecting the original image containing a bird’s nest. Figure 8 Part (c) shows the visualization result after Faster R-CNN algorithm detects the original image containing garbage. Fig. 9 This is the visualization result after the YOLOv3 algorithm detects the original image. Fig. 9 Part (a) shows the visualization result of the YOLOv3 algorithm detecting the original image containing two bird nests. Fig. 9 Part (b) shows the visualization result of the YOLOv3 algorithm detecting the original image containing a bird’s nest. Fig. 9 Part (c) shows the visualization result of the YOLOv3 algorithm detecting the original image containing garbage. Fig.10 This is the visualization result after the YOLOv5s algorithm detects the original image. Fig.10 Part (a) shows the visualization result of the YOLOv5s algorithm detecting the original image containing two bird nests. Fig.10 Part (b) shows the visualization result of the YOLOv5s algorithm detecting the original image containing a bird’s nest. Fig.10 Part (c) shows the visualization result of the YOLOv5s algorithm detecting the original image containing garbage. Fig.11 This is the visualization result after the YOLOv7-tiny algorithm detects the original image. Fig.11 Part (a) is the visualization result of the YOLOv7-tiny algorithm detecting the original image containing two bird nests. Fig.11 Part (b) shows the visualization result of the YOLOv7-tiny algorithm detecting the original image containing a bird’s nest. Fig.11 Part (c) shows the visualization result of the YOLOv7-tiny algorithm detecting the original image containing garbage. Fig.12 This is the visualization result after the YOLOv8n algorithm detects the original image. Fig.12 Part (a) is the visualization result of the YOLOv8n algorithm detecting the original image containing two bird nests. Fig.12 Part (b) shows the visualization result of the YOLOv8n algorithm detecting the original image containing a bird’s nest. Fig.12 Part (c) shows the visualization result of the YOLOv8n algorithm detecting the original image containing garbage. Fig.13 This is the visualization result after the YOLO-LAF algorithm detects the original image. Fig.13 Part (a) is the visualization result of the YOLO-LAF algorithm detecting the original image containing two bird nests. Fig.13 Part (b) shows the visualization result of the YOLO-LAF algorithm detecting the original image containing a bird’s nest. Fig.13 Part (c) shows the visualization result of the YOLO-LAF algorithm after detecting the original image containing garbage. Figures 7 to 13By comparing the visualization results, it can be seen that the Faster R-CNN and YOLOv3 algorithms have serious false detection and missed detection phenomena. For example, the Faster R-CNN and YOLOv3 algorithms did not detect garbage. Although the YOLOv5s, YOLOv7-tiny, and YOLOv8n algorithms have improved in performance and effect, there is still room for improvement in detection accuracy and speed. For example, YOLOv5s detects garbage as balloons. The improved algorithm YOLO-LAF algorithm in this application solves the problem of missed detection and false detection of small foreign objects in YOLOv8n. At the same time, the model detection accuracy is significantly improved, and the detection speed also meets the real-time requirements, which can be better applied to the actual transmission line foreign object detection.

[0113] Compared with the prior art, this application proposes an efficient YOLOv8n improved model YOLO-LAF model to improve the robustness of the power transmission line foreign object detection algorithm under different environmental conditions. In order to enhance the feature extraction capability of the network, this application designs a lightweight adaptive weight downsampling module and proposes an efficient multi-scale fusion module to enrich the features of small target semantic information and accelerate the detection efficiency of the model. The spatial and channel reconstruction convolution and C2f module fusion are introduced into the neck network to reduce the model parameters and calculation amount while ensuring accuracy. Experimental results show that the mAP of the YOLO-LAF algorithm of this application is improved by 2.6%, and the number of parameters and calculation amount are reduced by 7.1×10 6 M and 1.2GFLOPS, compared with the original model, it has better detection accuracy, robustness and more efficient detection speed, and has good application prospects.

[0114] The lightweight transmission line foreign body detection method proposed in this application is a lightweight improved algorithm based on the YOLOv8n network, which solves the problem that important information of small targets is easily lost during downsampling and the problem that the computing resources of the equipment are limited and the speed is slow. The main contributions of the lightweight improved algorithm based on the YOLOv8n network in this application are as follows:

[0115] (1) In order to improve the performance of feature extraction during model training, this application designs a lightweight adaptive weight downsampling module, which can better capture and retain important feature information, reduce information loss, and improve the quality of feature representation after downsampling by adaptively adjusting weights.

[0116] (2) By combining the Fasterblock module and the EMA attention mechanism, this application proposes an efficient multi-scale fusion module that can capture more levels of information by fusing features of different scales, while improving the efficiency and performance of the overall model.

[0117] (3) By introducing spatial and channel reconstruction convolution and fusing it with the C2f module to form the C2f-SCConv module, the feature representation is richer and more diverse. In addition, the C2f-SCConv module can significantly reduce parameters and computational complexity, effectively reduce redundant features, and maintain the high efficiency of the model.

[0118] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.14 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store lightweight transmission line foreign body detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a lightweight transmission line foreign body detection method is implemented.

[0119] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0120] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0121] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0123] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0124] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0125] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting foreign matter in a lightweight power transmission line, characterized in that: The lightweight transmission line foreign body detection method comprises: Acquire a training data set; the training data set includes a plurality of power transmission line image samples marked with the location and type of foreign objects; Based on the YOLOv8n network, the last three convolution blocks in the backbone network of the YOLOv8n network and the first convolution block in the neck network are replaced with a lightweight adaptive weight downsampling module, the C2f module of the backbone network of the YOLOv8n network is replaced with an efficient multi-scale fusion module, and after the connection layer of the neck network of the YOLOv8n network, a space and channel reconstruction convolution module is introduced to combine the space and channel reconstruction convolution module with the C2f module of the neck network of the YOLOv8n network to obtain a lightweight transmission line foreign body detection model; Using the training data set to train and optimize the lightweight power transmission line foreign object detection model, to obtain an optimized lightweight power transmission line foreign object detection model; The image of the power transmission line to be detected is input into the optimized lightweight power transmission line foreign object detection model, and the location and type of the foreign object in the image of the power transmission line to be detected are obtained using the optimized lightweight power transmission line foreign object detection model.

2. The method for detecting foreign matter in a lightweight power transmission line according to claim 1, characterized in that: The lightweight adaptive weight downsampling module comprises two branches, wherein one branch extracts weight information through average pooling, 1×1 convolution, array dimension transformation operation and Softmax activation function, and the other branch extracts weight information through depthwise separable convolution with a step size of 2. The weight information extracted by the two branches is subjected to a weighted sum operation to obtain the features extracted by the lightweight adaptive weight downsampling module.

3. The method for detecting foreign matter in a lightweight power transmission line according to claim 2, characterized in that: The calculation of the lightweight adaptive weight downsampling module is expressed as: LWM = Softmax (c 1×1 (AvgPool(X)))×ReLU(Norm(d k×k (X))); Wherein, LWM represents the features extracted by the lightweight adaptive weight downsampling module, c 1×1 represents a convolution of size 1×1, d k×k represents a depthwise separable convolution of size k×k, X represents the input feature map, AvgPool represents average pooling, Norm represents normalization, Softmax represents the Softmax activation function, ReLU represents the ReLU activation function, and k represents the size of the convolution kernel.

4. The method for detecting foreign matter in a lightweight power transmission line according to claim 1, characterized in that: The efficient multi-scale fusion module is obtained by introducing the EMA attention mechanism in the Fasterblock module.

5. The method for detecting foreign matter in a lightweight power transmission line according to claim 1, characterized in that: The space and channel reconstruction convolution module is combined with the C2f module of the neck network of the YOLOv8n network to obtain a C2f-SCConv module; the C2f-SCConv module is improved by introducing the space and channel reconstruction convolution module to form a new SCBlock module, and the bottleneck module in the C2f module of the neck network of the YOLOv8n network is replaced by the new SCBlock module.

6. The method for detecting foreign matter in a lightweight power transmission line according to claim 1, characterized in that: The space and channel reconstruction convolution module consists of a space reconstruction unit and a channel reconstruction unit.

7. The method for detecting foreign matter in a lightweight power transmission line according to claim 1, characterized in that: In the process of training and optimizing the lightweight power transmission line foreign object detection model using the training data set, a stochastic gradient descent optimizer is used for optimization.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lightweight transmission line foreign body detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting foreign matter in a lightweight power transmission line according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting foreign matter in a lightweight power transmission line according to any one of claims 1 to 7 is implemented.

Images