Power transmission line dangerous object identification method and system in complex rain and fog environment

Abstract

The invention discloses a power transmission line dangerous object identification method and system in a rain and fog complex environment, and relates to the technical field of power grid detection, and the method comprises the steps: constructing a power transmission line data set in the rain and fog environment, carrying out the depth marking of the power transmission line data set in the rain and fog environment through a Make Sense platform, obtaining a marked power transmission line data set, and obtaining a power transmission line dangerous object identification result; dividing the data into a training set and a verification set; a lightweight shared convolution detection head is used as a detection head of a YOLOv8n model, SPPF-LSKA is used to replace SPPF in a backbone feature extraction network of the YOLOv8n model, a Triplet-Attention lightweight attention mechanism is introduced after the SPPF-LSKA, and the YOLOv8n model for power transmission line dangerous object identification in a rain and fog environment is obtained; and training the YOLOv8n model for dangerous object identification of the power transmission line in the rainy and foggy environment by using a training set, and detecting the power transmission line by using the trained YOLOv8n model for dangerous object identification of the power transmission line in the rainy and foggy environment. According to the invention, high-efficiency, real-time and accurate all-weather detection of various dangerous objects of the power transmission line in a rain and fog complex environment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid detection, and in particular to a method and system for identifying dangerous objects in power transmission lines in a complex rainy and foggy environment. Background Art

[0002] The protection and prevention of damage to transmission lines plays an irreplaceable role in the protection of my country's electric energy and the stable development of all walks of life. In the current power transmission system, external threats mainly come from improper operation of large-scale construction equipment and circuit short circuits caused by bird nesting, as well as severe weather such as rain and fog. Therefore, security inspections of transmission channels in rain and fog environments are conducive to the safe and stable operation of the transmission system.

[0003] At present, the main methods of security inspection of power transmission lines are manual inspection, drone inspection and video-based online monitoring system of power transmission lines. Among them, the manual inspection method is not only costly, inefficient, time-consuming and labor-intensive, but also often causes missed reports and late reports. In addition, when encountering sudden situations such as ice and snow, earthquakes, and landslides, the security work of the power transmission line will be completely impossible to carry out; the online monitoring system inspection drone is expensive and has insufficient endurance, and the security inspection cost is high and cannot cope with sudden situations; the drone recognition speed is slow and it is difficult to deploy a large-scale target detection system, which leads to low target detection efficiency. In short, these methods are costly, inefficient, lack of real-time and reliability, and are prone to late reports, false reports and missed reports. Therefore, compared with manual and drone inspections, the current power system prefers to use video surveillance for security work. The video surveillance system has the advantages of uninterrupted monitoring all day and night, accurate alarm, good interactivity, and fast processing speed. It can provide timely feedback on intrusions and abnormal situations in the transmission channel, and plays an important role in the entire power security system.

[0004] However, the current online video monitoring system still has many shortcomings: first, the visible light camera leads to reduced image quality and blurred monitoring images under low illumination at night, which greatly reduces the accuracy of abnormal situation judgment and is prone to false alarms and omissions; second, the current online video monitoring system mainly uses the front-end camera to collect video information for the security of the transmission line, transmits video data to the back-end server regularly through the high-speed network, and completes the data processing at the back-end. The inspection mode has high operating costs and lacks real-time and reliability. Therefore, introducing artificial intelligence and embedded technology into the transmission line placement and inspection process to achieve low-cost, efficient and real-time monitoring of dangerous objects around the transmission line is of great significance for early warning of hidden dangers and damage of the transmission line and taking timely countermeasures.

[0005] With the rapid development of artificial intelligence, detection models based on deep convolutional neural networks have been introduced to improve target detection performance. At present, target detection algorithms based on deep learning can basically be divided into two categories: 1) The method based on region proposal is a two-step detection network that first generates candidate regions (RP) and then classifies samples through convolutional neural networks (CNN), such as the RCNN series of algorithms; 2) One-step detection models based on one-stage regression, such as the YOLO series, SSD, CenterNet, and detection models improved on this basis. Since the first type of target detection model uses candidate region generation and subsequent deep classification, it has significantly improved the accuracy of target detection and positioning accuracy compared to the second type of target detection model, but the target detection speed is relatively slow. In terms of applicability at the edge, the one-step detection model can convert the positioning problem of the target box into a regression problem while ensuring a relatively high accuracy rate. There is no need for the candidate region generation step, which simplifies the detection process. It not only greatly reduces the computing requirements, but is also suitable for deployment on resource-constrained edge devices. In addition, the inference speed has obvious advantages, and can be close to real-time performance, meeting the needs of fast processing at the edge.

[0006] However, the one-step detection model and the two-step detection model are large in size and difficult to deploy on edge devices with limited computing resources, so it is crucial to achieve model lightweighting. However, achieving model lightweighting will inevitably lead to the loss of some target information, and it is impossible to fully extract and distinguish the subtle features of different targets, which in turn leads to a decrease in model accuracy.

[0007] Therefore, a new edge computing solution for identifying hazardous objects on power transmission lines in complex rainy and foggy environments is urgently needed to achieve real-time monitoring of hazardous objects on transmission lines, improve the accuracy of hazardous object detection, and reduce costs. Summary of the invention

[0008] The purpose of the present invention is to provide a method and system for identifying dangerous objects in power transmission lines in complex rain and fog environments. By constructing a YOLOv8n model for identifying dangerous objects in power transmission lines in rain and fog environments, real-time, accurate, and all-weather detection of dangerous objects near power transmission lines is achieved, significantly reducing the inspection cost.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A method for identifying dangerous objects on a power transmission line in a complex rain and fog environment comprises the following steps:

[0011] S1. Construct a data set of power transmission lines in a rainy and foggy environment, and use the Make Sense platform to deeply annotate the data set of power transmission lines in the rainy and foggy environment to obtain an annotated data set of power transmission lines;

[0012] S2. Use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use SPPF-LSKA to replace the SPPF in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism after SPPF-LSKA to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments;

[0013] S3, dividing the labeled transmission line data set into a training set and a validation set, and using the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog;

[0014] S4, using the trained YOLOv8n model for identifying dangerous objects on power transmission lines in rainy and foggy environments to perform power transmission line detection.

[0015] Furthermore, the S1 constructs a data set for monitoring dangerous objects on power transmission lines in rainy and foggy environments, and uses the MakeSense platform to deeply annotate the data set, specifically including:

[0016] S11, obtaining video images of the periphery of the power transmission line from multiple perspectives and multiple targets in various scenes by using a drone, wherein the multiple perspectives include close-range, long-range, and side perspectives, and the multiple targets include excavators, heavy trucks, bulldozers, tower cranes, bird nests, and cranes;

[0017] S12, using RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of transmission lines in rain and fog environment, and generate a data set of transmission lines in rain and fog environment;

[0018] S13, using the Make Sense platform to deeply annotate the power transmission line dataset in rainy and foggy environments, and generated a label file that follows the CoCo standard format.

[0019] Furthermore, the S12 uses RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of the transmission line in the rain and fog environment, and generates a data set of the transmission line in the rain and fog environment, which specifically includes:

[0020] S121, simulate the effect of transmission lines in foggy environment, and generate a data set of transmission lines in foggy environment: create a new image with three channels in RGB mode set to the maximum value as a white background, and the size is the same as the video image collected by the drone, to simulate the initial effect of fog; in the mixing process, adjust the alpha threshold to achieve different degrees of fog coverage, and then obtain a data set of transmission lines in foggy environment;

[0021] S122, based on the data set of power transmission lines in foggy environment, the technology of image superposition and random number generation of raindrop parameters is used to simulate the effect of power transmission lines in rainy and foggy environment: create a blank image with the same size as the original image, use random numbers to generate raindrop parameters on the blank image, the parameters include the starting coordinates, length, density and direction of the raindrops, and finally superimpose the image with generated raindrop effect with the image under foggy effect to generate the data set of power transmission lines in rainy and foggy environment.

[0022] Furthermore, the method also includes: when a potential danger is detected and identified, the system automatically transmits the detection result to the cloud server via a communication device and triggers an alarm.

[0023] The present invention also provides a transmission line dangerous object identification system in a complex rain and fog environment, which is used to execute the transmission line dangerous object identification method in a complex rain and fog environment, comprising:

[0024] Monitoring equipment, used to collect video images of the periphery of power transmission lines from multiple perspectives and multiple targets in various scenarios through drones;

[0025] An edge device, used for identifying dangerous objects around the power transmission line in a complex environment of rain and fog according to the video image; the edge device includes an image processing module and a YOLOv8n model module;

[0026] A communication device, used to receive the identification result from the edge device, and when the identification result indicates that there are dangerous objects around the power transmission line, transmit the identification result to the cloud server to trigger an alarm;

[0027] The image processing module is used to preprocess the video image, construct a data set of power transmission lines in a rainy and foggy environment according to the preprocessed video image, and deeply annotate the data set of power transmission lines in a rainy and foggy environment using the Make Sense platform to obtain an annotated data set of power transmission lines;

[0028] The YOLOv8n model module includes model building module, training module, and detection module;

[0029] The model building module is used to use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use the SPPF-LSKA layer to replace the SPPF layer in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism layer after the SPPF-LSKA layer to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments;

[0030] The training module is used to divide the labeled transmission line data set into a training set and a validation set, and use the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, so as to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog;

[0031] The detection module is used to perform transmission line detection using the trained YOLOv8n model for identifying transmission line hazards in rainy and foggy environments.

[0032] Furthermore, the lightweight shared convolution detection head is used to perform independent calculations on the three shared convolution detection heads, and a Batch Normalization layer is introduced into the lightweight shared convolution detection head.

[0033] Furthermore, the monitoring device is a drone equipped with an industrial-grade USB driver-free camera; the edge device is a Huawei Atlas 200I DK A2 edge computing development board; and the communication device is a wireless module.

[0034] Furthermore, the video image is preprocessed, specifically including: using OpenCV to adjust the size of the collected video image, and decoding the adjusted video image.

[0035] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention provides a method and system for identifying dangerous objects in power transmission lines in a complex rainy and foggy environment, which are used to identify potential dangerous objects around power transmission lines in a complex rainy and foggy environment, such as excavators, heavy trucks, bulldozers, tower cranes, bird nests and cranes, etc. Through the front-end edge device, the detection of these dangerous objects can be performed in real time and efficiently, and the identification results can be sent directly to the client through the wireless device. In this process, there is no need to transmit all the collected data to the back-end, only the detected dangerous object data needs to be transmitted to the back-end, which not only saves the cost of data transmission, but also omits the back-end data processing link, significantly shortens the detection time, and meets the real-time requirements.

[0036] In addition, the present invention integrates edge computing and YOLOv8n algorithms to promote the automation and low-carbonization process of transmission line operation and maintenance. In order to solve the problem that the existing technology cannot be deployed on edge devices with limited computing resources, the present invention balances the complexity and accuracy of the model to make it suitable for deployment on edge devices and achieve efficient, real-time and low-cost target detection effects. The edge device unit structure is cleverly designed so that the system includes a set of visual devices and a set of edge computing devices. Moreover, the design of the system focuses on rapid integration and deployment, has strong environmental adaptability, and has high-performance edge computing capabilities, which optimizes energy efficiency. The modular design of the system and its compatibility with new technologies provide convenience for future technology upgrades and functional expansions. On the whole, it provides users with an efficient, reliable, easy-to-integrate and deployable transmission line monitoring solution, which significantly improves the safety monitoring capability and response speed of the transmission line, and provides strong technical support for the safety management of smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A schematic diagram of the overall network structure of a YOLOv8n model (i.e., an improved YOLOv8n model) for identifying dangerous objects on power transmission lines in a rainy and foggy environment constructed according to an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of the structure of a shared parameter detection head LSCSBN according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the structure of the lightweight attention mechanism Triplet-Attention according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of SPPF-LSKA according to an embodiment of the present invention;

[0042] Figure 5 is a schematic diagram of the structure of LSKA;

[0043] Figure 6 This is a flow chart of a method for identifying dangerous objects on a power transmission line in a complex rainy and foggy environment according to an embodiment of the present invention;

[0044] Figure 7: are feature heat maps of the original YOLOv8n and the improved YOLOv8n of the present invention, wherein A and B are the original images collected, C and D are the feature heat maps obtained after the original images collected are processed by the original YOLOv8n model, and E and F are the feature heat maps obtained after the original images collected are processed by the improved YOLOv8n model of the present invention;

[0045] Figure 8 This is a comparison chart of the model complexity results of the improved YOLOv8n model and the original model in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The purpose of the present invention is to provide a method and system for identifying dangerous objects in power transmission lines in complex rain and fog environments. By constructing a YOLOv8n model for identifying dangerous objects in power transmission lines in rain and fog environments, real-time, accurate, and all-weather detection of dangerous objects near power transmission lines is achieved, significantly reducing the inspection cost.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] like Figure 6 As shown, a method for identifying dangerous objects on a power transmission line in a complex rain and fog environment provided by an embodiment of the present invention includes the following steps:

[0051] S1. Construct a data set of power transmission lines in a rainy and foggy environment, and use the Make Sense platform to deeply annotate the data set of power transmission lines in the rainy and foggy environment to obtain an annotated data set of power transmission lines;

[0052] S2. Use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use SPPF-LSKA to replace the SPPF in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism after SPPF-LSKA to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments;

[0053] S3, dividing the labeled transmission line data set into a training set and a validation set, and using the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog;

[0054] S4, using the trained YOLOv8n model for identifying dangerous objects on power transmission lines in rainy and foggy environments to perform power transmission line detection, and using a validation set to verify the detection performance and computational complexity of the model.

[0055] In this embodiment, S1 constructs a data set for monitoring dangerous objects on power transmission lines in a rainy and foggy environment, and uses the MakeSense platform to deeply annotate the data set, specifically including:

[0056] S11, obtaining video images of the periphery of the power transmission line from multiple perspectives and multiple targets in various scenes by using drones, thereby ensuring the diversity and richness of data, wherein the multiple perspectives include close-range, long-range and side perspectives, and the multiple targets include excavators, heavy trucks, bulldozers, tower cranes, bird nests and cranes;

[0057] S12, using RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of transmission lines in rain and fog environment, and generate a data set of transmission lines in rain and fog environment; this step uses synthetic data enhancement technology to expand and diversify the self-built monitoring data set to improve the generalization ability of the target detection model;

[0058] S13, using the Make Sense platform to deeply annotate the power transmission line dataset in rainy and foggy environments, and generated a label file that follows the CoCo standard format.

[0059] In this embodiment, the S12 uses RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of the transmission line in a rainy and foggy environment, and generates a data set of the transmission line in the rainy and foggy environment, which specifically includes:

[0060] S121, simulate the effect of transmission lines in foggy environment, and generate a data set of transmission lines in foggy environment: create a new image with all three channels in RGB mode set to the maximum value as a white background, and the size is the same as the video image collected by the drone to simulate the initial effect of fog; in the mixing process, adjust the alpha threshold to achieve different degrees of fog coverage, and then obtain a data set of transmission lines in foggy environment; the details are as follows:

[0061] S1211 uses drones to obtain aerial images of power transmission lines in various scenarios, and by adjusting the viewing angle and focal length of the camera to avoid the problem of highly similar image content caused by a single camera viewing angle, it captures drone aerial images of power transmission lines in diverse scenarios, with a total of 7,425 original images. These images cover multiple viewing angles such as close distance, long distance and side, ensuring the diversity and richness of the data.

[0062] S1212 The present invention collects UAV aerial images of power transmission lines and constructs a self-built dataset containing six different targets, such as excavators, heavy trucks, bulldozers, tower cranes, bird nests, and cranes. In particular, considering the large size and susceptibility to occlusion of tower cranes, we specifically increase the number of samples in this category to enhance the model's recognition ability for such targets.

[0063] In order to improve the generalization ability of the target detection model, S1213 uses synthetic data enhancement technology to expand and diversify the self-built monitoring dataset. The specific operation is to accurately segment the detected target objects and fuse them into the new background image.

[0064] S1214 Since it is difficult to collect data sets in real rainy and foggy environments, in order to obtain a data set of power transmission lines in rainy and foggy environments, an RGB image processing method is used to simulate the effect of power transmission lines in foggy environments. The method is characterized in that a new image is created with the three channels in the RGB (red, green, and blue) mode set to the maximum value (i.e., white) as a white background, and the size is the same as the drone aerial image to simulate the initial effect of fog. In the mixing process, the threshold of the alpha (transparency channel) is adjusted to achieve coverage of different degrees of fog effects, thereby obtaining a data set of power transmission lines in foggy environments. Based on the data set of power transmission lines in foggy environments, the technology of image superposition and random number generation of raindrop parameters is used to simulate the effect of power transmission lines in rainy and foggy environments. The method is characterized in that a blank image with the same size as the original image is created, and the parameters of raindrops are generated on the blank image using random numbers. The parameters include the starting coordinates, length, density, and direction of raindrops. Finally, the image with the generated raindrop effect is superimposed with the image under the foggy effect to generate a data set of power transmission lines in rainy and foggy environments.

[0065] S122, based on the data set of power transmission lines in foggy environment, the technology of image superposition and random number generation of raindrop parameters is used to simulate the effect of power transmission lines in rainy and foggy environment: a blank image with the same size as the original image is created, and the parameters of raindrops are generated on the blank image using random numbers, and the parameters include the starting coordinates, length, density and direction of raindrops. Finally, the image with generated raindrop effect is superimposed with the image under foggy effect to generate the data set of power transmission lines in rainy and foggy environment, specifically including:

[0066] S1221, in order to improve the accuracy of the target detection model and ensure that the model can capture the specific features and spatial locations of dangerous objects around the transmission lines, we have detailedly annotated the six targets in the image. This embodiment uses the Make Sense platform to deeply annotate the data set of transmission lines in rainy and foggy environments, and generates label files that follow the CoCo standard format. These files detail the names of six different targets and their specific location information. These images and their corresponding annotation files are used in the training and verification stages of the model, effectively improving the recognition effect of target detection.

[0067] In a further embodiment, in order to ensure the effective use of data, the data set of dangerous goods monitoring of power lines in rainy and foggy environments is shuffled and divided into training set, validation set and test set in a ratio of 8:1:1. The training set is used to improve the model's feature extraction and recognition capabilities for the target, and the validation set is used to verify the model's detection accuracy and recognition capabilities.

[0068] In this embodiment, the method also includes: when a potential danger is detected and identified, the system automatically transmits the detection result to the cloud server via a communication device and triggers an alarm.

[0069] Example 2

[0070] As shown in the figure, an embodiment of the present invention provides a transmission line dangerous object identification system in a complex rain and fog environment, which is used to execute the transmission line dangerous object identification method in a complex rain and fog environment described in Example 1, including:

[0071] Monitoring equipment, used to collect video images of the power transmission line periphery from multiple perspectives and multiple targets in various scenarios through drones;

[0072] An edge device, used for identifying dangerous objects around the power transmission line in a complex environment of rain and fog according to the video image; the edge device includes an image processing module and a YOLOv8n model module;

[0073] A communication device is used to receive the recognition result from the edge device, and when the recognition result is that there are dangerous objects around the power transmission line, transmit the recognition result to the cloud server to trigger an alarm, thereby realizing a real-time alarm and rapid response mechanism;

[0074] The image processing module is used to preprocess the video image, construct a data set of power transmission lines in a rainy and foggy environment according to the preprocessed video image, and deeply annotate the data set of power transmission lines in a rainy and foggy environment using the Make Sense platform to obtain an annotated data set of power transmission lines;

[0075] The YOLOv8n model module includes model building module, training module, and detection module;

[0076] The model building module is used to use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use the SPPF-LSKA layer to replace the SPPF layer in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism layer after the SPPF-LSKA layer to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments;

[0077] The training module is used to divide the labeled transmission line data set into a training set and a validation set, and use the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, so as to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog;

[0078] The detection module is used to perform transmission line detection using the trained YOLOv8n model for identifying transmission line hazards in rainy and foggy environments.

[0079] In this embodiment, the lightweight shared convolution detection head is used to perform independent calculations on the three lightweight shared convolution detection heads' shared convolution layers, and a Batch Normalization layer is introduced into the lightweight shared convolution detection head.

[0080] In this embodiment, the monitoring device is a drone equipped with an industrial-grade USB driver-free camera; the edge device is a Huawei Atlas 200I DK A2 edge computing development board; and the communication device is a wireless module. The present invention supports the flexible deployment and management of AI models on the Huawei Cloud ModelArts platform, ensuring high reliability of communication.

[0081] In this embodiment, the video image is preprocessed, specifically including: using OpenCV to adjust the size of the captured video image, and decoding the adjusted video image.

[0082] Since the original YOLOv8n model has a relatively complex structure, it is difficult to deploy in edge devices with limited computing resources and achieve accurate detection. Therefore, this embodiment introduces a lightweight shared convolutional detection head LSCSBN based on the YOLOv8n model, adds SPPF-LSKA to the backbone feature extraction network to replace the original SPPF, and finally introduces the Triplet-Attention lightweight attention mechanism after SPPF-LSKA. The improved YOLOv8n model was trained on a data set of dangerous objects monitoring of power transmission lines in rainy and foggy environments, achieving good results. The overall structure of the improved YOLOv8n model (i.e., the YOLOv8n model for identifying dangerous objects on transmission lines in rainy and foggy environments) is shown in Figure 1 The following is an introduction to the structure of each part of the YOLOv8n model for identifying dangerous objects on power transmission lines in rainy and foggy environments:

[0083] 1. Introduction to lightweight shared convolutional detection head LSCSBN:

[0084] Since the original YOLOv8 model usually uses independent detection heads for different feature layers, the utilization rate of model parameters is low and there is a lack of information interaction between detection heads, which leads to high computational complexity of the model and reduced detection effect. In order to give full play to the target detection performance of YOLOv8 at the edge of limited resources, the detection head in YOLOv8n is improved, and a lightweight shared convolution detection head LSCSBN is introduced. It realizes parameter sharing by sharing Conv, Conv_Reg and Conv_Cls and aggregating three different input scale feature maps. This design does not require the design of independent parameters for each feature layer, which reduces the model parameters and thus reduces the computational complexity of the model. In addition, by optimizing the total number of parameters, the model can achieve higher detection performance in a shorter time. In addition, the shared parameter detection head can also help the model learn a common feature representation, reduce the dependence on specific level detail information, and thus improve the model's generalization ability in new data or different scenarios.

[0085] At the same time, referring to the practice of NASFPN, the three detection heads share the convolution layer, and the introduced BN (BatchNormalization) layer is calculated independently, thus avoiding the error of the sliding average. The specific LSCSBN structure diagram is as follows: Figure 2 shown.

[0086] 2. Introduction to Triplet-Attention lightweight attention mechanism:

[0087] After the introduction of the lightweight LSCSBN detection head, the detection accuracy and positioning accuracy of the improved model will be reduced. To address this issue, in the context of complex rain and fog interference such as power transmission line dangerous object detection, the introduction of an attention mechanism can enable the model to focus on the important information of the input data and ignore irrelevant information, thereby effectively improving the detection performance of the model. Among them, unlike other attention mechanisms, Triplet-Attention is a lightweight and efficient attention mechanism that builds channel modules and spatial modules based on very few parameters. It uses three parallel branch structures to capture interactions in different dimensions and calculate attention weights, and weights the weights of its three branches to weighted average, so that the model can share and constrain parameters, thereby improving the generalization and expression capabilities of the model. While the Triplet-Attention lightweight attention mechanism enhances the image feature extraction capability in the backbone network (Backbone), the model's parameter volume and computational complexity have hardly increased, and the model's complexity has not increased. The specific model structure diagram is as follows Figure 3 shown.

[0088] Triplet-Attention consists of three parallel branches. All three branches establish an interactive relationship between the width and height of the input tensor X. At the same time, the last two branches perform counterclockwise rotation operations on the W axis and the H axis to obtain X^. Then, the three branches pass through the Z-Pool layer to reduce the 0th dimension of the input tensor X to 2 dimensions, that is, to obtain the maximum merged eigenvalue and average merged mean value of the input tensor X, so that this layer can retain the rich representation of the actual input tensor X while reducing the amount of calculation by reducing the depth. The calculation formula of Z-Pool is as follows:

[0089] X ∧* =[MaxPool 0d (X ∧ ),AvgPool 0d (X ∧ )]

[0090] Among them, MaxPool 0d Represents maximum pooling, AvgPool 0d represents average pooling, X ∧ Represents the transformed input tensor X.

[0091] Next, the attention weights of each branch are generated by the standard convolution layer with a kernel size of k, the batch normalization layer, and the Sigmoid activation function. Then the two branches rotate the W axis and the H axis clockwise to convert them into the initial scale features, so that the size features of the input tensor and the output tensor are consistent. Finally, the output tensors of the three branches are added and the average is calculated. The specific formula is as follows:

[0092]

[0093] Among them, σ represents the Sigmoid activation function, ψ represents the standard convolution layer with kernel size k in the three parallel branches in Triplet-Attention, Represents the output value of the three branches through Z-Pool, X ∧ 1. X ∧ 2 represents the input tensors of the last two branches after rotation, and x3 represents the input tensor of the first branch.

[0094] 3. Introduction of SPPF-LSKA module:

[0095] Under the premise of minimizing the computational complexity of the model, the detection performance of the model is continuously improved. The present invention introduces the separable large convolution kernel attention LSKA into the SPPF of the original YOLOv8n model to form a new SPPF-LSKA layer to replace the SPPF layer. The model structure diagram is shown in the figure below: Figure 4 shown.

[0096] LSKA is a separable large convolution kernel attention module, which consists of three parts: initialization convolution layer, spatial expansion convolution layer and fusion application attention. Its structure is shown in the figure Figure 5 As shown in the figure, the first two one-dimensional initialization convolutional layers are used to extract the horizontal and vertical features of the input feature map, respectively, so that the model can focus more on the dangerous objects around the transmission line and generate a preliminary attention map. The specific calculation formula is:

[0097]

[0098] Where C is the number of input channels, F is the input feature map, H and W represent the height and width of the input tensor respectively, * represents convolution, d is the dilation rate, F is the input feature map, W and H represent the width and height of the feature map respectively, is the output obtained by deep convolution of the convolution kernel of size (2d-1)(2d-1) with the input features. The local spatial information is captured and the Z C The grid effect of mid-depth convolution is compensated.

[0099] Then, spatial dilation convolution layers with different dilation rates are used to further extract features, such as one-dimensional DW-D-Conv. This convolution layer covers a larger receptive field and captures a wider range of contextual information without increasing the amount of computation. The specific calculation formula of the DW-D-Conv convolution layer is:

[0100]

[0101] Among them, k represents the convolution kernel size, d represents the expansion rate, is the output value of the one-dimensional initialization convolution layer, Z C Represents the global spatial information of the spatially dilated convolution output.

[0102] After a series of convolution operations, LSKA fuses the obtained features through the last convolution layer (conv) to generate the final attention map. The specific operation steps are as follows:

[0103] A C =W 1×1 *Z C

[0104]

[0105] Among them, W 1×1 represents a 1x1 convolution kernel, represents the Hadamard product, Representative attention map A C And the input feature map F C In addition, each channel C in F is convolved with the corresponding channel in kernel W.

[0106] The final attention map is element-wise multiplied with the original input feature map, that is, the attention mechanism is applied. In this way, each element in the original feature map is weighted according to the value of the attention map, highlighting important features and ignoring unimportant features. In summary, LSKA decomposes the two-dimensional convolution kernel of the deep convolution layer into cascaded horizontal and vertical one-dimensional convolution kernels, reducing the computational complexity and adapting to deployment in edge devices with limited computing resources.

[0107] The improved YOLOv8n detection model was trained with the constructed transmission line hazardous object monitoring training set in rainy and foggy environments. The number of training times was 150 and the input image size was 640*640, and a deep learning model for identifying hazardous objects in transmission lines in rainy and foggy environments was obtained. In addition, the detection performance and computational complexity of the trained model were trained with the validation set. The detection results are shown in Table 1 below:

[0108] Table 1: Comparison of test results

[0109]

[0110] Among them, precision P, recall rate R and average precision Map are used to evaluate the detection accuracy of the model; floating point operations GFLOPS and model parameters parameters evaluate the computational complexity of the model. As can be seen from Table 1, the P, recall rate R and average precision Map of the improved model of the present invention are respectively improved by 1.3%, 1.2% and 1.3% compared with the original YOLOv8n model, and a better dangerous object recognition and detection effect is achieved in a complex environment of rain and fog; FLOPS is reduced by 16.1%, and parameters are reduced by 12.4%, which is suitable for deployment in edge devices with limited computing resources. Compared with the original YOLOv8n model, the improved YOLOv8n model achieves a balance between model complexity and detection accuracy.

[0111] The improved YOLOv8n model contains fewer parameters and computational complexity, and is suitable for deployment on the edge with limited resources. The comparison results of the parameters and computational complexity of the improved YOLOv8n model and the original YOLOv8n model are shown in Figure 8 In order to visualize the improved model's ability to extract features of small and occluded targets, two different images from the self-built dataset were selected for verification in the form of feature heat maps. The specific detection effects of the original YOLOv8n and the improved YOLOv8n are shown in Figure 7 .according to Figure 7 It can be seen from the feature heat map that the original YOLOv8n model may miss targets. With the improvement of the algorithm, the improved model improves the detection accuracy of targets, and missed targets are detected. At the same time, the improved YOLOv8n algorithm can accurately display the detected targets when generating feature heat maps, and the shapes are more regular, which better reflects the shape characteristics of the detected targets themselves.

[0112] In a further embodiment, the monitoring device is an industrial-grade USB driver-free camera. The industrial-grade USB driver-free camera can provide a high frame rate video stream to continuously track and analyze moving objects, and is easy to integrate and plug-and-play through a USB interface. The camera supports high-resolution output to ensure clear image details, while having low latency and wide dynamic range (WDR) to adapt to different lighting conditions. It is equipped with multiple types of image sensors such as CMOS and CCD to meet different needs. In addition, the camera has waterproof and dustproof features, suitable for outdoor or industrial environments, and due to the universal USB interface and standardized SDK, they can be easily integrated into existing systems.

[0113] The main functions of the monitoring equipment are:

[0114] The surrounding information is captured and collected through a USB driver-free camera, and the collected video images are transmitted to the edge computing node through the UVC protocol.

[0115] In this embodiment, the edge device is the Huawei Atlas 200I DK A2 edge computing development board. The Huawei Atlas200I DK A2 edge computing development board is a high-performance device designed for AI reasoning and edge computing. It is equipped with the Huawei Ascend 310 AI processor, providing up to 16TOPS of INT8 computing power, which is particularly suitable for performing image AI tasks. The development board is equipped with ample RAM and eMMC storage, and supports multiple interfaces such as Ethernet, USB, and HDMI, which is convenient for connecting sensors and other devices. In addition, it also supports the Huawei Cloud ModelArts platform, making the training, deployment, and management of AI models more convenient. The Linux operating system supports multiple programming languages, rich interfaces and hardware security features, and optimized energy efficiency, making the Atlas 200 DK A2 very suitable for long-term operation in edge environments.

[0116] The main functions of edge devices are:

[0117] OpenCV is used to resize the captured video images to 640x640 pixels to meet the input requirements of the deep learning model. The captured image data is then decoded and transmitted to the improved YOLOv8n deep learning model for dangerous object target identification. If a dangerous object is identified, the edge computing node will transmit the target detection result to the cloud server and trigger an alarm, which will be displayed on the visual interface to achieve real-time monitoring of dangerous objects on the transmission line.

[0118] In this embodiment, the communication device is a wireless module. Wireless modules are high-speed wireless communication devices that support 4G standards such as LTE, provide extensive network coverage and high-speed data transmission capabilities, and are suitable for a variety of application scenarios. They have the characteristics of multi-band support, low latency, easy integration, power consumption management, high security, etc., and as the technology matures, the cost-effectiveness gradually increases, and they are important components for realizing device interconnection and intelligent control.

[0119] The main functions of communication equipment are:

[0120] Provide 4G network for edge devices, which connect to the network through wireless modules and send data to the cloud.

[0121] The edge computing-based system design uses an industrial-grade USB driver-free camera to collect information about the power transmission line in real time, and transmits high-definition video streams to edge computing nodes through the UVC protocol. On the edge node, the video image is resized using OpenCV technology and optimized to 640x640 pixels to meet the input specifications of the deep learning model. Subsequently, the image data is decoded and fed into the improved YOLOv8n deep learning model deployed on the Huawei Atlas 200I DK A2 edge computing development board for accurate identification of dangerous objects. Once a potential threat is detected, the system automatically transmits the detection results to the cloud server through a secure 4G LTE wireless module and triggers an alarm to achieve real-time monitoring and rapid response of the power transmission line. In short, by using an efficient device combination of a high-frame-rate industrial-grade USB driver-free camera (monitoring device), Huawei Atlas 200I DK A2 edge computing development board (edge ​​device), and a high-speed 4G LTE wireless communication module (communication device), real-time and continuous monitoring of the environment around the power transmission line is achieved.

[0122] In addition, the system also includes a user-friendly visual interface for providing real-time video streaming, alarm information and historical data analysis, and has self-monitoring and remote maintenance functions to ensure the stable operation and timely update of the system, further improving the convenience of operation and monitoring efficiency. The entire system design takes into account environmental adaptability, redundancy mechanism, regulatory compliance and data protection, and has good scalability, which facilitates future technology upgrades and functional expansions.

[0123] In summary, the present invention provides a method and system for identifying dangerous objects on power transmission lines in a complex environment of rain and fog.

[0124] The remaining technical features in this embodiment can be flexibly selected by those skilled in the art according to actual conditions to meet different specific practical needs. However, it is obvious to those skilled in the art that it is not necessary to adopt these specific details to implement the present invention. In other examples, in order to avoid confusing the present invention, the well-known components, structures or parts are not specifically described, which are all within the technical protection scope defined by the technical solution claimed for protection in the claims of the present invention.

[0125] Modifications and changes made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the scope of protection of the claims attached to the present invention. In the above description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details are not necessary to practice the present invention. In other examples, in order to avoid confusing the present invention, well-known technologies, such as specific construction details, operating conditions and other technical conditions, are not specifically described.

[0126] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying dangerous objects on power transmission lines in a complex environment of rain and fog, characterized in that: The following steps are involved: S1. Construct a data set of power transmission lines in a rainy and foggy environment, and use the Make Sense platform to deeply annotate the data set of power transmission lines in the rainy and foggy environment to obtain an annotated data set of power transmission lines; S2. Use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use SPPF-LSKA to replace the SPPF in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism after SPPF-LSKA to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments; S3, dividing the labeled transmission line data set into a training set and a validation set, and using the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog; S4, using the trained YOLOv8n model for identifying dangerous objects in power transmission lines under rainy and foggy environments to detect dangerous objects around the power transmission lines.

2. The method for identifying dangerous objects on power transmission lines in a complex rain and fog environment according to claim 1 is characterized in that: S1, constructing a data set for monitoring dangerous objects on power transmission lines in a rainy and foggy environment, and deeply annotating the data set using the Make Sense platform, specifically includes: S11, obtaining video images of the periphery of the power transmission line from multiple perspectives and multiple targets in various scenes by using a drone, wherein the multiple perspectives include close-range, long-range, and side perspectives, and the multiple targets include excavators, heavy trucks, bulldozers, tower cranes, bird nests, and cranes; S12, using RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of transmission lines in rain and fog environment, and generate a data set of transmission lines in rain and fog environment; S13, using the Make Sense platform to deeply annotate the power transmission line dataset in rainy and foggy environments, and generated a label file that follows the CoCo standard format.

3. The method for identifying dangerous objects on power transmission lines in a complex rain and fog environment according to claim 1 is characterized in that: The S12 uses RGB image processing technology, random number generation of raindrop parameters and image overlay technology to simulate the effect of the transmission line in the rain and fog environment, and generates a data set of the transmission line in the rain and fog environment, specifically including: S121, simulate the effect of transmission lines in foggy environment, and generate a data set of transmission lines in foggy environment: create a new image with three channels in RGB mode set to the maximum value as a white background, and make its size the same as the video image collected by the drone to simulate the initial effect of fog; in the mixing process, adjust the threshold of alpha to achieve coverage of different degrees of fog effect, and then obtain a data set of transmission lines in foggy environment; S122, based on the data set of power transmission lines in foggy environment, the technology of image superposition and random number generation of raindrop parameters is used to simulate the effect of power transmission lines in rainy and foggy environment: create a blank image with the same size as the original image, use random numbers to generate raindrop parameters on the blank image, the parameters include the starting coordinates, length, density and direction of the raindrops, and finally superimpose the image with generated raindrop effect with the image under foggy effect to generate the data set of power transmission lines in rainy and foggy environment.

4. The method for identifying dangerous objects on power transmission lines in a complex rain and fog environment according to claim 1 is characterized in that: The method also includes: when a potential danger is detected and identified, the system automatically transmits the detection result to a cloud server via a communication device and triggers an alarm.

5. A transmission line dangerous object identification system in a complex rain and fog environment, used to execute the transmission line dangerous object identification method in a complex rain and fog environment as claimed in any one of claims 1 to 4, characterized in that: include: Monitoring equipment, used to collect video images of the periphery of power transmission lines from multiple perspectives and multiple targets in various scenarios through drones; An edge device, used for identifying dangerous objects around the power transmission line in a complex environment of rain and fog according to the video image; the edge device includes an image processing module and a YOLOv8n model module; A communication device, used to receive the identification result from the edge device, and when the identification result indicates that there are dangerous objects around the power transmission line, transmit the identification result to the cloud server to trigger an alarm; The image processing module is used to preprocess the video image, construct a data set of power transmission lines in a rainy and foggy environment according to the preprocessed video image, and deeply annotate the data set of power transmission lines in a rainy and foggy environment using the Make Sense platform to obtain an annotated data set of power transmission lines; The YOLOv8n model module includes model building module, training module, and detection module; The model building module is used to use the lightweight shared convolution detection head as the detection head of the YOLOv8n model, use the SPPF-LSKA layer to replace the SPPF layer in the backbone feature extraction network of the YOLOv8n model, and introduce the Triplet-Attention lightweight attention mechanism layer after the SPPF-LSKA layer to obtain the YOLOv8n model for transmission line hazardous object recognition in rainy and foggy environments; The training module is used to divide the labeled transmission line data set into a training set and a validation set, and use the training set to train the YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog, so as to obtain a trained YOLOv8n model for identifying dangerous objects in transmission lines under rain and fog; The detection module is used to detect dangerous objects around the transmission lines using the trained YOLOv8n model for identifying dangerous objects in transmission lines under rainy and foggy environments.

6. The power transmission line dangerous object identification system in a complex rain and fog environment according to claim 5 is characterized in that: The lightweight shared convolution detection head is used to perform independent calculations on the three shared convolution layers of the lightweight shared convolution detection heads, and a BatchNormalization layer is introduced into the lightweight shared convolution detection head.

7. The power transmission line dangerous object identification system in a complex rain and fog environment according to claim 5 is characterized in that: The monitoring device is a drone equipped with an industrial-grade USB driver-free camera; the edge device is Huawei Atlas200IDKA2 edge computing development board; and the communication device is a wireless module.

8. The power transmission line dangerous object identification system in a complex rain and fog environment according to claim 5 is characterized in that: The video image is preprocessed, specifically including: using OpenCV to adjust the size of the collected video image, and decoding the adjusted video image.

Images