Power transmission line tiny foreign matter detection method based on improved YOLACT

Abstract

The invention discloses a power transmission line tiny foreign matter detection method based on improved YOLACT, and belongs to the technical field of image recognition of power transmission lines. The method comprises the following steps: firstly, acquiring a power transmission line tiny foreign matter image, performing data enhancement by adopting modes of random translation, random rotation and the like, labeling data by adopting label, establishing a power transmission line tiny foreign matter semantic segmentation data set, and dividing a training set and a verification set according to a proportion; according to the method, improvement is carried out based on a YOLACT algorithm model, three 3 * 3 convolution kernels are adopted in a feature extraction part to replace a 7 * 7 convolution kernel in an original network of a YOLACT instance segmentation model, and cavity convolution is adopted in a Protonet part to replace conventional convolution, so that the improved YOLACT instance segmentation model is constructed; and training and verifying by using the training set and the verification set to obtain a network model for detection, and detecting the tiny foreign matters existing in the power transmission line by using the network model.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition of power transmission lines, and in particular relates to a method for detecting tiny foreign objects in power transmission lines based on neural network recognition. Background Art

[0002] As a key link in the power system and an important medium for power transmission, the normal operation of transmission lines is inseparable from the reliability of the power grid and the sustainable development of the national economy. However, transmission lines are exposed to the wild environment all year round, and foreign objects are very likely to invade, which in turn leads to power accidents. The number of overhead transmission lines built in suburban towns is gradually increasing, and the external environment faced by line operation is becoming increasingly complex, and the threats faced are also increasing. In recent years, environmental protection pressure has increased. The slag piles on construction sites are usually covered with dust-proof nets, which are often not pressed and are easily blown away by the wind; with the further development of agriculture, the construction of vegetable greenhouses has also increased year by year. The greenhouse film is easily damaged in windy weather and blown onto the lines in other places; in addition, kite flying in spring and autumn is very popular among the people, and kite flying under high-voltage lines and kite line breakage often occur. Especially in recent years, strong wind and strong convective weather have increased, and it is common for films, dust nets and other strip-shaped or entangled objects to be blown up by the wind and hung on overhead transmission lines, posing a serious threat to the safe and reliable operation of overhead transmission lines. Tripping of overhead transmission lines caused by foreign objects such as kites, plastic sheets, tin foil strips, and dust nets are common forms of external force damage. The main characteristics of tripping accidents are as follows: first, they are highly time-sensitive. In the spring and autumn kite-flying seasons, it is easy for kite lines to break, causing kites or kite lines to hang on overhead transmission lines; after strong convective and windy weather in summer, plastic sheets, tin foil strips, and dust nets are also easily blown onto overhead transmission lines; second, they are highly regional. Usually, the lines around greenhouses, construction sites, and fruit trees are susceptible to plastic sheets and dust nets, respectively. The third is the difficulty in prevention. In recent years, overhead transmission lines have been distributed in many points and over a wide area. It is also difficult to track and persuade people flying kites offline in time. Severe convective weather is sudden and difficult to accurately predict. Foreign objects are scattered over a large area under strong winds, and it is not easy to grasp the situation of foreign objects on the line. These have brought challenges to the prevention of foreign objects in overhead transmission lines. Nowadays, the materials of kite lines are diverse, with cotton ropes, nylon ropes, fishing lines and steel wires as the majority. Among them, kite lines containing metal wires pose the greatest threat to the safe and reliable operation of overhead transmission lines. Once a line break occurs, it will be blown off and entangled on the conductor and ground wire under the action of wind, which will cause single-phase grounding or phase-to-phase short circuit of the overhead transmission line, directly causing line tripping accidents and even casualties. Although kite strings made of other insulating materials are not conductive and will not immediately cause the line to trip when they are scraped and entangled on overhead transmission lines, when encountering humid weather such as rain, snow or heavy fog, the kite strings will absorb water and become good conductors, eventually causing the overhead transmission lines to trip, posing a serious threat to the safe and reliable operation of the lines.

[0003] Similarly, for insulating materials such as dust screens and plastic sheets, if they encounter humid weather such as rain and fog, their insulation performance may deteriorate, causing short circuits, tripping and power outages. The tin foil strips commonly used to prevent birds in orchards are directly conductive, and once the length reaches a certain value, it will directly cause discharge and tripping, which poses a great threat to the safe and reliable operation of overhead power transmission lines.

[0004] When the inspectors find foreign objects on the conductors and ground wires of overhead transmission lines, if the foreign objects have affected the safe operation of the lines, they will apply for power outage to the dispatching department of the unit. After approval, they will arrange for staff to use slides or wiring to reach the foreign objects and use relevant tools to remove the foreign objects. Since power outages to handle foreign objects reduce the reliability of power supply of the power grid, increase the risk of regional power outages, and may cause large economic losses, they are generally applicable to situations where the line trips or the safety distance of live processing is insufficient. In order to ensure the safe and reliable operation of the power system, it is of great significance to detect tiny foreign objects on the transmission line. With the vigorous development of computer technology, the application of machine vision in the field of power inspection can significantly improve work efficiency, reduce resource waste, and promote the intelligent development of power grids to a new level. However, the existing technology usually takes photos of transmission lines to realize foreign object detection, but in this process, there is a lack of unified and standardized identification methods for smaller tiny foreign objects, resulting in unstable recognition accuracy of tiny foreign objects, no unified processing method, and the accuracy rate continues to be low and unstable, so that the target detection algorithm can only output a rough location, and still cannot effectively and accurately locate the tiny foreign objects on the transmission line and realize the function of classification detection. Summary of the invention

[0005] The present invention aims at the problem that the detection accuracy of tiny foreign objects in power transmission lines is low, and further proposes a method for detecting tiny foreign objects in power transmission lines based on improved YOLACT.

[0006] A method for detecting tiny foreign objects in a power transmission line based on improved YOLACT, the method comprising the following steps:

[0007] S1: Construct a foreign body image dataset: collect data on foreign bodies in power transmission lines by taking photos on site with drones and summarizing foreign bodies in power transmission lines, perform data enhancement operations on foreign body images in power transmission lines, expand the dataset, and generate a foreign body image dataset suitable for deep learning;

[0008] S2: Mark the edge contours and categories of foreign objects that intrude into the transmission line, and determine the training set and verification set based on the foreign object image dataset;

[0009] S3: Build an improved YOLACT instance segmentation model: Use the ResNet residual network as the feature extraction network, use three 3×3 convolution kernels in the feature extraction part to replace the 7×7 convolution kernel in the original network of the YOLACT instance segmentation model, and use the hole convolution to replace the conventional convolution in the Protonet part, so as to build an improved YOLACT instance segmentation model as a foreign body detection network model;

[0010] S4: Use the training set to train the foreign object detection network model, and then use the validation set to evaluate the performance of the foreign object detection network model to obtain a foreign object detection network model that can be used for foreign objects;

[0011] S5: Input the transmission line image to be detected into the foreign object detection network model, obtain the coordinate position, category and confidence level of the detection frame, and generate the corresponding detection frame.

[0012] Furthermore, in the process of collecting data on foreign objects in power transmission lines by means of on-site photography by drones and summary of foreign objects in power transmission lines, it is necessary to cover data on foreign objects in power transmission lines in a variety of scenarios.

[0013] Furthermore, the data enhancement operation for the transmission line foreign body image includes random rotation and image scaling.

[0014] Furthermore, the process of marking the edge contours and categories of foreign objects invading the transmission line is implemented using Labelme labeling software.

[0015] Furthermore, the YOLACT instance segmentation model consists of four parts: feature extraction network backbone, feature pyramid network FPN, mask generation branch Protonet and prediction head Prediction Head.

[0016] Backbone includes 5 convolutional layers, corresponding to C1 to C5, which are used to extract multi-scale feature maps; the feature pyramid of FPN is recorded as P3 to P7 layers from large to small according to the feature map, and the features of C3 to C5 correspond to P3 to P5 layers respectively; after the FPN processing is completed, the P3 layer is input to the Protonet part, and P3-P7 are all sent to the Prediction Head part; Protonet generates k 138×138 prototype masks, and the prediction head generates the confidence of k mask prototype images and removes repeated detection boxes through non-maximum suppression operations. After linearly combining the two and setting the threshold, the instance segmentation result can be generated, showing the category, confidence and bounding box of the foreign body on the transmission line.

[0017] Furthermore, in step S2, a test set is determined based on the foreign body image data set; and in step S4, the test set is also used to test the detection effect of the foreign body detection network model.

[0018] The present invention has the following beneficial effects:

[0019] Compared with the prior art, the method of the present invention is a special identification and processing method specifically for tiny foreign objects with smaller volume. Combined with the shape and suspension characteristics of tiny foreign objects, it fully considers the impact of tiny foreign objects on the safe and stable operation of transmission lines, and compares and improves the YOLACT instance segmentation algorithm based on deep learning to improve the accuracy of tiny foreign object detection. It can quickly locate and accurately classify and identify tiny foreign objects that invade transmission lines, and meet the needs of power inspection in terms of segmentation accuracy and reasoning speed. It realizes a standardized, unified and stable processing method for line operation and maintenance, ensuring that overhead transmission lines reduce the interference of tiny foreign objects and ensure safe operation under continuous self-protection state. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the improved YOLACT network used in the present invention;

[0021] Figure 2 This is a flow chart of training and testing the instance segmentation model for tiny foreign body detection of the present invention.

[0022] Figure 3 This is a schematic diagram of using the trained YOLACT model to analyze and test tiny foreign objects in power transmission lines in the present invention. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The present invention proposes a method for detecting tiny foreign objects in a power transmission line based on an improved YOLACT, the method comprising the following steps:

[0025] S1: Construction of a dataset of tiny foreign objects images: Through on-site drone photography and data aggregation of tiny foreign objects on power transmission lines, we collected data on tiny foreign objects on power transmission lines, covering a variety of scenarios. We used random rotation, scaling and other methods to perform data enhancement operations on images of foreign objects on power transmission lines, expand the dataset, generate additional samples suitable for deep learning, and enhance the generalization ability of the model.

[0026] S2: Labelme annotation software is used to annotate the edge contours and categories of tiny foreign objects that invade the transmission line, and the sets are divided into training set, validation set and test set according to a certain ratio;

[0027] S3: Build and improve the YOLACT instance segmentation model: Use the ResNet residual network as the backbone network, use three 3×3 convolution kernels to replace the 7×7 convolution kernel in the original network in the feature extraction part, and use the hole convolution to replace the regular convolution in the Protonet part to build a more optimized YOLACT instance segmentation model;

[0028] S4: Use the training set to train the improved YOLACT instance segmentation model, then use the validation set to evaluate the model performance, and use the test set to test the learning effect of the instance segmentation model; input the image to be detected into the trained power line tiny foreign body instance segmentation model, obtain the coordinate position, category and confidence level of the detection box, and generate the corresponding detection box.

[0029] In fact, the YOLACT network is mainly composed of four parts: feature extraction network (backbone), feature pyramid network (FPN), mask generation branch (Protonet), and prediction head (Prediction Head). ResNet-101 is used as the feature extraction network, so that the network maintains good accuracy while deepening. The input image of tiny foreign bodies on the transmission line is output through the five convolutional layers of the backbone, corresponding to C1 to C5, which are used to extract multi-scale feature maps. The obtained multi-scale features are passed to the P3 to P7 layers of the FPN network. After the FPN processing is completed, the P3 layer is input to the Protonet part, and P3-P7 are all sent to the Prediction Head part. Protonet generates k 138×138 prototype masks, and the prediction head generates the confidence of k mask prototype images and removes repeated detection boxes through non-maximum suppression operations. After linearly combining the two and setting the threshold, the instance segmentation result can be generated, showing the category, confidence and bounding box of tiny foreign bodies on the transmission line.

[0030] Although the C5 layer in the original YOLACT network uses a larger 7×7 convolution kernel to better capture the original texture features of the image, it also results in a large amount of computation. The receptive field brought by three 3×3 convolution kernels is the same as that of the 7×7 convolution kernel, but it can greatly reduce the number of parameters and improve network performance.

[0031] The improvement of using dilated convolution to replace conventional convolution in the Protonet part can increase the receptive field without increasing the calculation of the convolution process. In the task of detecting tiny foreign objects in power transmission lines, the proportion of foreign objects in the entire image is extremely small. In order to ensure that the spatial information of the feature layer is not excessively lost, the use of dilated convolution can prevent the loss of information of small foreign objects due to downsampling and the singleness of convolution, bringing better feature fusion effect to the algorithm.

[0032] In the present invention, by improving the instance segmentation model, the model can learn richer feature information to achieve improved detection accuracy and speed. The method proposed in the present invention has good performance and promotion and application prospects for the identification and analysis of tiny foreign objects in transmission lines.

[0033] Example

[0034] The analysis method used in the embodiment of the present invention specifically includes the following steps:

[0035] S1. Construction of a dataset of tiny foreign objects images: Through on-site drone photography and data aggregation of tiny foreign objects on power transmission lines, we collected data on tiny foreign objects on power transmission lines, covering a variety of scenarios. We used random rotation, scaling and other methods to perform data enhancement operations on the images of foreign objects on power transmission lines, expand the dataset, generate additional samples suitable for deep learning, and enhance the generalization ability of the model.

[0036] In this embodiment, no less than 2,000 images of tiny foreign objects on the power transmission lines are collected and expanded, including data of tiny foreign objects on the power transmission lines in multiple scenarios. The sufficient data provides data support for the training of subsequent instance segmentation models.

[0037] S2: Labelme annotation software is used to annotate the edge contours and categories of tiny foreign objects that invade the transmission line, and the sets are divided into training set, validation set and test set according to a certain ratio;

[0038] This embodiment uses software to mark the location and category of foreign matter contained in each foreign matter image, and automatically generates labels in the form of json files for use in the next step of model training. 10% of the samples are randomly selected as the verification set, and the remaining samples are used as the training set.

[0039] S3: Build and improve the YOLACT instance segmentation model, use the ResNet residual network as the backbone network, use three 3×3 convolution kernels to replace the 7×7 convolution kernel in the original network in the feature extraction part, and use the hole convolution to replace the regular convolution in the Protonet part to build a more optimized YOLACT instance segmentation model. The model structure is as follows: Figure 1 shown.

[0040] The YOLACT network is mainly composed of four parts: feature extraction network (backbone), feature pyramid network (FPN), mask generation branch (Protonet), and prediction head (Prediction Head). ResNet-101 is used as the feature extraction network, so that the network maintains good accuracy while deepening.

[0041] The input image of tiny foreign bodies on the transmission line is passed through 5 convolutional layers to output corresponding deep features, corresponding to C1 to C5, which are used to extract multi-scale feature maps. The output deep feature maps are 112×112, 56×56, 28×28, 14×14, and 7×7, respectively. The obtained multi-scale features are passed to the P3 to P7 layers of the FPN network. After the FPN processing is completed, the P3 layer is input to the Protonet part, and P3-P7 are all sent to the Prediction Head part. Protonet generates k 138×138 prototype masks, and the prediction head generates the confidence of k mask prototype images and removes duplicate detection boxes through non-maximum suppression operations. After linearly combining the two and setting the threshold, the instance segmentation result can be generated, showing the category, confidence and bounding box of tiny foreign bodies on the transmission line.

[0042] In this instance segmentation model, the high resolution of low-level features and the semantic information of high-level features are simultaneously utilized to identify foreign objects at three feature layers of different scales, and the aspect ratios of the anchor boxes are set to 2, 1, and 1 / 2, respectively.

[0043] In step S3, the original network C5 layer uses a larger 7×7 convolution kernel to better capture the original texture features of the image, but it also results in a large amount of calculation. The receptive field brought by three 3×3 convolution kernels is consistent with that of the 7×7 convolution kernel, but it can greatly reduce the number of parameters and improve network performance. The multi-layer superposition of small convolution kernels increases the nonlinearity of the network transmission process, further deepens the depth of the network, and makes the network have richer semantic feature resolution capabilities.

[0044] Atrous convolution can increase the receptive field without increasing the computation of the convolution process. In the task of detecting tiny foreign objects in power transmission lines, the proportion of foreign objects in the entire image is extremely small. In order to ensure that the spatial information of the feature layer is not excessively lost, the use of atrous convolution can prevent the loss of information about small foreign objects due to downsampling and convolution, thus bringing better feature fusion effects to the algorithm.

[0045] S4: Use the training set to train the improved YOLACT instance segmentation model, then use the validation set to evaluate the model performance, and use the test set to test the learning effect of the instance segmentation model; input the image to be detected into the trained transmission line micro-foreign body instance segmentation model to obtain the coordinate position, category and confidence level of the detection frame, and generate the corresponding detection frame. The flowchart is as follows: Figure 2 shown.

[0046] In this embodiment, epochs is set to 100 and batchsize is set to 16 during model training. The model uses Mosaic data enhancement at the input end. A selected image and three random images are randomly cropped and then spliced ​​into one image as training data. Many small target samples are added, which greatly enhances the diversity of the data and the robustness of the model.

[0047] The trained instance segmentation model was applied to the actual power transmission line environment for testing, showing the specific location of tiny foreign objects and performing real-time analysis. Figure 3 shown.

[0048] The above only expresses the preferred implementation mode of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting tiny foreign objects in power transmission lines based on improved YOLACT, characterized in that: The method comprises the following steps: S1: Construct a foreign body image dataset: collect data on foreign bodies in power transmission lines by taking photos on site with drones and summarizing foreign bodies in power transmission lines, perform data enhancement operations on foreign body images in power transmission lines, expand the dataset, and generate a foreign body image dataset suitable for deep learning; S2: Mark the edge contours and categories of foreign objects that intrude into the transmission line, and determine the training set and verification set based on the foreign object image dataset; S3: Build an improved YOLACT instance segmentation model: Use the ResNet residual network as the feature extraction network, use three 3×3 convolution kernels in the feature extraction part to replace the 7×7 convolution kernel in the original network of the YOLACT instance segmentation model, and use the hole convolution to replace the conventional convolution in the Protonet part, so as to build an improved YOLACT instance segmentation model as a foreign body detection network model; S4: Use the training set to train the foreign object detection network model, and then use the validation set to evaluate the performance of the foreign object detection network model to obtain a foreign object detection network model that can be used for foreign objects; S5: Input the transmission line image to be detected into the foreign object detection network model, obtain the coordinate position, category and confidence level of the detection frame, and generate the corresponding detection frame.

2. According to claim 1, a method for detecting tiny foreign bodies in a power transmission line based on improved YOLACT, characterized in that: In the process of collecting data on foreign objects in transmission lines by means of on-site photography by drones and summary of foreign objects in transmission lines, it is necessary to cover data on foreign objects in transmission lines in various scenarios.

3. According to claim 2, a method for detecting tiny foreign bodies in a power transmission line based on improved YOLACT, characterized in that: The data enhancement operations for the transmission line foreign body images include random rotation and image scaling.

4. According to claim 1, a method for detecting tiny foreign bodies in a power transmission line based on improved YOLACT, characterized in that: The process of marking the edge contours and categories of foreign objects invading the transmission line is achieved using Labelme labeling software.

5. A method for detecting tiny foreign bodies in a power transmission line based on improved YOLACT according to any one of claims 1 to 4, characterized in that: The YOLACT instance segmentation model consists of four parts: feature extraction network backbone, feature pyramid network FPN, mask generation branch Protonet and prediction head Prediction Head. Backbone includes 5 convolutional layers, corresponding to C1 to C5, for extracting multi-scale feature maps; the feature pyramid of FPN is recorded as P3 to P7 layers from large to small according to the feature map, and the features of C3 to C5 correspond to P3 to P5 layers respectively; after the FPN processing is completed, the P3 layer is input to the Protonet part, and P3-P7 are all sent to the Prediction Head part; Protonet generates k 138×138 prototype masks, and the prediction head generates the confidence of k mask prototype images and removes repeated detection boxes through non-maximum suppression operation. After linearly combining the two and setting the threshold, the instance segmentation result can be generated, showing the category, confidence and bounding box of the foreign body in the transmission line.

6. The method for detecting tiny foreign bodies in power transmission lines based on improved YOLACT according to claim 5, characterized in that: In step S2, a test set is determined based on the foreign body image data set; in step S4, the test set is also used to test the detection effect of the foreign body detection network model.

Images