A Fast Detection Method for Conductor and Ground Wire Defects Based on a Lightweight Cascade Network

Abstract

A fast detection method for conductor and ground wire defects based on a lightweight cascaded network, comprising: manually annotating the conductor and ground wire images collected by a drone to generate a training set and a test set for conductor and ground wire defects. In the training module, use the image data of the annotated training set for conductor and ground wire defects to train the TCDNet model, and use the test set for testing to obtain a model weight file with the highest accuracy; send the trained model weight file into the model compression and pruning module to perform sparse training on the model weight file, iterate continuously to achieve compression and pruning, and output a lightweight TCDNet model weight file; deploy the lightweight TCDNet model weight file into the inference module, and then send the conductor and ground wire image to be detected into the inference module to obtain the detection result of the conductor and ground wire defects, and output the confidence, position and size of the defects respectively. The present invention does not require complicated manual data analysis, and has a high defect discovery rate, fast speed and few missed detections.

Description

Technical Field

[0001] The present invention belongs to the technical fields of deep learning, artificial intelligence, and distribution network, and relates to deep convolutional neural network (CNN), compression and pruning of models, and defect detection. Specifically, it relates to a method for quickly detecting conductor defects based on a lightweight cascaded network. Background Art

[0002] A conductor is a common facility in the power system and is one of the most important components in the transmission line. It relies on the conductor to transmit electricity to users and forms a power network with it to balance the power supply in various places. However, due to the complex and changeable field environment where the transmission line is located, the conductor will be corroded by rainwater and affected by micro-vibration for a long time, resulting in easy occurrence of fretting wear of the conductor. The superposition of wear and stress concentration effect leads to the generation and expansion of radial cracks in the conductor at this place. Under the action of periodic stress, the cracks gradually develop and finally lead to fatigue fracture of the conductor. Such cracks in the conductor are very likely to affect the safe operation of the entire transmission line. At worst, it may cause tripping, and at worst, it may cause the conductor to break or the tower to collapse, resulting in large-scale power outages. Common conductor defects include broken strands, loose strands, damage, etc.

[0003] Currently, the defect detection of conductors can be achieved through the drone cruise photography technology. By operating the drone to collect on-site data at key positions of the tower, and then transmitting the image data back to the server, and finally analyzing the data manually. This method consumes a large amount of resources during the data transmission process, and the efficiency of manual data analysis is low, making it difficult to achieve real-time monitoring. In addition, some scholars have proposed using object detection models such as YOLOv3, YOLOv4, and Faster-RCNN for the defect detection of conductors, but neither the accuracy nor the speed has reached a balance. The situation where the accuracy is high and the speed is slow and the speed is fast and the accuracy is low is common in the current conductor defect detection. Therefore, the industrial community extremely needs a detection method that is simple and convenient to implement, has a relatively fast speed, and a relatively high accuracy, and can batch, intelligently, and quickly identify the defects on the conductor based on the visible light images taken by the drone during patrol inspection. Summary of the Invention

[0004] In view of this, the present invention provides a method for quickly detecting conductor defects based on a lightweight cascaded network.

[0005] The object of the present invention is achieved through the following technical solutions. A method for quickly detecting conductor defects based on a lightweight cascaded network is carried out by using a training module, a model compression and pruning module, and an inference module. The method includes the following steps:

[0006] Manually annotate the rectangular frames of the conductor images collected by the drone to generate a conductor defect training set and a test set;

[0007] In the training module, the TCDNet model is trained with the image data of the labeled ground wire defect training set, and tested with the test set to obtain a model weight file with the highest accuracy;

[0008] The trained model weight file is fed into the model compression and pruning module to perform sparse training on the model weight file, iterate continuously to achieve compression and pruning, and output a lightweight TCDNet model weight file;

[0009] The lightweight TCDNet model weight file is deployed to the inference module. The ground wire image to be detected is then fed into the inference module to obtain the detection results of the ground wire defects, and the confidence, position, and size of the defects are output respectively.

[0010] Furthermore, the training module uses the two-stage cascaded object detection network TCDNet as the baseline detection model for ground wire defects. The neck of TCDNet adopts the feature pyramid pattern, and the head is composed of two cascaded detection heads at the front and back levels on this basis; in the training stage, different intersection over union (IoU) thresholds are set for the two cascaded detection heads at the front and back levels to define positive and negative samples. The output of the previous detection head is the input of the next detection head, and the IoU threshold keeps rising. Different IoU thresholds are used to divide positive and negative samples. The detection head has two branches for class prediction and coordinate prediction, that is, classification calculation and coordinate regression are performed on the candidate targets respectively.

[0011] Furthermore, the total loss function during training consists of a location loss function and a classification loss function. Among them, the location loss function is the same as that used in Fast-RCNN, which is the L1 loss, as follows:

[0012]

[0013]

[0014] Among them, x i is a single input sample of the neural network, W is the weight parameter of the neural network, f(x i , W) is the coordinate regression mapping function for a single sample, and g i is the true coordinate regression encoding of this sample. L loc (·, ·) is the location loss function for a single sample;

[0015] The classification loss is the cross-entropy loss function, that is

[0016]

[0017] Among them, x i is a single input sample of the neural network, W is the weight parameter of the neural network, h(x i) is the classification mapping function for a single sample, y i is the true classification label of the sample, L cls (·,·) is the classification loss function for a single sample.

[0018] Furthermore, the backbone network of the two-stage tandem target detection network TCDNet consists of 5 multi-hop connection units, with a total of 21 convolutional layers and 4 pooling layers.

[0019] Furthermore, the trained model weight file is sent to the model compression and pruning module, and the model weight file is sparsely trained, iterated continuously, compressed and pruned, and a lightweight TCDNet model weight file is output, including:

[0020] The TCDNet model obtained after training the training module is used as the baseline model, and the compression and pruning module performs another sparse training, pruning and fine-tuning. For the baseline model, a scaling factor γ corresponding to the importance of each group / block / channel level structure in the TCDNet model is assigned, or the scaling factor γ in the BN layer is directly used. The scaling factor γ corresponding to each level structure is initialized with a large variance Gaussian distribution, and then the L1 norm regularization is applied to the scaling factor γ. At the same time, the subgradient optimization algorithm is used for sparse training. The optimization objective function is as follows:

[0021]

[0022] Where φ(γ) = |γ|, Г is the set of scaling factors γ, λ is the regularization weight coefficient, C(f(x,W),y) is the cost function of a single sample (x,y), and W is the weight parameter of the neural network;

[0023] For the sparsely trained model, the channel / group / block corresponding to the scaling factor γ close to 0 in the model is pruned, and then the model is fine-tuned to obtain a lightweight model.

[0024] The beneficial effects of the present invention are:

[0025] The present invention proposes a method for rapid identification of ground wire defects in intelligent inspection of power transmission lines. It is mainly based on cutting-edge artificial intelligence and computer vision target detection technology, uses deep convolutional neural networks as feature extractors, and designs a two-stage series target detection network to detect ground wire defects. At the same time, it also uses model compression and pruning technology to lightweight the network model to improve the computing speed. Compared with the existing ground wire defect detection method, it does not require complicated manual data analysis, and has a high defect detection rate, fast speed and few missed detections. Once the model is trained, it is easy to deploy and has strong portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By reading the detailed description in the preferred specific embodiments below, various other advantages and benefits of the present invention will become clear to those of ordinary skill in the art. The accompanying drawings of the specification are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts; and throughout the drawings, the same reference numerals are used to represent the same components.

[0027] Figure 1 It is a schematic diagram of the process for labeling ground wire defect data and dividing the data set;

[0028] Figure 2 It is a schematic diagram of the structure of the ground wire defect baseline detection model TCDNet;

[0029] Figure 3 It is a schematic diagram of the structure of MSLCU;

[0030] Figure 4 It is a schematic diagram of general structured pruning;

[0031] Figure 5 It is a detection and recognition result diagram of ground wire defects;

[0032] Figure 6 It is the overall flowchart of one embodiment of a fast ground wire defect detection method based on a lightweight cascaded network of the present invention. Specific Embodiments

[0033] The following will refer to the attached Figures 1 to 6 The specific embodiments of the present invention will be described in more detail. Although specific embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be completely conveyed to those skilled in the art.

[0034] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. The specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of components as the criterion for distinction. For example, the terms "comprising" or "including" mentioned throughout the specification and claims are open-ended terms, so they should be interpreted as "including but not limited to". The subsequent description in the specification is the preferred embodiment for implementing the present invention, but the description is for the purpose of the general principles of the specification and is not used to limit the scope of the present invention. The protection scope of the present invention shall be subject to what is defined by the appended claims.

[0035] To facilitate the understanding of the embodiments of the present invention, the following will further explain with specific embodiments in conjunction with the accompanying drawings, and the accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0036] As Figure 6 shown, one embodiment of a fast detection method for conductor and ground wire defects based on a lightweight cascaded network of the present invention includes the following steps:

[0037] (1) Manually annotate the conductor and ground wire images collected by the drone with rectangular frames to generate a training set and a test set for conductor and ground wire defects;

[0038] (2) In the training module, use the image data of the annotated training set for conductor and ground wire defects to train the TCDNet model, and use the test set for testing to obtain a model weight file with the highest accuracy;

[0039] (3) Send the model weight file trained in the previous step into the model compression and pruning module, perform sparse training on the model weight file, and iterate continuously to achieve compression and pruning, and output a lightweight TCDNet model weight file;

[0040] (4) Deploy the lightweight TCDNet model weight file into the inference module, and then send the conductor and ground wire image to be detected into the inference module to obtain the detection result of the conductor and ground wire defects, and output the confidence, position and size of the defects respectively.

[0041] The fast detection method for conductor and ground wire defects based on a lightweight cascaded network mainly includes three sub-modules: a training module, a compression and pruning module, and an inference module. The specific implementation steps are as follows:

[0042] (1) Training module

[0043] First, manually annotate the conductor and ground wire pictures collected by the drone to generate a training set and a test set (divided at 8:2), and the process is as Figure 1As shown, the convolutional neural network model of the training module is trained with the picture data of the training set and tested with the test set.

[0044] As Figure 2 shown, the present invention uses the two-stage cascade target detection network TCDNet (Two-stage Cascade Detection Network, TCDNet) as the basic detection model for conductor defects. The backbone of TCDNet consists of 5 MSLCUs (Multiple Skipping Layer Connection Units), with a total of 21 convolutional layers and 4 pooling layers. As Figure 3 shown, "CBL" in the MSLCU is a fixed combination composed of Conv (convolutional layer), BN (batch normalization layer), and Relu (activation function layer). The use of MSLCU can make the gradient backpropagation smoother during model training and easier to converge. The neck of TCDNet adopts the FPN (Feature Pyramid) mode, and the head is based on this and cascades two levels of detection heads. During the training stage, different IOU (Intersection over Union, the same below) thresholds are set for the front and rear cascaded levels of detection heads to define positive and negative samples. The output of the previous detection head is the input of the next detection head, and the IOU threshold keeps rising, such as taking values of 0.5 and 0.6 respectively. Using different IOU thresholds to divide positive and negative samples enables each level of detection head to focus on detecting proposals (candidate targets) with an IOU within a certain range. Since the output IOU is generally greater than the input IOU, the detection effect will get better and better. The detection head has two branches, cls_pred (class prediction) and bbox_pred (coordinate prediction), that is, classification calculation and coordinate regression are performed on the candidate targets respectively.

[0045] The total loss function during training consists of a location loss function and a classification loss function. Specifically, the location loss function is the same as that used in Fast-RCNN, both are L1 losses, as follows:

[0046]

[0047]

[0048] Among them, x i is a single input sample of the neural network, and W is the weight parameter of the neural network. f(x i , b i ) is the coordinate regression mapping function for a single sample, and g i is the true coordinate regression encoding of this sample, and L loc (·,·) is the location loss function for a single sample.

[0049] The classification loss is the cross-entropy loss function, that is

[0050]

[0051] where x i is a single input sample of the neural network, W is the weight parameter of the neural network, h(x i ) is the classification mapping function for a single sample, y i is the true classification label of this sample, and L cls (·,·) is the classification loss function for a single sample.

[0052] Compared with other object detection algorithms, the detection of the ground wire defect in the present invention adopts the form of cascading two-level detection heads, and different IOU thresholds are set to define positive and negative samples. The output of the previous detection head is the input of the next detection head, and the IOU value keeps rising, continuously refining the detection result, so that the detection result becomes more and more accurate.

[0053] In this embodiment, in order to train the ground wire defect detection model TCDNet, the transmission line was cruised and photographed in the wild by a drone. A total of 3680 transmission line pictures were collected. After removing the pictures with poor shooting quality and extremely high overlap and no defects, 3330 pictures were left for training the model, and 385 pictures were used as the test set. The ratio of defective and non-defective samples was both 5:1. All training set pictures were uniformly scaled to 3000*1800 and sent into the network for training, and random flipping was used as the online data augmentation method, and at the same time the pictures were normalized. The initial learning rate was set to 0.0075, and the warmup and step-down strategies were adopted, with a momentum of 0.9 and a regularization term weight_decay of 0.0001. A total of 12 epochs were trained, and the simplest stochastic gradient descent method SGD was used as the optimizer.

[0054] To verify the effectiveness of the ground wire defect detection model TCDNet of the present invention, other detection models were also trained. The specific detection effects are compared in Table 1, and the frame rates were all tested on a 2080Ti graphics card.

[0055] Table 1

[0056] model detection rate false detection ratio frame rate fps YOLOv3 74.6％ 6.8 5.2 YOLOv4 78.1％ 5.1 4.1 Faster-RCNN 81.7％ 5.7 2.5 TCDNet (ours) 85.9％ 3.2 3.4 TCDNet (ours, after compression and pruning) 84.5％ 3.5 4.8

[0057] Among them: M1 is the total number of all correct targets output by the algorithm, M2 is the total number of all targets output by the algorithm, and M is the total number of manually marked defective targets in all test pictures. It can be seen from Table 1 that the detection model TCDNet of the present invention has the highest detection accuracy, the discovery rate can reach 85.9%, and the false detection ratio is also relatively low, only 3.2. However, at present, the detection speed of TCDNet is still relatively slow, only 3.4fps.

[0058] (2) Compression and pruning module

[0059] Take the TCDNet model obtained by training in the previous training module as the baseline (baseline model), and perform another round of sparsification training, pruning, and fine-tuning by the compression and pruning module.

[0060] For the baseline, such as Figure 4 Assign a corresponding scaling factor γ to measure the importance of each group / block / channel level structure in the network model, or directly use the scaling factor γ in the BN layer for convenience. The corresponding scaling factor γ of each level structure is initialized with a large variance Gaussian distribution to improve the sparsity and compression rate of the model, and then apply L1 norm regularization to the scaling factor γ; at the same time, use the sub-gradient optimization algorithm for sparsification training, and the optimization objective function is as follows:

[0061]

[0062] Where φ(γ)=|γ|, Г is the set of scaling factors γ, and λ is the weight coefficient of the regularization term. C(f(x,W),y) is the cost function of a single sample (x,y), and W is the weight parameter of the neural network.

[0063] For the model after sparsification training, cut the channels / groups / blocks corresponding to the scaling factors γ that are close to 0 in the model, and then fine-tune the model to obtain a lightweight model.

[0064] Compared with other model pruning methods, the method in the present invention adopts structured pruning, which is simple to implement and does not require modifying the underlying algorithm library; uses the sub-gradient optimization algorithm for sparsification training, has a higher compression rate and faster compression speed; and can perform multi-level compression on the model according to needs, which is more flexible.

[0065] (3) Inference module

[0066] Deploy the lightweight detection model obtained by the compression and pruning module to the server or edge device, and provide the corresponding input interface. The input of the ground wire picture to be detected into the inference module can detect the defective targets in the picture. Taking Figure 5For example, as in the training process, the image of the ground wire to be measured is also scaled to the size of 3000*1800 for inference. After the image to be measured is input into the inference module, through a series of calculations of the object detection model after training and compression, the detected positions of the ground wire defects (marked by rectangular frames on the original image) and the class confidence are output.

[0067] As can be seen from Table 1, compared with the existing detection methods, the detection model TCDNet designed in the present invention has been compressed and pruned, and its inference speed can reach 4.8fps, which is faster than most of the existing detection models. Moreover, the discovery rate only drops slightly and can remain at 84.5%. Generally speaking, the fast detection method for ground wire defects designed in the present invention can better assist the front-line grid team members in identifying and locating the ground wire defects, saving a large amount of manpower and material resources.

[0068] Although the embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields. The above specific embodiments are merely illustrative and guiding, rather than restrictive. Under the inspiration of this specification and without departing from the scope protected by the claims of the present invention, those of ordinary skill in the art can also make many forms, and these all belong to the scope of protection of the present invention.

Claims

1. A fast detection method for conductor and ground wire defects based on a lightweight cascaded network, characterized in that It is carried out by using a training module, a model compression and pruning module, and an inference module. The method includes the following steps: Manually annotate the ground wire images collected by the drone with rectangular boxes to generate a ground wire defect training set and a test set; In the training module, use the annotated ground wire defect training set image data to train the TCDNet model, and use the test set to test it to obtain a model weight file with the highest accuracy; Send the trained model weight file into the model compression and pruning module to perform sparse training on the model weight file, iterate continuously to achieve compression and pruning, and output a lightweight TCDNet model weight file; Deploy the lightweight TCDNet model weight file into the inference module, and then send the ground wire image to be detected into the inference module to obtain the detection result of the ground wire defect, and output the confidence, position, and size of the defect respectively; The training module uses the two-stage cascaded object detection network TCDNet as the baseline detection model for ground wire defects. The neck of TCDNet adopts a feature pyramid pattern, and the head is composed of two cascaded detection heads at the front and back levels on this basis; in the training stage, different intersection over union (IoU) thresholds are set for the two cascaded detection heads at the front and back levels to define positive and negative samples. The output of the previous detection head is the input of the next detection head, and the IoU threshold keeps rising. Different IoU thresholds are used to divide positive and negative samples. The detection head has two branches: class prediction and coordinate prediction, that is, classification calculation and coordinate regression are performed on the candidate targets respectively; The total loss function during training consists of a location loss function and a classification loss function. Among them, the location loss function is the same as that used in Fast-RCNN, which is the L1 loss, as follows: ； ； Among them, is a single input sample of the neural network, are the weight parameters of the neural network, is the coordinate regression mapping function for a single sample, is the true coordinate regression encoding of the sample, is the position loss function for a single sample; The classification loss is the cross-entropy loss function, that is ； Among them, is a single input sample of the neural network, is the weight parameter of the neural network, is the classification mapping function for a single sample, then is the true classification label of this sample, is the classification loss function for a single sample.

2. The method according to claim 1, wherein The backbone network of the two-stage cascaded object detection network TCDNet consists of 5 multi-hop layer connection units, with a total of 21 convolutional layers and 4 pooling layers.

3. The method according to claim 1, wherein Send the trained model weight file into the model compression and pruning module to perform sparse training on the model weight file, iterate continuously to achieve compression and pruning, and output a lightweight TCDNet model weight file, specifically including: Take the TCDNet model obtained after the training module finishes training as the baseline model, and perform another round of sparse training, pruning, and fine-tuning by the compression and pruning module. For the baseline model, assign a scaling factor γ corresponding to measuring the importance to each group / block / channel level structure in the TCDNet model, or directly use the scaling factor γ in the BN layer. The scaling factor γ corresponding to each level structure is initialized with a large variance Gaussian distribution, and then L1 norm regularization is applied to the scaling factor γ. At the same time, the subgradient optimization algorithm is used for sparse training, and the optimization objective function is as follows: ； where , Г is the set of scaling factors γ, λ is the regularization term weight coefficient, is the cost function of a single sample (x, y), are the weight parameters of the neural network; For the model after sparse training, crop the channels / groups / blocks corresponding to the scaling factor γ close to 0 in the model, and then fine-tune the model to obtain a lightweight model.

Images