Device for detecting defects of insulators and method for obtaining the device

Abstract

The application discloses a neural network for detecting insulator defects and a method for obtaining the neural network, and relates to the technical field of neural networks; the neural network comprises a VSPENet network (V network), a decoding network and a detection network which are connected in sequence, the V network comprises first to fifth stage networks which are connected in sequence, the first stage network comprises a deformable convolution network and a max-pooling layer which are connected in sequence, the second to fifth stage networks have the same structure, the second stage network comprises a feature fusion module, a convolution normalization function, a hollow space pyramid module and a max-pooling layer which are connected in sequence; the method comprises dividing the network into five stages based on a VoVNet coding network to obtain an improved V network, and replacing a coding network in an existing CenterNet network; the first to fifth stage networks and the like have good recognition capability when the insulator of a power transmission line is damaged and has irregular flashover shapes, and the insulator facet defect detection accuracy is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks, and in particular to a device for detecting defects of insulators and a method for obtaining the device. BACKGROUND

[0002] Insulators are crucial power devices in power system transmission. Due to the influence of severe weather such as storms and lightning, insulators are prone to faults such as body damage and flashover discharge, affecting the stability and safety of the power system. If the power safety hazards cannot be effectively found and eliminated in the first time, it will cause the overall power line to fail to operate, resulting in a large consumption of manpower, material resources and financial resources. Therefore, it is of great research significance to select foreign objects hanging and insulator defects for research in order to quickly and accurately eliminate the faults of the power transmission line and timely handle them to ensure the safe and stable operation of the power system.

[0003] Traditional power transmission line detection based on manual inspection requires professional personnel to carry relevant detection equipment to climb high-voltage line towers to inspect the power transmission line. This inspection method is low in efficiency, has many types of component faults, is easily affected by external environment and weather, and is often carried out in the wild, requiring high professional ability of the workers and being extremely dangerous, which cannot meet the daily inspection needs of the power transmission line. In the 1990s, China began to explore the use of unmanned aerial vehicle inspection technology to conduct daily inspection of the power transmission line. Compared with manual inspection, this new detection form is more flexible and diverse, safe and convenient to operate, and avoids many shortcomings of manual inspection. However, this mode only enables the unmanned aerial vehicle to complete data collection, and the ground inspection personnel still need to manually inspect the massive line data set in a complex and inefficient manner. Due to visual fatigue, it is easy to miss detection, and the work intensity is large.

[0004] The prior art extracts the insulator contour according to the horizontal gradient feature and restores the internal pixels, compares the insulator sheet interval and the gray scale similarity with the set threshold value at the same time, and determines whether the insulator has defects. The disadvantage of this traditional detection method is that the image feature extraction process is very complex and slow, and the algorithm performance is easily affected by the complex geographical background and various weather conditions of the power transmission line, which greatly limits the practical application of the algorithm and has low generalization ability.

[0005] Problems and considerations of the prior art:

[0006] How to solve the technical problems of poor recognition ability when the insulator of the power transmission line is damaged and has irregular flashover shape, lack of utilization of shallow feature information in the feature extraction process, and low accuracy of insulator facet defect detection. SUMMARY

[0007] The application provides a neural network for detecting insulator defects and a method for obtaining the neural network, and solves the technical problems of poor recognition ability when a power transmission line insulator is damaged or flashover shape is irregular, lack of utilization of shallow feature information in a feature extraction process, and low accuracy of insulator facet defect detection.

[0008] To solve the above technical problems, the technical solution adopted by the application is as follows:

[0009] A neural network for detecting insulator defects comprises a CenterNet network formed by sequentially connecting an encoding network, a decoding network and a detection network, the encoding network is a VSPENet network, the VSPENet network comprises first to fifth stage networks connected in sequence, the first stage network comprises a deformable convolution network and a max pooling layer connected in sequence, the second, third, fourth and fifth stage networks have the same structure, the second stage network comprises a feature fusion module, a convolution normalization function, a hollow spatial pyramid module and a max pooling layer connected in sequence.

[0010] Further technical solutions are as follows: the feature fusion module comprises first to fifth feature layers, first to third fusioners, a first feature fusion layer and a second feature fusion layer, the output end of the first feature layer is connected with the input end of the first fusioner, the output end of the third feature layer is connected with the input end of the first fusioner, the output end of the first fusioner is connected with the convolution normalization function in sequence through the first feature fusion layer, the second fusioner, the second feature fusion layer and the third fusioner, the output end of the fifth feature layer is connected with the input end of the second fusioner, all the output ends of the first to fifth feature layers are connected with the input end of the third fusioner, and the output end of the first feature fusion layer is connected with the output end of the second feature fusion layer.

[0011] Further technical solutions are as follows: the deformable convolution network of the first stage network comprises three 3x3 deformable convolution networks connected in sequence; and the convolution normalization function comprises a 1x1 convolution.

[0012] Further technical solutions are as follows: the CenterNet network containing the VSPENet network is a network for improving network accuracy on the premise of ensuring network detection speed, the VSPENet network is a network for transmitting part of information in the input, so that the final result still has sufficient semantic information on the basis of reducing the amount of parameters and does not cause excessive loss of information, and the deformable convolution network is a network for expanding the receptive field and improving the accuracy of insulator damage detection.

[0013] The further technical scheme is characterized in that the feature fusion module is a one-time aggregation module, and the one-time aggregation module is a module for enhancing the correlation between feature maps, increasing the usage rate of deep and shallow features, avoiding gradient loss, generating features with a larger receptive field, improving detection accuracy, having better perception of edge and texture detail information of the insulator, and improving detection efficiency when the shape of the insulator is irregular.

[0014] The further technical scheme is characterized in that the hollow space pyramid module is a module formed by integrating the SE attention mechanism module into the end of the existing hollow space pyramid module, and is used to obtain the weight of each feature channel in the existing hollow space pyramid module, obtain more useful channel information, retain as much feature information as possible, and reduce the amount of calculation.

[0015] The further technical scheme is characterized in that the second-stage network comprises a feature fusion module, a convolution normalization function, a hollow space pyramid module and a maximum pooling layer connected in sequence, the third-stage network comprises a feature fusion module, a convolution normalization function, a hollow space pyramid module and a maximum pooling layer connected in sequence, the fourth-stage network comprises a feature fusion module, a convolution normalization function, a hollow space pyramid module and a maximum pooling layer connected in sequence, the fifth-stage network comprises a feature fusion module, a convolution normalization function, a hollow space pyramid module and a maximum pooling layer connected in sequence, the maximum pooling layer of the first-stage network is connected with the feature fusion module of the second-stage network, the maximum pooling layer of the second-stage network is connected with the feature fusion module of the third-stage network, the maximum pooling layer of the third-stage network is connected with the feature fusion module of the fourth-stage network, and the maximum pooling layer of the fourth-stage network is connected with the feature fusion module of the fifth-stage network.

[0016] A method for obtaining a neural network comprises the following steps: based on a VoVNet encoding network, dividing the network into five stages, obtaining an improved VSPENet network, replacing the encoding network in an existing CenterNet network with the VSPENet network, and obtaining an improved CenterNet network.

[0017] The further technical scheme is characterized in that the method further comprises the following steps: connecting each feature layer in a feature fusion module together, for enhancing the correlation between feature maps, increasing the usage rate of deep and shallow features, and avoiding gradient loss; and fusing the first feature layer, the third feature layer and the fifth feature layer in a feature fusion module, for generating features with a larger receptive field, improving detection accuracy, and obtaining a one-time aggregation module.

[0018] Further technical solutions are as follows: further comprising the steps of: integrating the SE attention mechanism module into the end of the existing hole space pyramid module to obtain an improved hole space pyramid module, obtaining the weight of each feature channel in the existing hole space pyramid module, obtaining more useful channel information, retaining all feature information as much as possible, and reducing the amount of calculation.

[0019] The beneficial effects of the above technical solutions are as follows:

[0020] First, a neural network for detecting defects of insulators, comprising an encoding network, a decoding network and a detection network connected in sequence and forming a CenterNet network, the encoding network is a VSPENet network, the VSPENet network comprises first to fifth stage networks connected in sequence, the first stage network comprises a deformable convolution network and a max-pooling layer connected in sequence, the second stage network, the third stage network, the fourth stage network and the fifth stage network have the same structure, the second stage network comprises a feature fusion module, a convolution normalization function, a hole space pyramid module and a max-pooling layer connected in sequence. The technical solution has good recognition ability when the insulator of the power transmission line is damaged and the flashover shape is irregular, the feature extraction process makes good use of shallow feature information, and the insulator facet defect detection accuracy is high.

[0021] Second, the feature fusion module comprises first to fifth feature layers, first to third fusioners, a first feature fusion layer and a second feature fusion layer, the output end of the first feature layer is connected with the input end of the first fusioner, the output end of the third feature layer is connected with the input end of the first fusioner, the output end of the first fusioner is connected with the convolution normalization function in sequence through the first feature fusion layer, the second fusioner, the second feature fusion layer and the third fusioner, the output end of the fifth feature layer is connected with the input end of the second fusioner, all output ends of the first to fifth feature layers are connected with the input end of the third fusioner, and the output end of the first feature fusion layer is connected with the output end of the second feature fusion layer. The technical solution, the feature fusion module is a one-time aggregation module, the one-time aggregation module is used for enhancing the correlation between feature maps, increasing the usage rate of deep and shallow features, avoiding gradient loss, generating larger receptive field features, improving detection accuracy, having better perception ability for edge and texture detail information of the insulator, and improving the detection efficiency when the shape of the insulator is irregular.

[0022] Third, a method for obtaining a neural network includes the following steps: based on a VoVNet encoding network, dividing the network into five stages, obtaining an improved VSPENet network, replacing the encoding network in the existing CenterNet network with the VSPENet network, and obtaining an improved CenterNet network. The technical scheme has good recognition ability for irregular insulator damage and flashover shapes, good utilization of shallow feature information in the feature extraction process, and high accuracy of insulator facet defect detection.

[0023] See the specific embodiment part for details. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is the structure diagram of the VSPENet network in embodiment 1 of the present application;

[0025] Figure 2 is a flowchart of embodiment 2 of the present application;

[0026] Figure 3 is a flowchart of research and development example 1;

[0027] Figure 4 is a flowchart of research and development example 2;

[0028] Figure 5 is a flowchart of research and development example 3;

[0029] Figure 6 is a structure diagram of an existing CenterNet network;

[0030] Figure 7 is a structure diagram of a one-time aggregation module;

[0031] Figure 8a is a data diagram of ordinary convolution;

[0032] Figure 8b is a data diagram of deformable convolution;

[0033] Figure 9 is a structure diagram of a feature fusion module;

[0034] Figure 10 is a structure diagram of an improved hollow space pyramid module;

[0035] Figure 11 is a screenshot of an insulator damage and flashover picture set;

[0036] Figure 12a is a picture of the original picture;

[0037] Figure 12b is a picture after cropping processing;

[0038] Figure 12cis the picture after rotation processing;

[0039] Figure 12d is the picture after brightness processing;

[0040] Figure 13a is the marked picture of broken insulator;

[0041] Figure 13b is the marked picture of insulator flashover;

[0042] Figure 14a is the mAP curve of CenterNet-VoVNet;

[0043] Figure 14b is the mAP curve of CenterNet-VSPENet;

[0044] Figure 15a is the data graph of original network detection result;

[0045] Figure 15b is the data graph of original network detection result;

[0046] Figure 15c is the data graph of original network detection result;

[0047] Figure 15d is the data graph of original network detection result;

[0048] Figure 15e is the data graph of original network detection result;

[0049] Figure 15f is the data graph of original network detection result;

[0050] Figure 15g is the data graph of original network detection result;

[0051] Figure 15h is the data graph of original network detection result;

[0052] Figure 15i is the data graph of original network detection result;

[0053] Figure 15j is the data graph of original network detection result;

[0054] Figure 15k is the data graph of original network detection result;

[0055] Figure 15l is the data graph of original network detection result;

[0056] Figure 16a is the data graph of improved network detection result;

[0057] Figure 16b is a data graph of the improved network detection result;

[0058] Figure 16c is a data graph of the improved network detection result;

[0059] Figure 16d is a data graph of the improved network detection result;

[0060] Figure 16e is a data graph of the improved network detection result;

[0061] Figure 16f is a data graph of the improved network detection result;

[0062] Figure 16g is a data graph of the improved network detection result;

[0063] Figure 16h is a data graph of the improved network detection result;

[0064] Figure 16i is a data graph of the improved network detection result;

[0065] Figure 16j is a data graph of the improved network detection result;

[0066] Figure 16k is a data graph of the improved network detection result;

[0067] Figure 16l is a data graph of the improved network detection result. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work, fall within the scope of protection of the present application.

[0069] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0070] Embodiment 1:

[0071] As Figure 1As shown, the application discloses a kind of neural networks for detecting insulator defects, including sequentially connected coding network, decoding network and detection network and form CenterNet network, the coding network is VSPENet network, VSPENet network includes sequentially connected first to fifth stage network, first stage network includes sequentially connected deformable convolution network and maximum pooling layer, the structure of second stage network, third stage network, fourth stage network and fifth stage network is same, second stage network includes sequentially connected feature fusion module, convolution normalization function, hollow spatial pyramid module and maximum pooling layer.

[0072] Feature fusion module includes first to fifth feature layer, first to third fusion ware, first feature fusion layer and second feature fusion layer, the output of first feature layer is connected with the input of first fusion ware, the output of third feature layer is connected with the input of first fusion ware, the output of first fusion ware is sequentially connected with convolution normalization function through first feature fusion layer, second fusion ware, second feature fusion layer and third fusion ware, the output of fifth feature layer is connected with the input of second fusion ware, the input of third fusion ware is connected with the output of all first to fifth feature layer, the output of first feature fusion layer is connected with the output of second feature fusion layer.

[0073] Second stage network includes sequentially connected feature fusion module, convolution normalization function, hollow spatial pyramid module and maximum pooling layer, third stage network includes sequentially connected feature fusion module, convolution normalization function, hollow spatial pyramid module and maximum pooling layer, fourth stage network includes sequentially connected feature fusion module, convolution normalization function, hollow spatial pyramid module and maximum pooling layer, fifth stage network includes sequentially connected feature fusion module, convolution normalization function, hollow spatial pyramid module and maximum pooling layer, the maximum pooling layer of first stage network is connected with the feature fusion module of second stage network, the maximum pooling layer of second stage network is connected with the feature fusion module of third stage network, the maximum pooling layer of third stage network is connected with the feature fusion module of fourth stage network, the maximum pooling layer of fourth stage network is connected with the feature fusion module of fifth stage network.

[0074] The deformable convolution network of first stage network includes three 3x3 deformable convolution network, three 3x3 deformable convolution network is sequentially connected.

[0075] Convolution normalization function includes 1x1 convolution.

[0076] The CenterNet network containing the VSPENet network is a network for improving network accuracy under the premise of ensuring network detection speed, the VSPENet network is a network for transmitting part of information in the input, so that the final result still has sufficient semantic information on the basis of reducing the parameter amount, and does not cause excessive loss of information, and the deformable convolution network is a network for expanding the receptive field and improving the insulator damage detection accuracy.

[0077] The feature fusion module is a one-time aggregation module, the one-time aggregation module is a module for enhancing the relevance between feature maps, increasing the usage rate of deep and shallow features, avoiding gradient loss, generating larger receptive field features, improving detection accuracy, having better perception ability for edge and texture detail information of the insulator, and improving detection efficiency when the insulator shape is irregular.

[0078] The cavity space pyramid module is a module formed by integrating the SE attention mechanism module into the end of the existing cavity space pyramid module, and is used for obtaining the weight of each feature channel in the existing cavity space pyramid module, obtaining more useful channel information, retaining all feature information as much as possible, and reducing the amount of calculation.

[0079] Embodiment 2:

[0080] As shown in Figure 2 The method for obtaining the neural network in embodiment 1 comprises the following steps:

[0081] Based on the VoVNet encoding network, the network is divided into five stages to obtain an improved VSPENet network, the VSPENet network is replaced with the existing CenterNet network to obtain an improved CenterNet network.

[0082] The deformable convolution network of the first stage network uses three 3x3 deformable convolution networks, and the three 3x3 deformable convolution networks are connected in sequence.

[0083] Each feature layer in one feature fusion module is connected together, which is used for enhancing the relevance between feature maps, increasing the usage rate of deep and shallow features, avoiding gradient loss, fusing the first feature layer, the third feature layer and the fifth feature layer in one feature fusion module, which is used for generating larger receptive field features, improving detection accuracy, and obtaining a one-time aggregation module.

[0084] The SE attention mechanism module is integrated into the end of the existing cavity space pyramid module to obtain an improved cavity space pyramid module, which is used for obtaining the weight of each feature channel in the existing cavity space pyramid module, obtaining more useful channel information, retaining all feature information as much as possible, and reducing the amount of calculation.

[0085] Embodiment 3

[0086] The application discloses a method for detecting insulator defects, comprising the following steps:

[0087] Obtaining the insulator image on site, inputting the CenterNet network of embodiment 1, and detecting the insulator defects.

[0088] The concept of the application:

[0089] With the rapid development of modern science and technology and the emphasis on the safety of power transmission lines, robots are widely used in the inspection of high-voltage power transmission lines and the elimination of line faults due to their lightweight characteristics. In recent years, deep learning algorithms have developed rapidly and have good performance in multiple application scenarios. Therefore, the combination of deep learning algorithms and robot automatic inspection equipment for detecting power transmission lines can greatly improve the efficiency of power transmission line inspection, thereby ensuring the reliable operation of power transmission lines and greatly improving the intelligent inspection degree of power lines, which has important practical significance for promoting the intelligent construction of power grids.

[0090] The application point lies in that a VSPENet network structure is designed to replace the original coding network of the CenterNet network. Firstly, a lightweight network structure based on deformable convolution is designed, then a feature fusion structure is designed to make the network better fuse multi-layer features, and finally an improved hollow space pyramid module is introduced.

[0091] 1. The most basic technical problem to be solved

[0092] The application solves the problem of poor recognition ability of the damaged insulators of the power transmission line and the irregular flashover shape, and the problem of lack of utilization of shallow feature information in the feature extraction process, and improves the accuracy of insulator facet defect detection.

[0093] 2. Core technical solution

[0094] For example Figure 1As shown, the improved VSPENet network structure of the application on the basis of VoVNet network, VSPENet network is composed of deformable convolution network, feature fusion module, convolution normalization function, hollow space pyramid module and maximum pooling layer. The first stage is composed of 3 3 3 deformable convolution network and maximum pooling. The sampling network of deformable convolution can deform freely, which expands the receptive field and increases the accuracy of insulator damage detection. The second, third, fourth and fifth stages are composed of feature fusion module, convolution normalization function, hollow space pyramid module and maximum pooling layer. The 1, 3 and 5 feature layers in the feature fusion module shared by the second, third, fourth and fifth stages are fused. The first layer feature layer is up-sampled and fused with the third layer feature layer after 1 1 convolution, and the first layer and the third layer fusion completed feature fusion layer is up-sampled and fused with the fifth layer feature layer after 1 1 convolution. The feature fusion layer has better perception ability for edge, texture and other detail information of insulator, and increases the detection efficiency when the shape of insulator is irregular. Finally, after the convolution normalization function, the improved hollow space pyramid module is added in the second, third, fourth and fifth stages and the spatial attention mechanism is fused, so that the network automatically obtains the weight of each feature channel and filters out the key information.

[0095] Distinctive technical features: fusion of deformable convolution, design of feature fusion structure, fusion of hollow space pyramid module and attention mechanism.

[0096] 3 Beneficial technical effects

[0097] The VSPENet insulator defect detection method provided by the embodiment of the application is based on the improved VoVNet network model, which increases two-dimensional offset in the convolution grid position, controls the convolution sampling position by changing the offset, realizes free deformation of the sampling grid, increases the sampling field of view, and can well extract insulator defect features. The feature fusion structure fully utilizes deep semantic information and shallow detail information, fully fuses the features extracted by the backbone network, and improves the detection ability of the network model for the insulator defect target of the power transmission line. The fusion of the hollow space pyramid module and the attention mechanism overcomes the problems of local information loss and lack of long-distance information caused by single hollow convolution grid effect, thereby improving the accuracy of model detection.

[0098] 4 Invention concept

[0099] The insulator damage and flashover defect have irregular shape, large difference in form and part of the shielding. In order to the normal operation of the power equipment in China, the detection accuracy of some existing mainstream network models for insulator defect target is not ideal. In order to solve the problem existing in target detection, the CenterNet target detection network without anchor is used, and the key points are used to find the center point of the target object. And a new CenterNet coding network is designed according to the characteristics of the specific detection target, so that it has certain real-time performance and is more suitable for deployment in mobile terminal equipment.

[0100] Research process:

[0101] Research example 1:

[0102] As Figure 3 shown, the purpose of the present application is to provide an insulator defect detection method based on improved lightweight CenterNet, which solves the problem of low detection accuracy of insulator damage, flashover and the like in complex environment in the prior art.

[0103] As Figure 6 shown, the existing CenterNet network structure diagram. The CenterNet network structure is composed of three parts: coding network, decoding network and detection network. The input image is first processed by the coding network, and the high-level semantic features are extracted from it, and the down-sampling processing is carried out at the same time. The high-level semantic features are deconvoluted by the decoding network. Then the feature map obtained is respectively through the center point prediction, the center point offset and the target size 3 branches. Finally, the target information is obtained by fusion to obtain the boundary box of the target.

[0104] In the target detection task, the coding network plays an important role in the model, mainly responsible for extracting the shallow and deep features of the input image, and outputting different scale features containing the semantic information of the original image. First, the convolution operation, batch normalization, maximum pooling and other methods are used to extract the features of the input image; then the obtained feature layer is stacked together through the residual structure composed of convolution layer module and identity module; the feature extraction is completed at the last piece, and the final feature layer is obtained. The pros and cons of the coding network will directly affect the detection effect of the model. The original coding network of CenterNet, Hourglass and DLA-34, has high computational complexity and more parameters, which is not conducive to the deployment of mobile terminal model. The detection accuracy of ResNet does not meet the detection requirements of power transmission lines. The present application uses the VoVNet coding network with good performance and high speed as the basis, and designs the VSPENet network structure, which improves the network accuracy on the premise of ensuring the network detection speed.

[0105] As Figure 7As shown, the VoVNet architecture employs a one-time aggregation module structure. It utilizes two connection methods in each convolutional layer of the one-time aggregation module: one connects each feature map to the last feature map, enhancing the correlation between feature maps, increasing the utilization of deep and shallow features, and avoiding gradient loss; the other connects to the next feature map to generate features with a larger receptive field, further improving detection accuracy. The VoVNet network consists of a start block with three convolutional layers and four stages of one-time aggregation modules. The main parameter reduction operation in the start block structure involves introducing a bottleneck layer in one branch to first reduce the number of channels before downsampling. Another branch performs max pooling on the original input before concatenation, aiming to pass some information from the input and ensure that the final result retains sufficient semantic information while reducing the number of parameters, without excessive information loss. VoVNet uses three 3×3 convolutional layers in the first stage, and one one-time aggregation module is placed in each of the second, third, fourth, and fifth stages. A 3×3 max pooling layer is used after each stage for connection and downsampling between the one-time aggregation modules.

[0106] The irregular shapes of damaged and flashover defects in power transmission line insulators, coupled with the use of ordinary convolution operations in the VoVNet network structure with fixed-size and fixed-shape geometry, result in different objects in the feature map having the same receptive field size.

[0107] In the first stage, the VSPENet of this invention uses three 3×3 deformable convolutional networks. Compared with standard convolution, the sampling grid of deformable convolution can be freely deformed.

[0108] like Figure 8a The image shows a data graph of a standard convolution.

[0109] like Figure 8b The image shows a data graph of deformable convolution.

[0110] like Figure 1 As shown, deformable convolution corresponds to a deformable convolutional network. The learning range of the convolution kernel is adaptively adjusted according to the shape of the target. A two-dimensional offset is added to the convolutional grid position. By changing the offset, the convolutional sampling position is controlled, allowing the sampling grid to deform freely, thus increasing the sampling field of view. The deformable convolution formula is shown in equation (1):

[0111] (1)

[0112] In equation (1), Location point Input feature map Features in;

[0113] For the position In the feature map The features in the output feature map are fused.

[0114] The number of sampling positions of the convolution kernel;

[0115] The weight of the first Sampling position;

[0116] The pre-specified offset of the first Sampling position;

[0117] The modulation scalar of the first Sampling point, which ranges from 0 to 1;

[0118] The learnable offset of the first Sampling position.

[0119] Shallow features usually contain detailed information and high-frequency information, while deep features contain more abstract and semantic information. Shallow features have better perception of the edges, textures and other detailed information of the target, while deep features have better perception of the overall shape and semantic information of the target. Since the original VoVNet model only uses the last layer feature map, it cannot fully capture the important information of shallow features, resulting in inaccurate detection and classification when dealing with targets with large morphological differences or occluded targets. The present application fuses the features of the first, third and fifth layers of the one-time aggregation module in the last four stages.

[0120] As Figure 9 The structure of the feature fusion module is shown. First, 1x1 convolution is used to reduce the dimension of the channel, keeping the number of channels consistent, and then a more simple and fast bilinear interpolation method is used for upsampling to obtain the feature, so that the high and low resolution feature maps remain consistent, improving the performance of the network.

[0121] The shape of the insulator defect is irregular, the morphological characteristics are different, and most of the defects are occluded. The empty space pyramid module overcomes the problem of local information loss and lack of long-distance information caused by the grid effect of single empty convolution. The calculation formula of the empty convolution is as follows:

[0122] (2)

[0123] In formula (2), The output of the empty convolution;

[0124] The input of the empty convolution;

[0125] for a convolution kernel of size ;

[0126] for different sampling levels of the convolution kernel.

[0127] The cavity spatial pyramid module first adopts a 1x1 convolution kernel channel dimension reduction. Then, in order to achieve the purpose of obtaining the global information of the image, the module also adopts 3 different dilated rate cavity convolution sampling multi-scale feature sampling, and at the same time adopts the global average pooling method to process the features. Finally, 1x1 convolution is used to fuse the features. The cavity spatial pyramid module increases the receptive field while obtaining multi-scale feature information, but it treats the results processed by different cavity convolution layers in each branch equally, and does not distinguish the importance of different channels after fusion, which may cause some important feature information to be ignored or confused, resulting in information loss. The application introduces a channel attention mechanism after the cavity spatial pyramid module, which enables the network to automatically obtain the weight of each feature channel, and filters out the key information.

[0128] The SE attention mechanism module mainly includes compression and excitation two parts. The mechanism first performs global average pooling compression operation on the input HxWxC feature map in the spatial dimension, and obtains the global feature of 1x1xC. The formula is as formula (3). Then, two fully connected layers and ReLU activation function are used to strengthen the connection between the obtained global feature channels, while reducing the calculation amount. And the learned channel is normalized through the Sigmoid activation function, and the calculation steps are as formula (4), finally the normalized feature is multiplied with the original feature, as formula (5).

[0129] (3)

[0130] In formula (3), is the output feature of the i-th feature after the average pooling layer; is the average pooling operation; is the input feature map; , are two dimensions of the feature map; is a feature map size variable.

[0131] (4)

[0132] In formula (4), is the channel weight output by the activation function; is the excitation operation; is the full connection layer; is the Sigmoid activation function; is the ReLU activation function.​

[0133] (5)

[0134] In formula (5), is a feature map for completing weight distribution; is a feature map weight distribution operation.

[0135] The SE attention mechanism module is integrated into the end of the dilated spatial pyramid module.

[0136] As shown in Figure 10 , the improved dilated spatial pyramid module. Specifically, the features extracted by the three 3x3 convolutions with different dilation rates are first fused by Concat connection to realize feature fusion, and then reduced in channel by 1x1 convolution; then the obtained features are input into the SE module to quickly extract more useful channel information; finally, they are fused with the 1x1 convolution and 3x3 convolution of the dilated spatial pyramid module to retain all feature information as much as possible.

[0137] Research Example 2:

[0138] As shown in Figure 4 , a large number of inspection videos are shot by a high-definition camera carried by a drone, and a large number of insulator damage and insulator flashover pictures are obtained from the inspection videos, including insulator defect pictures in multiple scenes such as sunny and cloudy days, mountains, forests, and cities.

[0139] As shown in Figure 11 , the insulator damage and flashover pictures.

[0140] As shown in Figure 12a to Figure 12d , the collected images are processed by random cropping, rotation, and brightness contrast methods, and the data set is expanded using the Mixup image enhancement method. The obtained insulator damage and flashover pictures after processing and the expanded pictures are used as a sample library of real insulator defects, and the first 20% are selected as the test set and the last 80% are selected as the training set.

[0141] As shown in Figure 13a , the insulator damage label map.

[0142] As shown in Figure 13b , the insulator flashover label map.

[0143] The image annotation tool labelImg was used to mark the areas of insulator damage and flashover, respectively, to determine the regions to be marked. The marked bounding boxes for these areas were then generated. Insulator damage and flashover were represented using "burst" and "flashover," respectively. After annotation, corresponding XML files were generated, containing important information such as image size, target object name, and the target object's location in the image.

[0144] A baseline network for the training set was constructed. The network parameters with the best performance after 200 iterations were selected to train the baseline network. The test set was then input into the baseline network to test and obtain the insulator damage and flashover test results, and the network performance was verified.

[0145] The evaluation metrics of this invention are average accuracy (AP) and mAP. The AP value for each category is calculated from the average cross-ratio threshold between the predicted and actual values. AP reflects the accuracy of the algorithm, and a higher value indicates better algorithm performance. The formula for calculating the AP value is shown in equation (10). mAP is obtained by averaging the average accuracy of all categories, as shown in equation (7).

[0146] (10)

[0147] (7)

[0148] In equation (10), P(r) is the value of the r-th term.

[0149] In equation (7), The number of categories of the target to be detected.

[0150] like Figure 14a The figure shows the mAP curve of CenterNet-VoVNet.

[0151] like Figure 14b The figure shows the mAP curve of CenterNet-VSPENet.

[0152] Comparing the mAP curves of CenterNet-VoVNet and CenterNet-VSPENet, we can see that as the number of training iterations increases, the mAP value of the improved CenterNet significantly improves, eventually stabilizing at around 87.7%. After approximately 150 epochs of training, the mAP value of the original model gradually stabilizes. Throughout the entire training process, the mAP value of the improved model is consistently higher than that of the original model.

[0153] Experimental results show that the improved model optimizes the network and increases model accuracy.

[0154] Research Example 3:

[0155] As shown in Figure 5 , the data set obtained in research example 2 is respectively input into the VoVNet network model and the VSPENet network model for training, and two network models are obtained to test the detection ability of insulator damage and flashover.

[0156] As shown in Figure 15a to Figure 15l , it is a data graph of the original network detection result.

[0157] As shown in Figure 16a to Figure 16l , it is a data graph of the improved network detection result.

[0158] Figure 15a to Figure 15l , Figure 16a to Figure 16l , the results generated by VoVNet network and VSPENet network for detecting the same data set.

[0159] As shown in Figure 15a , Figure 15b , Figure 15c , Figure 15d , Figure 15e , Figure 15f and Figure 16a , Figure 16b , Figure 16c , Figure 16d , Figure 16e , Figure 16f As shown in , it can be seen that the insulator flashover defect is easily affected by the shooting angle and light, resulting in unclear defect target. In the original model, detection will lead to missed detection, while the improved CenterNet algorithm has better detection accuracy for insulator flashover defects.

[0160] As shown in Figure 15a to Figure 15l , the shooting angle causes the flashover defect target features to be blocked, resulting in missed detection, and the improved model can well avoid these missed detection situations.

[0161] As shown in Figure 15h , due to the similarity between the insulator flashover defect target and the insulator itself, the original model missed detection.

[0162] As shown in Figure 15i , Figure 15j and Figure 16i , Figure 16j As shown in , under good detection conditions, the original algorithm can detect defective insulators, but the algorithm proposed in this paper has higher confidence than the original algorithm, and the prediction result is also more accurate. In the case of irregular insulator damage shape, the original algorithm has missed detection.

[0163] As shown in Figure 15k andFigure 16k As shown in FIG. 6, it can be seen that a small area of damage is misdetected as flashover by the original network structure, and the improved model does not have this misdetection.

[0164] As shown in FIG. 7, it can be seen that for a more serious insulator damage, the original model cannot learn the characteristics of such targets well, and there will be a missed detection, resulting in a decrease in detection accuracy. The improved model well improves this shortcoming. In the case of more serious damage and less obvious features, the accuracy is slightly reduced, but it can still be detected, avoiding the situation of missed detection. Figure 15l Figure 16l As shown in FIG. 7, it can be seen that for a more serious insulator damage, the original model cannot learn the characteristics of such targets well, and there will be a missed detection, resulting in a decrease in detection accuracy. The improved model well improves this shortcoming. In the case of more serious damage and less obvious features, the accuracy is slightly reduced, but it can still be detected, avoiding the situation of missed detection.

[0165] After the application runs for a period of time, the on-site technical personnel feedbacks the benefits:

[0166] The neural network comprises a VSPENet network, a decoding network and a detection network connected in sequence, the VSPENet network comprises first to fifth stage networks connected in sequence, the first stage network comprises a deformable convolution network and a max-pooling layer connected in sequence, the second stage network, the third stage network, the fourth stage network and the fifth stage network have the same structure, the second stage network comprises a feature fusion module, a convolution normalization function, a dilated spatial pyramid module and a max-pooling layer connected in sequence; the method comprises: dividing the network into five stages based on the VoVNet coding network to obtain an improved VSPENet network; and replacing the coding network in the existing CenterNet network with the VSPENet network to obtain an improved CenterNet network; the first to fifth stage networks and the like have good recognition ability for irregular insulator damage and flashover shapes of the power transmission line, the feature extraction process well utilizes shallow feature information, and the insulator small surface defect detection accuracy is high.​

Claims

1. A device for detecting insulator defects, the device being used to acquire an image of an insulator in the field, input it into a CenterNet network, and detect the insulator defects, the CenterNet network comprising an encoding network, a decoding network, and a detection network connected in sequence, characterized in that: The encoding network is a VSPENet network, which includes five sequentially connected first- to fifth-stage networks. The first-stage network includes a deformable convolutional network and a max-pooling layer connected in sequence. The second, third, fourth, and fifth-stage networks have the same structure. The second-stage network includes a feature fusion module, a convolutional normalization function, a dilated spatial pyramid module, and a max-pooling layer connected in sequence. The feature fusion module includes first to fifth feature layers, first to third fusion units, a first feature fusion layer, and a second feature fusion layer. The output of the first feature layer is connected to the input of the first fusion unit. Next, the output of the third feature layer is connected to the input of the first fusion layer. The output of the first fusion layer is connected to the convolution normalization function sequentially through the first feature fusion layer, the second fusion layer, the second feature fusion layer, and the third fusion layer. The output of the fifth feature layer is connected to the input of the second fusion layer. All outputs of the first to fifth feature layers are connected to the input of the third fusion layer. The output of the first feature fusion layer is connected to the output of the second feature fusion layer. The deformable convolutional network of the first stage network includes three 3×3 deformable convolutional networks, which are connected sequentially. The convolution normalization function includes a 1×1 convolution.

2. The device for detecting insulator defects according to claim 1, characterized in that: CenterNet, which includes VSPENet, is a network designed to improve network accuracy while ensuring network detection speed. VSPENet is a network designed to pass on some information from the input so that the final result has sufficient semantic information while reducing the number of parameters and without causing excessive information loss. Deformable convolutional networks are designed to expand the receptive field and improve the accuracy of insulator damage detection.

3. The device for detecting insulator defects according to claim 1, characterized in that: The feature fusion module is a one-time aggregation module. It is used to enhance the correlation between feature maps, increase the utilization rate of deep and shallow features, avoid gradient loss, generate features with a larger receptive field, improve detection accuracy, have better perception of the edge and texture details of insulators, and improve detection efficiency when the insulator shape is irregular.

4. The device for detecting insulator defects according to claim 1, characterized in that: The Hollow Space Pyramid module is formed by integrating the SE attention mechanism module into the end of the existing Hollow Space Pyramid module. It is used to obtain the weight of each feature channel in the existing Hollow Space Pyramid module, obtain more useful channel information, retain all feature information as much as possible, and reduce the amount of computation.

5. The apparatus for detecting insulator defects according to claim 1, characterized in that: The second-stage network consists of a feature fusion module, a convolutional normalization function, a dilated spatial pyramid module, and a max-pooling layer connected in sequence. The third-stage network consists of a feature fusion module, a convolutional normalization function, a dilated spatial pyramid module, and a max-pooling layer connected in sequence. The fourth-stage network consists of a feature fusion module, a convolutional normalization function, a dilated spatial pyramid module, and a max-pooling layer connected in sequence. The fifth-stage network consists of a feature fusion module, a convolutional normalization function, a dilated spatial pyramid module, and a max-pooling layer connected in sequence. The max-pooling layer of the first-stage network is connected to the feature fusion module of the second-stage network. The max-pooling layer of the second-stage network is connected to the feature fusion module of the third-stage network. The max-pooling layer of the third-stage network is connected to the feature fusion module of the fourth-stage network. The max-pooling layer of the fourth-stage network is connected to the feature fusion module of the fifth-stage network.

6. A method for obtaining an apparatus for obtaining the apparatus for detecting insulator defects as described in any one of claims 1 to 5, characterized in that: The process includes the following steps: based on the VoVNet coding network, the network is divided into five stages to obtain an improved VSPENet network; the VSPENet network is then used to replace the coding network in the existing CenterNet network to obtain the improved CenterNet network.

7. The method for obtaining the apparatus according to claim 6, characterized in that: It also includes the following steps: connecting each feature layer in a feature fusion module together to enhance the correlation between feature maps, increase the utilization rate of deep and shallow features, avoid gradient loss, and fusing the first, third and fifth feature layers in a feature fusion module to generate features with a larger receptive field, improve detection accuracy, and obtain a one-time aggregation module.

8. The method for obtaining the apparatus according to claim 6, characterized in that: It also includes the following steps: integrating the SE attention mechanism module into the end of the existing hollow space pyramid module to obtain an improved hollow space pyramid module, which is used to obtain the weight of each feature channel in the existing hollow space pyramid module, obtain more useful channel information, retain all feature information as much as possible, and reduce the amount of computation.

Images