Power transmission line connection fitting detection method and device, medium, equipment and product

Abstract

The invention relates to a power transmission line connection fitting detection method and device, a medium, equipment and a product, and relates to the technical field of deep learning target detection. The method comprises the steps of determining a sample data set of a power transmission line connection fitting according to an unmanned aerial vehicle inspection video; carrying out defogging processing on the sample data set through a dark channel prior defogging algorithm to obtain a first-stage long-shot image data set; intercepting an image in the first-stage long-shot image data set according to a target detection prediction frame of the image in the first-stage long-shot image data set to obtain a second-stage connection hardware fitting data set; building a power transmission line connection fitting detection network based on a double attention YOLOv5 algorithm based on the two-stage connection fitting data set, and performing network training on the power transmission line connection fitting detection network; and carrying out power transmission line connection fitting detection through the power transmission line connection fitting detection network, and outputting a detection result. The method can improve the detection accuracy and detection speed of the power transmission line connection fitting.

Description

Technical Field

[0001] The present disclosure relates to the technical field of deep learning target detection, and in particular, to a method, device, medium, equipment and product for detecting power transmission line connection fittings. Background Art

[0002] The power industry is an important pillar of the national economy. With the continuous expansion of the scale of the power grid, the number of power grid equipment built in complex environments is also increasing day by day, which puts higher requirements on the safe maintenance of power system equipment. Transmission line connection fittings have the characteristics of complex structure and small and difficult to detect parts. They are exposed to complex natural environments all year round and will rust due to factors such as haze, rain, snow, light, and wind erosion. Severe rust failures will cause parts to fall off or fail, bringing safety hazards to the stability of transmission line operation. It is difficult for drones and line patrol robots to capture high-quality real-time images, which creates great difficulties for feature extraction in later image processing. Therefore, using appropriate detection methods to obtain real-time status information for connection fittings is helpful for transmission line fault diagnosis and timely warning, which is of great significance for realizing intelligent detection and real-time fault diagnosis of transmission line connection fittings. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, medium, equipment and product for detecting power transmission line connection fittings.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for detecting power transmission line connection fittings is provided, comprising: Determine the sample data set of power transmission line connection fittings based on drone inspection videos; Defogging the sample data set by using a dark channel prior defogging algorithm to obtain a first-stage distant view image data set; The images in the first-stage distant image dataset are intercepted according to the target detection prediction frames of the images in the first-stage distant image dataset to obtain a second-stage connection hardware dataset; Building a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm based on the two-stage connection fittings dataset and performing network training on the power transmission line connection fittings detection network; The power transmission line connection fittings detection network is used to perform power transmission line connection fittings detection, and the detection results are output.

[0005] The defogging process is performed on the sample data set by using a dark channel prior defogging algorithm to obtain a stage distant image data set, including: Optionally, a dark channel of the first image is calculated by an image downsampling method to obtain a dark channel image corresponding to the first image; the first image is any image in the sample data set; determining the ambient atmospheric light of the first image according to the dark channel image; Determine a transmittance image corresponding to the first image according to the dark channel image and the ambient atmospheric light; The first image is repaired by using the transmittance image based on an atmospheric scattering model to obtain a stage distant view image corresponding to the first image.

[0006] Optionally, after determining the transmittance image corresponding to the first image according to the dark channel image and the ambient atmospheric light, the method further includes: The transmittance image is optimized based on boundary condition constraints of the radiation cube criterion.

[0007] Optionally, building a power transmission line connection fittings detection network based on a dual attention YOLOv5 algorithm based on the two-stage connection fittings dataset and performing network training on the power transmission line connection fittings detection network includes: The YOLOv5 algorithm is used as the baseline network of the transmission line connection fittings detection network, and the self-attention mechanism and the spatial channel attention mechanism are integrated in the backbone network to improve feature enhancement and small target visibility; Inserting a Ghost module into the backbone network to perform model lightweight processing; The BiFPN structure is used to enhance the fusion effect of the baseline network on multi-scale features; The transmission line connection fittings detection network is trained based on the second-stage connection fittings data set.

[0008] Optionally, after the YOLOv5 algorithm is used as the baseline network of the power transmission line connection fittings detection network, and the self-attention mechanism and the spatial channel attention mechanism are integrated in the backbone network for feature enhancement and small target visualization improvement, the method further includes: SPPF is added as a spatial pyramid pooling layer after the self-attention mechanism to enhance the recognition ability of the transmission line connection fittings detection network for targets of different sizes.

[0009] Optionally, the encoder of the self-attention mechanism is obtained by stacking the MLP layer and the MSA layer n times, wherein each stacking includes one MLP layer and one MSA layer.

[0010] Optionally, the Ghost module includes multiple stacked Ghost convolutions.

[0011] Optionally, PANet is used as a feature fusion network in the BiFPN structure.

[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a transmission line connection fittings detection device, comprising: An acquisition module is used to determine a sample data set of transmission line connection fittings based on drone inspection videos; A first processing module is used to perform defogging processing on the sample data set by using a dark channel prior defogging algorithm to obtain a stage distant image data set; A second processing module is used to intercept the images in the first-stage distant image dataset according to the target detection prediction frame of the images in the first-stage distant image dataset to obtain a second-stage connection hardware dataset; A network construction module, used to build a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm based on the two-stage connection fittings dataset and perform network training on the power transmission line connection fittings detection network; The output module is used to detect the power transmission line connection fittings through the power transmission line connection fittings detection network and output the detection results.

[0013] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the power transmission line connection fittings detection method provided in the first aspect of the present disclosure are implemented.

[0014] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the power transmission line connection fittings detection method provided in the first aspect of the present disclosure.

[0015] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method for detecting power transmission line connection fittings provided in the first aspect of the present disclosure.

[0016] Through the above technical scheme, a sample data set of power line connection fittings is determined based on drone inspection videos; the sample data set is defogged by a dark channel priori defogging algorithm to obtain a first-stage distant image data set; the images in the first-stage distant image data set are intercepted according to the target detection prediction frame of the images in the first-stage distant image data set to obtain a second-stage connection fittings data set; based on the second-stage connection fittings data set, a power line connection fittings detection network based on the dual attention YOLOv5 algorithm is built and the network training of the power line connection fittings detection network is performed; the power line connection fittings detection is performed through the power line connection fittings detection network, and the detection results are output. This method can improve the accuracy and speed of power line connection fittings detection.

[0017] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The present invention is a flow chart showing a method for detecting connection fittings of a power transmission line according to an exemplary embodiment.

[0019] Figure 2 The present invention is a flow chart showing a method for detecting connection fittings of a power transmission line according to an exemplary embodiment.

[0020] Figure 3 The present invention is a flow chart showing a method for detecting connection fittings of a power transmission line according to an exemplary embodiment.

[0021] Figure 4 It is a dual attention YOLOv5 algorithm network structure diagram.

[0022] Figure 5 It is a schematic diagram of the self-attention mechanism structure.

[0023] Figure 6 It is a structural diagram of the convolution process of the Ghost module.

[0024] Figure 7 It is the coordinate transformation relationship diagram of the anchor box bounding box.

[0025] Figure 8 The figure is a schematic diagram showing a device for detecting connection fittings of a power transmission line according to an exemplary embodiment. Fig. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0027] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0028] Figure 1 FIG. 1 is a flow chart showing a method for detecting power transmission line connection fittings according to an exemplary embodiment. Figure 1 As shown, the method comprises the following steps: In step S11, a sample data set of transmission line connection fittings is determined based on the drone inspection video.

[0029] Exemplarily, a drone inspection video of a transmission line is obtained, and a single-frame on-site image of a transmission line connection fitting is obtained by cropping the inspection video. Each single-frame on-site image can be annotated using an annotation tool, thereby obtaining a corresponding label and taking the set of annotated single-frame on-site images as a sample data set. The present disclosure does not limit the type of the connection fitting. For example, the connection fitting can include at least one of: a U-Shackle (US), a triangular yoke plate (TYP), an adjustment plate (ADP), a tension clamp (TEC), a clamp bolt (CLB), and an insulator (In). In addition, the annotation process can determine the actual position of the connection fitting components in the single-frame on-site image through Labelmg software, update the annotation frame according to the target position and size information of each fitting, determine the label information for different categories of fittings, generate the corresponding label file in the xml format according to the annotated single-frame on-site image, and make a sample data set of the transmission line connection fittings according to the label file.

[0030] In addition, after step S11, the number of samples in the sample data set may be small. Therefore, the sample data set can be expanded by using methods such as image flipping, scaling, mirroring, color enhancement, adding noise and MixUp, and the expanded data set is divided into a training set and a test set. 80% of the images in the data set are randomly selected as the training set and 20% as the test set to train and test the model.

[0031] Among them, MixUp improves the generalization performance of the model by generating a regularizer, and uses linear interpolation to construct new training samples to mix images of different categories for data enhancement. The working principle of MixUp is calculated as follows:

[0032] in, and The training sample corresponds to The mixing coefficient distribution; in addition, the data enhancement method can make the sample data set more representative and universal, which is conducive to improving the detection accuracy of subsequent network models and helping to improve the generalization performance of the model.

[0033] In step S12, the sample data set is defogged using a dark channel priori defogging algorithm to obtain a stage distant image data set.

[0034] For example, the dark channel prior defogging algorithm can be used to improve the visual quality of images, especially in outdoor environments where the visibility of images is reduced due to suspended particles in the atmosphere (such as fog, smoke, dust, etc.). The algorithm can improve image visibility, restore color authenticity, enhance image contrast, etc. by estimating the fog component in the image and removing it. Therefore, the dark channel prior defogging algorithm can be used to defog the data in the sample dataset to obtain a stage distant image dataset.

[0035] In step S13, the images in the first-stage distant image dataset are intercepted according to the target detection prediction frames of the images in the first-stage distant image dataset to obtain a second-stage connection fittings dataset.

[0036] For example, the target detection prediction frame of the image in the first-stage perspective image dataset of the power transmission line connection fittings can be obtained, and the image can be intercepted according to the prediction frame, and the intercepted image can be used as the image in the second-stage connection fittings dataset. In addition, the intercepted image can be used to determine the real position of the connection fitting parts through Labellmg software, update the annotation frame according to the target position and size information of each fitting, determine the label information for different categories of fittings, generate the corresponding xml format label file according to the labeled intercepted image, and produce the second-stage connection fittings dataset of the power transmission line connection fittings according to the label file.

[0037] In step S14, a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm is built based on the two-stage connection fittings dataset and network training is performed on the power transmission line connection fittings detection network.

[0038] For example, the input sample size of the YOLOv5 network is 640×640. The preprocessing process performs adaptive padding on the sample. The training process calculates the predicted box through anchors and obtains the difference between the predicted box and the labeled box for reverse update operation. The backbone network is composed of the Focus module and the CSP structure. The input image is sliced ​​by the Focus module, and the input feature map is changed from 640×640×3 to 320×320×12. Then, a 320×320×32 feature map is obtained through 32 convolution kernels. Finally, the feature maps of different scales are connected through the spatial pyramid structure. The neck part of the model uses the FPN+PAN structure to fuse different feature layers. FPN (Feature Pyramid Network) conveys strong semantic information from top to bottom, and PAN (Pyramid Attention Network) conveys strong position information from bottom to top. In the prediction end of YOLOv5, GIOU_Loss is used as the loss function of the bounding box, and the cross entropy loss is used as the category loss function.

[0039] Taking the YOLOv5 network as the baseline network, a dual-attention power line connection fittings detection network is built. In the backbone network part, the self-attention mechanism (Vision Transformer, ViT) and the spatial channel attention mechanism CBAM can be integrated as an improved method for feature enhancement and small target visualization. The Ghost module is inserted to replace the traditional convolution for model lightweight processing, and the BiFPN (weighted bidirectional feature pyramid network) module is used to enhance the fusion effect of the original network on multi-scale features.

[0040] In step S15, the power transmission line connection fittings detection is performed through the power transmission line connection fittings detection network, and the detection result is output.

[0041] Through the above technical scheme, a sample data set of power line connection fittings is determined based on drone inspection videos; the sample data set is defogged by a dark channel priori defogging algorithm to obtain a first-stage distant image data set; the images in the first-stage distant image data set are intercepted according to the target detection prediction frame of the images in the first-stage distant image data set to obtain a second-stage connection fittings data set; based on the second-stage connection fittings data set, a power line connection fittings detection network based on the dual attention YOLOv5 algorithm is built and the network training of the power line connection fittings detection network is performed; the power line connection fittings detection is performed through the power line connection fittings detection network, and the detection results are output. This method can improve the accuracy and speed of power line connection fittings detection.

[0042] Figure 2FIG. 1 is a flow chart showing a method for detecting power transmission line connection fittings according to an exemplary embodiment. Figure 2 As shown, step S12 includes the following steps: In step S121, a dark channel of a first image is calculated by an image downsampling method to obtain a dark channel image corresponding to the first image; the first image is any image in the sample data set.

[0043] In step S122, the ambient atmospheric light of the first image is determined according to the dark channel image.

[0044] In step S123, a transmittance image corresponding to the first image is determined according to the dark channel image and the ambient atmospheric light.

[0045] In step S124, the first image is repaired by using the transmittance image based on the atmospheric scattering model to obtain a stage distant view image corresponding to the first image.

[0046] For example, we first obtain an atmospheric physical model of light scattering through suspended particles in the air, and find the maximum value of the pixel set of each channel of the RGB image to estimate the atmospheric light. We obtain the transmittance through the color attenuation prior and scene depth estimation method, and the calculation formula is as follows based on the conversion relationship between the RGB and HSV color space models: (1) in is the saturation channel, , represents a three-dimensional vector in RGB color space, is the color channel index, transmittance The expression of is shown in the following formula (2): (2) in To estimate the parameters, which vary with the image selection, the following calculation formula (3) is further used: (3) in represents ambient light, is the image removal rate ( ). The original dark channel prior knowledge calculates the dark channel value for each pixel, which increases the computational time for large-size images. The dark channel can be calculated by image downsampling (using 1×1 patch instead of 15×15 patch), and then restored to its original size using interpolation. Therefore, equation (3) can be refined into equation (5) through equation (4): (4) (5) in It is direct light The adjustment parameters of is the entire image, where .

[0047] Using downsampling in the transmittance calculation process will lose some edge information, so optionally, after step S123, the method may further include: optimizing the transmittance image based on boundary condition constraints of the radiation cube criterion.

[0048] For example, the image boundary can be restricted and smoothed by boundary constraints and Gaussian homomorphic filtering. The boundary restriction based on the radiation cube criterion can reduce the complexity of dark channel calculation and avoid the occurrence of staircase phenomenon. Assume that the image boundary is , through the atmospheric scattering model, the boundary condition of the transmittance image can be obtained as , is the boundary limit of the transmittance image. The solution to the boundary limit is calculated as follows: (6) in, and are the upper and lower boundaries of the scene image respectively. The observed intensity is constrained by the boundary condition. and scene brightness By constraining the image within a certain range, the scene scattering problem of the image is alleviated, and an equivalent dark channel image with the staircase effect removed is obtained.

[0049] After determining the boundary conditions, the improved Canny operator edge detection and Otsu (Otsu algorithm, an algorithm for determining the image binarization threshold) transmittance image segmentation can be used to obtain the positioning compensation image of the boundary area and the internal area, and then the edge information of the transmittance image is enhanced and the internal information is smoothed by Gaussian homomorphic filtering. The expression (7) is as follows: (7) in is the transfer function of the Gaussian filter, is the gain coefficient of high and low frequency, is the sharpening constant, Indicates frequency The distance to the filter center, is the cutoff frequency. After Gaussian homomorphic filtering, the edge area where the depth of field changes suddenly can be significantly enhanced.

[0050] use The function corrects the grayscale histogram envelope curve of the transmittance image to solve the halo phenomenon caused by the high brightness area after dark channel processing. The calculation formula (8) of the correction function is as follows: (8) in is the amplitude proportionality coefficient, is the histogram curve to be fitted, is the correction index.

[0051] Finally, the haze-free image is restored through the atmospheric scattering model The dark image is adjusted by the tone adjustment function, and the calculation formula (9) is as follows: (9) in, is the output function after hue adjustment; is the maximum brightness value; is a bias parameter used to adjust the detail information of dark areas; for The maximum pixel value.

[0052] Figure 3 FIG. 1 is a flow chart showing a method for detecting power transmission line connection fittings according to an exemplary embodiment. Figure 3 As shown, step S14 includes the following steps: In step S141, the YOLOv5 algorithm is used as the baseline network of the power transmission line connection fittings detection network, and the self-attention mechanism and the spatial channel attention mechanism are integrated in the backbone network to improve feature enhancement and small target visibility; For example, the transmission line connection fittings detection network is as follows: Figure 4 As shown, the specific process can refer to steps S141 to S144.

[0053] In the main part, the integration of self-attention mechanism ViT and spatial channel attention mechanism CBAM can be used as an improved method for feature enhancement and small target visualization; spatial channel attention mechanism CBAM can capture the detailed information in the feature map and characterize the fine-grained information of the image from the channel and spatial levels. The working principle of CBAM is to The attention map is inferred along the 2D dimension (channel and space) and optimized with its corresponding feature map. The channel information attention mechanism calculation formula (10) is as follows: (10) in, is the sigmoid nonlinear activation function, and , and Represents the hidden weights and output weights in the MLP (Multilayer Perceptron) layer, respectively, and their inputs use shared weights and , and They represent the use of average pooling and maximum pooling to generate the feature maps above the corresponding space, For the reduction rate.

[0054] In order to compensate for the deviation of channel attention in position information, a spatial attention module can be introduced, and its calculation formula (11) is as follows: (11) in, is a 7×7 convolution operation, and They represent the global average pooling features and maximum pooling features on the channel respectively.

[0055] The self-attention mechanism ViT can divide the input image into a certain number of patches as sequence information and pass it to the word embedding and position encoding modules. The model structure diagram is as follows: Figure 5 The Transformer-based encoding structure is a 1D vector that embeds the sequence input, which solves the problem that the global features of the existing neural network are difficult to obtain. The 1D position embedding that can be learned through the linear layer can effectively retain the image position information. Performing global self-attention on the entity object introduces The computational complexity and memory usage of Transformer are low, and the linear computational complexity of Transformer is conducive to calculating the saliency of image visual information.

[0056] Optionally, the encoder of the self-attention mechanism is obtained by stacking the MLP layer and the MSA layer n times, wherein each stack includes one of the MLP layer and one of the MSA (Multi-Head Self-Attention) layer.

[0057] For example, the encoder of the self-attention mechanism ViT can be stacked by two independent sub-layers: MLP layer and MSA layer In each layer Perform the following operation: Input The obtained (Q, K, V) triplet is used as the input of self-attention, and the calculation formula (12) is as follows: (12) in, are the weights of three related linear mapping vectors; Then the process of self-attention can be expressed as formula (13): (13) in, is the dimension of the feature vector, The self-attention modules are connected in series to form a multi-head self-attention mechanism and serve as the output: .

[0058] in, , The value is , the output of the MSA module plus the residual connection structure are used as the input of the MLP layer.

[0059] Optionally, SPPF (Spatial Pyramid Pooling – Fast) is added after the self-attention mechanism as a spatial pyramid pooling layer to enhance the recognition capability of the transmission line connection fittings detection network for targets of different sizes.

[0060] For example, SPPF is introduced as a spatial pyramid pooling layer after the self-attention mechanism module ViT to improve the multi-scale information fusion capability of the model by converting input features of different sizes into vector information of a specific dimension. The main structure of SPPF is composed of convolution and maximum pooling.

[0061] In step S142, a Ghost module is inserted into the backbone network to perform model lightweight processing; For example, the lightweight module Ghost and the convolution structure stack are used to construct a deep neural network in the backbone network of the model, which can be used to remove unnecessary redundant parameters and calculations in the backbone network of the model. Its structure is similar to the residual connection in the residual network. The residual connection structure is composed of the Ghost module, BN and ReLU activation function units, which can improve the calculation efficiency while ensuring the depth of the network. Among them, the Ghost module is as follows: Figure 6 As shown in the figure, the Ghost module can be composed of a stack of Ghost convolutions. A small number of feature maps are generated from regular convolutions through smaller filters, and then a series of linear operations are used to generate new feature maps similar to them. The combination of the two sets of feature maps is the output of the model.

[0062] If the input feature map size is , the corresponding output size is , the convolution kernel size is , after convolution operation produce The parameter quantity is then transformed linearly Generate the Ghost feature map, and the calculation formula (14) is as follows: (14) Among them, m is the number of channels of the feature map, s is the number of linear transformations, and n is the number of generated feature maps. It represents the number of effective linear transformations under the identity mapping. The output result is the concatenation of the original feature map and the Ghost feature map. When using Ghost convolution, the amount of model calculation is greatly reduced compared to the original conventional convolution through cheap linear operations, thereby significantly improving the model reasoning speed of the multi-target detection task of transmission line connection fittings.

[0063] In summary, inserting the self-attention mechanism module ViT module after the Ghost module of the backbone network can enhance the global receptive field of the model on the feature map and capture richer strong semantic information for subsequent input. In step S143, the fusion effect of the baseline network on multi-scale features is enhanced by using the BiFPN structure; For example, the BiFPN structure adds an extra edge between the input node and the output node to ensure cross-scale calculation without adding additional parameters and computation. In order to avoid the performance degradation caused by simple feature concatenation, BiFPN introduces a set of learnable weight factors to characterize the weight ratio of different input features.

[0064] Optionally, PANet is used as a feature fusion network in the BiFPN structure.

[0065] For example, PANet aggregates multi-scale feature information by adding, and the aggregation calculation process of the nth layer of the model is as follows (15): (15) (15) in, The sampling operation that matches the corresponding resolution, represents the convolution operation for processing features, Indicates the number of The first layer of BiFPN is represented by learnable weights and bidirectional cross-scale normalized feature fusion. The layer aggregation calculation process is as follows (16): (16) (16) Among them, each level of channel layer The corresponding weight factor Multiply to get the optimal feature scale representation, and then divide by the weighted sum of each scale information to fit the original input, is the adjustment factor, which is a variable parameter. After the feature information is fused, the feature map is mapped to the sampling path through convolution.

[0066] In step S144, network training is performed on the power transmission line connection fittings detection network based on the second-stage connection fittings data set.

[0067] For example, in the prediction end of the model, The value of measures the distance from all cluster sample points to the cluster center. The coordinate transformation relationship diagram of the anchor box bounding box is as follows: Figure 7 As shown in the figure, three prediction heads of different scales are generated to perform dense prediction of multi-scale information on three levels. The improved K-means algorithm obtains 9 anchors of different sizes through clustering, where three anchors of similar sizes are grouped together to predict target information of large, medium, and small scales respectively. For the sigmoid operation, by compression arrive interval to enhance the positioning accuracy of the bounding box, is the prediction scale, is the confidence level, is the feature map grid coordinate, is the scaling factor, Map width and height for anchors, It can be regarded as a learning prediction target. The center of the bounding box in the figure is placed in the second row and second column of the grid. The predicted bounding boxes with low scores are filtered out by the confidence threshold and the predicted box is obtained by non-maximum suppression (NMS). In order to reduce the deviation caused by anchor clustering, we use The value of measures the distance from all cluster sample points to the cluster center, replacing the Euclidean distance in the original algorithm. The calculation formula (17) of the new distance measurement function is as follows: (17) Among them, boxes are annotation boxes, and anchors are formed by cluster centers. It is the intersection-over-union ratio of the annotation box and the cluster center. The larger the IOU between the bounding box and the corresponding cluster center (anchor), the closer the metric distance. Using IOU as the metric standard, a more suitable bounding box can be found through 9 anchors. The anchors calculated by the K-means clustering algorithm are randomly mutated in width and height through the genetic algorithm. The anchors are optimized iteratively multiple times to better adapt to the target scale.

[0068] In addition, the constructed transmission line connection fittings detection network can also be trained and tested, the current number of iterations is defined as A, and A=0 is initialized; the maximum number of iterations is Amax, and the specific value of Amax is determined by the generalization performance obtained by applying the model to the data set. The critical number of iterations at which the model does not overfit on the data set is the optimal number, and the parameters of each layer in the transmission line connection fittings detection network are initialized for parameter adjustment.

[0069] For the test training effect, the optimal weight file generated in the A-time iteration result can be selected as the test weight, and the test set can be input into the transmission line connection fittings detection network for testing and obtaining the prediction results.

[0070] For the network accuracy evaluation standard, the classification accuracy P, recall R, and macro-F1 are statistically analyzed respectively. Macro-F1 combines the harmonic mean of P and R to improve precision and Recall while minimizing the difference between the two. , the calculation formulas of P, R, and macro-F1 are as follows: (18) (19) in, To predict the number of correct positive samples; is the number of positive samples predicted incorrectly; is the number of negative samples predicted correctly; is the number of negative class samples predicted incorrectly. Combining the true value and predicted value data of each defect in the confusion matrix, the classification accuracy, recall rate and value.

[0071] It is not universal to measure the accuracy of a model only by Precision or Recall. Therefore, the average AP value of multiple validation sets can be calculated by the average accuracy mAP. mAP is determined by the area enclosed by the Precision-Recall curve and the horizontal and vertical axes. n is the actual number of groups calculated for AP, and the approximate calculation formula for the difference (20) is as follows: (20) In summary, the present invention aims to solve the problem of insufficient sample data sets by expanding the data sets by methods such as image inversion, translation, scaling, contrast adjustment, and MixUp. The YOLOv5 network is improved to solve the difficulties of complex detection environments, diverse object sizes, and rich detail features, and the feature extraction effect of the model is improved by designing a dual-attention YOLOv5 algorithm. The self-attention mechanism ViT and the spatial channel attention mechanism CBAM are integrated into the backbone network as an improved method for feature enhancement and small target visualization. The Ghost module is inserted to replace the traditional convolution for model lightweight processing, and the BiFPN module is used to enhance the fusion effect of the original network on multi-scale features, so that the model can obtain higher detection accuracy on the test set.

[0072] In order to comprehensively evaluate the effect of the model, this paper uses several different attention mechanism modules and model compression networks as replacement components of the YOLOv5 model backbone, and verifies the obtained results. As shown in Tables 1 and 2, the following parameters are compared: Params, FLOPs, AR, mAP @ 0.5: 0.95, FPS, and the best results are shown in bold.

[0073] In Tables 1 and 2, the first two columns list the names of various models and their corresponding backbone structures, and columns 3 to 7 are the performance evaluation indicators of various models. The results of the two-stage independent detection are shown in Tables 1 and 2.

[0074] Table 1

[0075] Table 2

[0076] It can be seen from Tables 1 and 2 that CBAM achieved the best result of 0.995 in the replacement results of the attention mechanism module, and the Ghost module achieved the best result of 0.994 in the model compression replacement results. This method performed best among all backbone replacement results, reaching 0.996, which is 0.3 percentage points higher than the baseline model.

[0077] In the second detection phase, the AR score of the CBAM in the replacement backbone network achieved the best result of 0.989 in their respective groups, and our method achieved 0.994, which was 0.8 percentage points higher than the baseline model. Taking mAP @ 0.5: 0.95 as the average accuracy index, the CBAM and Ghost module replacement components achieved the best scores of 0.9425 and 0.9187 in their respective groups, respectively, and our method achieved 0.9480, which was 4.6 percentage points higher than the baseline model, which was the best level. In the scoring of the second phase test, the Ghost module achieved the best result of 0.8624 in its group, while our method still achieved the highest overall score of 0.8674, which was 4.3 percentage points higher than the baseline model. Replacing the Ghost module component in m AP @ 0.5: 0.95 has more advantages in comprehensive effects, with a floating point calculation of 42.3 G per second, which is better than the baseline model and other lightweight models. Although the introduction of the Transformer module leads to excessive computation and too many model parameters, this method improves the performance of various precision parameters and verifies the potential application value of Vision Transformer in visual tasks such as target detection, which may be better than the traditional mainstream CNN model. This fully demonstrates that the power transmission line connection fittings detection network provided by the present disclosure can not only enhance the global receptive field of the model, but also capture local significant features and solve the problem of difficulty in detecting small targets and overlapping targets in complex environments.

[0078] Through the above technical scheme, a sample data set of power line connection fittings is determined based on drone inspection videos; the sample data set is defogged by a dark channel priori defogging algorithm to obtain a first-stage distant image data set; the images in the first-stage distant image data set are intercepted according to the target detection prediction frame of the images in the first-stage distant image data set to obtain a second-stage connection fittings data set; based on the second-stage connection fittings data set, a power line connection fittings detection network based on the dual attention YOLOv5 algorithm is built and the network training of the power line connection fittings detection network is performed; the power line connection fittings detection is performed through the power line connection fittings detection network, and the detection results are output. This method can improve the accuracy and speed of power line connection fittings detection.

[0079] Figure 8 8 is a schematic diagram of a transmission line connection fittings detection device 800 according to an exemplary embodiment. The device 800 includes: an acquisition module 810, a first processing module 820, a second processing module 830, a network construction module 840 and an output module 850; The acquisition module 810 is used to determine a sample data set of transmission line connection fittings based on the drone inspection video; The first processing module 820 is used to perform defogging on the sample data set by using a dark channel prior defogging algorithm to obtain a stage distant image data set; The second processing module 830 is used to intercept the images in the first-stage distant image data set according to the target detection prediction frame of the images in the first-stage distant image data set to obtain a second-stage connection hardware data set; The network construction module 840 is used to build a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm based on the two-stage connection fittings data set and perform network training on the power transmission line connection fittings detection network; The output module 850 is used to perform power transmission line connection fittings detection through the power transmission line connection fittings detection network and output the detection results.

[0080] Optionally, the first processing module 820 is used to: Calculate the dark channel of the first image by an image downsampling method to obtain a dark channel image corresponding to the first image; the first image is any image in the sample data set; Determine the ambient atmospheric light of the first image according to the dark channel image; Determine a transmittance image corresponding to the first image according to the dark channel image and the ambient atmospheric light; The first image is repaired by using the transmittance image based on an atmospheric scattering model to obtain a stage distant view image corresponding to the first image.

[0081] Optionally, the first processing module 820 is further configured to optimize the transmittance image based on boundary condition constraints of a radiation cube criterion.

[0082] Optionally, the network construction module 840 is used to: The YOLOv5 algorithm is used as the baseline network of the transmission line connection fittings detection network. The self-attention mechanism and spatial channel attention mechanism are integrated into the backbone network to improve feature enhancement and small target visibility. Insert the Ghost module into the backbone network to perform model lightweight processing; The BiFPN structure is used to enhance the fusion effect of the baseline network on multi-scale features; The transmission line connection fittings detection network is trained based on the two-stage connection fittings data set.

[0083] Optionally, the network construction module 840 is further used to: SPPF is added as a spatial pyramid pooling layer after the self-attention mechanism to enhance the recognition ability of the transmission line connection fittings detection network for targets of different sizes.

[0084] Optionally, the encoder of the self-attention mechanism is obtained by stacking the MLP layer and the MSA layer n times, wherein each stacking includes one MLP layer and one MSA layer.

[0085] Optionally, the Ghost module includes multiple stacked Ghost convolutions.

[0086] Optionally, PANet is used as a feature fusion network in the BiFPN structure.

[0087] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0088] Through the above technical scheme, a sample data set of power line connection fittings is determined based on drone inspection videos; the sample data set is defogged by a dark channel priori defogging algorithm to obtain a first-stage distant image data set; the images in the first-stage distant image data set are intercepted according to the target detection prediction frame of the images in the first-stage distant image data set to obtain a second-stage connection fittings data set; based on the second-stage connection fittings data set, a power line connection fittings detection network based on the dual attention YOLOv5 algorithm is built and the network training of the power line connection fittings detection network is performed; the power line connection fittings detection is performed through the power line connection fittings detection network, and the detection results are output. This method can improve the accuracy and speed of power line connection fittings detection.

[0089] Fig. 9 FIG. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment. Fig. 9 As shown, the electronic device 900 may include: a processor 901 , a memory 902 . The electronic device 900 may also include one or more of a multimedia component 903 , an input / output (I / O) interface 904 , and a communication component 905 .

[0090] The processor 901 is used to control the overall operation of the electronic device 900 to complete all or part of the steps in the above-mentioned power transmission line connection fittings detection method. The memory 902 is used to store various types of data to support the operation of the electronic device 900, and these data may include, for example, instructions for any application or method used to operate on the electronic device 900, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 903 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 902 or sent through the communication component 905. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 904 provides an interface between the processor 901 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 905 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0091] In an exemplary embodiment, the electronic device 900 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned transmission line connection fittings detection method.

[0092] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned power transmission line connection fitting detection method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 902 including program instructions, and the above-mentioned program instructions can be executed by the processor 901 of the electronic device 900 to complete the above-mentioned power transmission line connection fitting detection method.

[0093] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned power transmission line connection fittings detection method when executed by the programmable device.

[0094] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0095] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0096] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for detecting power transmission line connection fittings, characterized in that: include: Determine the sample data set of power transmission line connection fittings based on drone inspection videos; Defogging the sample data set by using a dark channel prior defogging algorithm to obtain a first-stage distant view image data set; The images in the first-stage distant image dataset are intercepted according to the target detection prediction frames of the images in the first-stage distant image dataset to obtain a second-stage connection hardware dataset; Building a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm based on the two-stage connection fittings dataset and performing network training on the power transmission line connection fittings detection network; The power transmission line connection fittings detection network is used to perform power transmission line connection fittings detection, and the detection results are output.

2. The method according to claim 1, characterized in that: The defogging process is performed on the sample data set by using a dark channel prior defogging algorithm to obtain a stage distant image data set, including: Calculate the dark channel of the first image by an image downsampling method to obtain a dark channel image corresponding to the first image; the first image is any image in the sample data set; determining the ambient atmospheric light of the first image according to the dark channel image; Determine a transmittance image corresponding to the first image according to the dark channel image and the ambient atmospheric light; The first image is repaired by using the transmittance image based on an atmospheric scattering model to obtain a stage distant view image corresponding to the first image.

3. The method according to claim 2, characterized in that After determining the transmittance image corresponding to the first image according to the dark channel image and the ambient atmospheric light, the method further includes: The transmittance image is optimized based on boundary condition constraints of the radiation cube criterion.

4. The method according to claim 1, characterized in that: The method of building a power transmission line connection fittings detection network based on a dual attention YOLOv5 algorithm based on the two-stage connection fittings data set and performing network training on the power transmission line connection fittings detection network includes: The YOLOv5 algorithm is used as the baseline network of the transmission line connection fittings detection network, and the self-attention mechanism and the spatial channel attention mechanism are integrated in the backbone network to improve feature enhancement and small target visibility; Inserting a Ghost module into the backbone network to perform model lightweight processing; The BiFPN structure is used to enhance the fusion effect of the baseline network on multi-scale features; The transmission line connection fittings detection network is trained based on the second-stage connection fittings data set.

5. The method according to claim 4, characterized in that After the YOLOv5 algorithm is used as the baseline network of the power transmission line connection fittings detection network, and the self-attention mechanism and the spatial channel attention mechanism are integrated in the backbone network for feature enhancement and small target visualization improvements, the method further includes: SPPF is added as a spatial pyramid pooling layer after the self-attention mechanism to enhance the recognition ability of the transmission line connection fittings detection network for targets of different sizes.

6. The method according to claim 4, characterized in that The encoder of the self-attention mechanism is obtained by stacking the MLP layer and the MSA layer n times, wherein each stacking includes one MLP layer and one MSA layer.

7. The method according to claim 4, characterized in that The Ghost module includes multiple stacked Ghost convolutions.

8. The method according to claim 4, characterized in that In the BiFPN structure, PANet is used as a feature fusion network.

9. A transmission line connection fittings detection device, characterized in that: include: An acquisition module is used to determine a sample data set of transmission line connection fittings based on drone inspection videos; A first processing module is used to perform defogging processing on the sample data set by using a dark channel prior defogging algorithm to obtain a stage distant image data set; A second processing module is used to intercept the images in the first-stage distant image dataset according to the target detection prediction frame of the images in the first-stage distant image dataset to obtain a second-stage connection hardware dataset; A network construction module, used to build a power transmission line connection fittings detection network based on the dual attention YOLOv5 algorithm based on the two-stage connection fittings dataset and perform network training on the power transmission line connection fittings detection network; The output module is used to detect the power transmission line connection fittings through the power transmission line connection fittings detection network and output the detection results.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images