Power transmission line fault detection method and system, computer device and storage medium

Abstract

The application discloses a power transmission line fault detection method and system, computer equipment and a storage medium, the method constructs a labeled data set and divides a training / test set by shooting a power transmission line fault image by a unmanned aerial vehicle; a fault detection network is built, a multi-level feature map of a training image is extracted by using a backbone feature extraction network, cross-scale feature fusion is realized through a feature pyramid network, and a frequency domain feature fusion module is additionally arranged to realize frequency domain gradient fixed fusion on low-level features; meanwhile, high-level features and frequency domain fusion features are input into a foreground-background weight map creation module for foreground perception processing, so that the feature expression of a target region is enhanced; a joint loss function is designed by combining classification regression results and foreground perception features to train the network, and the network performance is verified through a test set. The method innovatively introduces a frequency domain fusion and foreground perception mechanism, retains the advantages of multi-scale features, strengthens the discriminability of fault features, and improves the power transmission line fault detection precision in a complex scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent monitoring and fault detection of power systems, and particularly relates to a power transmission line fault detection method and system, a computer device and a storage medium. BACKGROUND

[0002] In China, power transmission lines, as the key carriers of power transmission, cover a wide range and pass through various complex geographical environments, including high mountains, gorges, forests, and densely populated urban areas. With the rapid economic development and continuous growth of power demand, the scale of power transmission lines is expanding, which poses many severe challenges to the safe operation of power transmission lines.

[0003] Currently, the inspection and fault detection of power transmission lines mainly rely on traditional manual inspection and some auxiliary means based on image processing and computer technology. Manual inspection not only consumes a lot of manpower, material resources and time, but also is inefficient, especially in some areas with poor transportation and complex terrain, it is difficult to find and eliminate faults in time. While the use of unmanned aerial vehicles to take a large number of pictures and the use of machine learning, deep learning and other technologies to analyze images to achieve automatic detection and positioning of targets, although it has improved the inspection efficiency to some extent, but still has many problems. For example, insulators, shock absorbers and other targets occupy a small number of pixels in the image, which belong to the category of small targets, and due to the randomness of the shooting angle of the unmanned aerial vehicle, the target imaging may be distorted, twisted and other situations, which greatly affects the accuracy of the detection result. In addition, the background environment of the power transmission line is complex, and the target is easy to be blocked, how to reduce the missed detection rate has become a big difficulty in this field. At the same time, small targets carry limited information in the original image, and after multiple downsampling, their resolution is further reduced, and the feature information is gradually weakened, making the detection difficulty significantly increased, which easily leads to failure of fault positioning in the actual application of power transmission line detection, thereby causing serious hidden dangers to power safety.

[0004] In view of the above problems, it is necessary to propose a feature fusion and foreground perception target detection method for power transmission line fault scenes. This method aims to effectively improve the detection accuracy and efficiency of different target detection models for power transmission line faults, reduce the missed detection rate, timely and accurately locate the fault target, and ensure the safe and stable operation of the power transmission line. However, there are many difficulties in the specific implementation process, mainly in the following aspects:

[0005] (1) Designing an efficient feature fusion strategy is one of the key challenges currently facing. In power transmission line images, there are rich multi-scale and multi-modal features. How to fully extract and integrate these features to enhance the model's ability to represent targets in complex backgrounds is a problem that needs to be solved. Currently, due to the different sources and scales of features, it is difficult to find a general and effective method to reasonably fuse these features, making it difficult for the model to capture multi-scale subtle features while accurately grasping the overall characteristics of the target in a complex background.

[0006] (2) It is difficult to build an accurate foreground perception model. It is not easy to accurately distinguish targets from backgrounds and reduce the influence of background interference on detection results. The background environment of the power transmission line is complex and variable, including natural environments such as mountains and trees, and man-made environments such as buildings. How to let the model accurately identify the target in such a complex background is an important problem to be solved.

[0007] (3) Small targets are a thorny problem in power transmission line image detection. Since small targets carry limited information in the original image, after multiple downsampling, the resolution is reduced, and the feature information is gradually weakened, which can easily cause fault positioning failure and seriously affect power safety. How to effectively preserve the key information of small targets during image processing and model operation is another difficulty in achieving accurate fault detection.

[0008] Based on the above problems, the power transmission line fault detection scheme of the present application is proposed. SUMMARY

[0009] The present application provides a power transmission line fault detection method, system, computer device and storage medium, which aims to improve the detection accuracy of the existing target detection model for power transmission line faults, and solves the technical problems of the existing power transmission line fault detection technology, such as being easily disturbed by the background in complex environments, the fault target features being not obvious, the target scale varying greatly, and the data being unbalanced.

[0010] The technical scheme adopted by the present application to solve its technical problems is:

[0011] The power transmission line fault detection method comprises the following steps:

[0012] S100: Collect images of different types of faults of the power transmission line by aerial photography of the unmanned aerial vehicle, make a detection data set of the power transmission line fault scene containing labeling information, and divide it into a training set and a test set;

[0013] S200: Build a fault detection network, including a backbone feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module, and a foreground and background weight map creation module;

[0014] S300: input the image in the training set after convolution processing to the backbone feature extraction network to extract the features of the image and generate multi-level feature maps; input the feature maps of different levels to the feature pyramid network for feature fusion to obtain a feature map set of different scales;

[0015] S400: input the feature map set of different scales to the classification and regression sub-network to obtain classification and regression results, wherein the low-level feature layers in the feature map set of different scales are also input to a frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed; the high-level feature layers and the fused features output by the frequency domain feature fusion module are respectively input to a foreground-background weight map creation module for foreground perception processing to obtain high-level feature maps and fused feature maps after foreground perception processing;

[0016] S500: based on the classification and regression results, the high-level feature maps and the fused feature maps after foreground perception processing, design a loss function to train the fault detection network, when a preset training end condition is reached, obtain the trained fault detection network, input the test set to the trained fault detection network for feature extraction, feature fusion and classification and regression to obtain detection results, and evaluate the detection results.

[0017] Preferably, S100 comprises:

[0018] S110: adopt a patrol unmanned aerial vehicle with a high-definition camera and a stable flight control system to patrol the power transmission line according to a preset route, collect various fault images, and record the shooting time, location and angle information;

[0019] S120: label the image data according to 8 target detection types of ceramic insulators, glass insulators, damaged insulators, connection components, bird nests, suspension clamps, good shock absorbers and damaged shock absorbers;

[0020] S130: randomly divide the data set into a training set, a validation set and a test set to ensure that various fault images are contained and the proportion is consistent with the original data set, and save the image files and the label information respectively.

[0021] Preferably, in S400, the low-level feature layers in the feature map set of different scales are also input to a frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed, comprising:

[0022] S410: from the feature map set of different scales , select the low-level feature maps as , perform up-sampling operation on the low-level feature maps to make all the low-level feature maps have the same size as the first low-level feature map ; wherein, has the highest spatial resolution, It has the richest semantic information. Indicates the selected first A low-level feature map, The number of low-level feature maps selected, and ;

[0023] S420: After converting the feature map with unified size from the spatial domain to the frequency domain representation, feature fusion is performed, and the gradient of the fused features is fixed.

[0024] Preferably, S410 specifically comprises:

[0025] Let the target size be The upsampling method using bilinear interpolation is employed for the feature map. Its arbitrary position Upsampled pixel values The calculation is shown in formula (1):

[0026] (1)

[0027] in, The height of the target feature map, The width of the target feature map. It is a bilinear interpolation upsampling function;

[0028] S420 specifically involves: processing each upsampled low-level feature map... The transformation from the spatial domain to the frequency domain is performed using the two-dimensional discrete Fourier transform, and its calculation process is shown in formula (2):

[0029] (2)

[0030] in, for In the frequency domain representation, Spatial domain coordinates, For frequency domain coordinates, The coordinates of the upsampled feature map in the spatial domain Pixel value at that location, The height of the feature map, The width of the feature map. This is the kernel function for the two-dimensional discrete Fourier transform;

[0031] The fusion is performed using a weighted summation method, and the fusion formula is shown in equation (3):

[0032] (3)

[0033] in, This is the fused frequency domain feature map. For the first The weights of the upsampled low-level feature maps, and satisfying Weight Determined through experimental training.

[0034] Preferably, the fused features output from the high-level feature layer and the frequency domain feature fusion module in S400 are respectively input to the foreground / background weight map creation module for foreground perception processing, resulting in a high-level feature map and a fused feature map after foreground perception processing, including:

[0035] S430: Calculate the scaling ratio between the fused feature layer and the high-level feature layer output from the feature pyramid network and the original image.

[0036] Based on the marked true bounding box information, the position of the foreground on the original image is obtained;

[0037] S440: Based on the mapped foreground position information, create a weight map for the fused feature layer and each higher-level feature layer. The weight map has the same size as the feature layer. In the weight map, weights are assigned to the foreground and background regions respectively. and weight ,in ;

[0038] S450: Expand the weight map to include the batch number of the corresponding fused feature layer and higher-level feature layers. and number of channels The same weight matrix, the size of the expanded weight matrix is The expanded weight matrix is ​​multiplied element-wise with the corresponding fusion feature layer and high-level feature layer to obtain the fusion feature layer and high-level feature layer after foreground perception processing.

[0039] Preferably, S430 specifically comprises:

[0040] The size of the fused feature layer is The original image size is Calculate the scaling ratio of the image to the original image in height and width. and The calculation formula is as follows:

[0041] (4)

[0042] (5)

[0043] The size of the high-level feature layer is The original image size is Calculate the scaling ratio of its height and width compared to the original image. and The calculation formula is as follows:

[0044] (6)

[0045] (7)

[0046] Let the coordinates of the upper left corner of the real box be , and the coordinates of the lower right corner be , the foreground position information on the original image is mapped to the corresponding feature layer respectively for the high-level feature layer and the fusion feature layer, and the mapping calculation formula of the high-level feature layer is as follows:

[0047] (8)

[0048] (9)

[0049] (10)

[0050] (11)

[0051] Where is the floor function, is the upper left corner coordinate of the high-level feature layer after mapping, is the lower right corner coordinate of the high-level feature layer after mapping;

[0052] The mapping calculation formula of the fusion feature layer is as follows:

[0053] (12)

[0054] (13)

[0055] (14)

[0056] (15)

[0057] Where is the floor function, is the upper left corner coordinate of the fusion feature layer after mapping, is the lower right corner coordinate of the fusion feature layer after mapping;

[0058] The weight maps of the high-level and fusion layers in S440 are defined respectively and , and the pixel coordinates are , and the value rules are as follows:

[0059] (16)

[0060] (17)

[0061] S450 in with expanding the weight matrix in the channel dimension with , respectively, element-wise multiplication with the corresponding fusion feature layer and high-level feature layer, to obtain the fusion feature layer after foreground perception processing and the high-level feature layer , specifically:

[0062] (18)

[0063] (19)

[0064] wherein, represents element-wise multiplication operation.

[0065] Preferably, S500 includes:

[0066] S510: Bilinear interpolation up-sampling operation is performed on the high-level feature map after foreground perception processing to make it have the same size as the fusion feature map after foreground perception processing;

[0067] S520: A total loss function is designed, including a classification loss function, a regression loss function, and an alignment loss function, and the alignment loss between the high-level feature map after foreground perception processing and the fusion feature map after foreground perception processing is calculated based on the alignment loss function. The alignment loss is calculated based on the alignment loss function. The alignment degree between the high-level feature map after foreground perception processing and the fusion feature map after foreground perception processing is calculated based on the alignment loss function, specifically:

[0068] (20)

[0069] (21)

[0070] wherein, is the alignment loss, is the target probability map, is the normalization factor, is the batch size, is the feature map height, is the feature map width, is the channel summation variable, is the total number of channels of the feature map;

[0071] S530: According to the classification loss function and the regression loss function, the classification loss and the regression loss are calculated according to the classification and regression results, and the total loss is obtained according to the classification loss, the regression loss and the alignment loss, specifically:

[0072] (22)

[0073] wherein, , and are weight coefficients of classification loss , bounding box regression loss and feature alignment loss , which are adjusted and optimized through experiments;

[0074] S540: training the fault detection network based on the total loss function, when the preset training end condition is reached, obtaining the trained fault detection network, inputting the test set into the trained fault detection network for feature extraction, feature fusion and classification and regression, obtaining the detection result, and evaluating the detection result.

[0075] The power line fault detection system comprises a data set acquisition module, a fault detection network building module, a feature extraction and fusion module, a foreground perception processing module and a training and testing module.

[0076] The data set acquisition module is used to collect images of different types of faults of the power line by aerial photography of the unmanned aerial vehicle, make a detection data set of the power line fault scene containing labeled information, and divide it into a training set and a test set.

[0077] The fault detection network building module is used to build a fault detection network, including a backbone feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module and a foreground-background weight map creation module.

[0078] The feature extraction and fusion module is used to input the images in the training set after convolution processing into the backbone feature extraction network to extract the features of the images, and generate multi-level feature maps; input the different level feature maps into the feature pyramid network for feature fusion, and obtain a feature map set of different scales.

[0079] The foreground perception processing module is used to input the feature map set of different scales into the classification and regression subnetwork to obtain classification and regression results, wherein the low-level feature layer in the feature map set of different scales is also input into the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed; the high-level feature layer and the fused features output by the frequency domain feature fusion module are respectively input into the foreground-background weight map creation module for foreground perception processing, and the high-level feature map and the fused feature map after foreground perception processing are obtained.

[0080] The training and testing module is used for designing a loss function based on the classification and regression results, the high-level feature map after foreground perception processing and the fusion feature map, training the fault detection network, obtaining the trained fault detection network when a preset training end condition is reached, inputting the test set into the trained fault detection network for feature extraction, feature fusion and classification and regression, obtaining a detection result, and evaluating the detection result.

[0081] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the power transmission line fault detection method when executing the computer program.

[0082] A computer readable storage medium stores a computer program, and the computer program implements the steps of the power transmission line fault detection method when executed by a processor.

[0083] The power transmission line fault detection method, system, computer device and storage medium first use an inspection unmanned aerial vehicle to collect power transmission line fault images and make detection data sets for model training and testing, then build a fault detection network, extract features from the backbone feature extraction network, select low-level feature maps from the feature pyramid network output for preliminary fusion in the frequency domain and fix the gradient, then obtain the scaling ratio of the fusion feature layer and the high-level feature layer of the feature pyramid network output and the original image, obtain the foreground position information according to the real box and map it to the feature map, create a weight map to realize foreground perception; then input the training set into the model, let the high-level feature map after foreground perception learn from the fusion feature map after foreground perception, update the model parameters according to the set alignment loss, classification loss and regression loss function; finally, repeat the training and optimization of the model, input the test set into the trained model to obtain the detection result and evaluate it. Through these steps, the detection of the power transmission line fault is realized, and the method is expected to improve the efficiency and accuracy of different target detection models for power transmission line fault detection, thereby assisting the intelligent transformation of power transmission line operation and maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0084] Figure 1 The flowchart of the power transmission line fault detection method in an embodiment of the present application;

[0085] Figure 2 The principle diagram of the power transmission line fault detection method in an embodiment of the present application;

[0086] Figure 3 The comparison diagram of the output feature heat maps of the improved and unimproved fault detection networks, wherein (a) is the output feature heat map of the unimproved fault detection network, and (b) is the output feature heat map of the improved fault detection network. DETAILED DESCRIPTION

[0087] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0088] In one embodiment, as shown in Figure 1 and Figure 2 The power line fault detection method comprises the following steps:

[0089] S100: Collect images of different types of power line faults by aerial photography of a drone, make a detection data set of power line fault scenes containing labeling information, and divide it into a training set and a test set;

[0090] S200: Build a fault detection network, including a backbone feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module, and a foreground-background weight map creation module;

[0091] S300: After the images in the training set are convoluted, input them into the backbone feature extraction network to extract the features of the images, and generate multi-level feature maps; input the different level feature maps into the feature pyramid network for feature fusion, and obtain a feature map set of different scales;

[0092] S400: Input the feature map set of different scales into the classification and regression subnetwork to obtain classification and regression results, wherein the low-level feature layers in the feature map set of different scales are also input into the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed; the high-level feature layers and the fused features output by the frequency domain feature fusion module are respectively input into the foreground-background weight map creation module for foreground perception processing, and obtain the high-level feature map and the fused feature map after foreground perception processing;

[0093] S500: Based on the classification and regression results, the high-level feature map and the fused feature map after foreground perception processing, design a loss function to train the fault detection network, when the preset training end condition is reached, obtain the trained fault detection network, input the test set into the trained fault detection network for feature extraction, feature fusion, and classification and regression, obtain the detection results, and evaluate the detection results.

[0094] In one embodiment, S100 comprises:

[0095] S110: Use a patrol drone with a high-definition camera and a stable flight control system to patrol the power line according to a preset route, collect various fault images, and record the shooting time, location, and angle information;

[0096] S120: Label the image data according to 8 target detection types of ceramic insulators, glass insulators, damaged insulators, connection components, bird nests, suspension clamps, good shock absorbers, and damaged shock absorbers.

[0097] S130: Randomly divide the dataset into training, validation, and test sets, ensuring that they contain images of various types of faults and that the proportions are consistent with the original dataset. Save the image files and annotation information separately.

[0098] Specifically, a total of 1,733 images of various faults were collected, including images of damaged insulators, damaged vibration dampers, and bird nests. Information such as the shooting time, location, and angle were also recorded. The dataset was randomly divided into a training set of 1,000 images, a validation set of 250 images, and a test set of 483 images to ensure that it included images of various faults and that the proportions were basically consistent with the original dataset. The image files and annotation information were saved separately.

[0099] Furthermore, the multi-scale features of the transmission line fault image output by the backbone feature extraction network are input into this network. The feature pyramid network will output a set of feature maps at different scales. , in Indicates the first Feature map of the layer It has the highest spatial resolution. It has the richest semantic information.

[0100] In one embodiment, the low-level feature layers in the feature map sets of different scales in S400 are also input to the frequency domain feature fusion module to perform preliminary feature fusion in the frequency domain and fix the gradient of the fused features, including:

[0101] S410: From feature map sets of different scales In the middle, the low-level feature map is selected and denoted as An upsampling operation is performed on the low-level feature maps so that all low-level feature maps have the same characteristics as the first low-level feature map. Same size; among which, It has the highest spatial resolution. It has the richest semantic information. Indicates the selected first A low-level feature map, The number of low-level feature maps selected, and ;

[0102] S420: After converting the size-unified feature maps from the spatial domain to the frequency domain, feature fusion is performed, and the gradient of the fused features is fixed. This avoids unnecessary updates to the fused features during backpropagation, allowing high-level features to learn from the fused features during subsequent network training.

[0103] In one embodiment, S410 specifically includes:

[0104] Let the target size be The upsampling method using bilinear interpolation is employed for the feature map. Its arbitrary position Upsampled pixel values The calculation is shown in formula (1):

[0105] (1)

[0106] in, The height of the target feature map, The width of the target feature map. It is a bilinear interpolation upsampling function;

[0107] S420 specifically involves: processing each upsampled low-level feature map... The transformation from the spatial domain to the frequency domain is performed using the two-dimensional discrete Fourier transform, and its calculation process is shown in formula (2):

[0108] (2)

[0109] in, for In the frequency domain representation, Spatial domain coordinates, For frequency domain coordinates, The coordinates of the upsampled feature map in the spatial domain Pixel value at that location, The height of the feature map, The width of the feature map. This is the kernel function for the two-dimensional discrete Fourier transform;

[0110] The fusion is performed using a weighted summation method, and the fusion formula is shown in equation (3):

[0111] (3)

[0112] in, This is the fused frequency domain feature map. For the first The weights of the upsampled low-level feature maps, and satisfying Weight Determined through experimental training.

[0113] In one embodiment, the fused features output by the high-level feature layer and the frequency domain feature fusion module in S400 are respectively input to the foreground / background weight map creation module for foreground perception processing, resulting in a high-level feature map and a fused feature map after foreground perception processing, including:

[0114] S430: Calculate the scaling ratio between the fused feature layer and the high-level feature layer output from the feature pyramid network and the original image.

[0115] According to the labeled real frame information, the position of the foreground on the original image is obtained;

[0116] S440: According to the mapped foreground position information, a weight map is created for the fusion feature layer and each high-level feature layer, which has the same size as the feature layer size. In the weight map, the foreground region and the background region are respectively assigned weights and weight , wherein ;

[0117] S450: The weight map is expanded into a weight matrix with the same number of batches and channels as the corresponding fusion feature layer and high-level feature layer. The size of the expanded weight matrix is , and the expanded weight matrix is respectively multiplied with the corresponding fusion feature layer and high-level feature layer element by element to obtain the fusion feature layer and high-level feature layer after foreground-aware processing.

[0118] In one embodiment, S430 is specifically:

[0119] The size of the fusion feature layer is , and the size of the original image is . The scaling ratios of the fusion feature layer and the original image in height and width are calculated as and , and the calculation formula is as follows:

[0120] (4)

[0121] (5)

[0122] The size of the high-level feature layer is , and the size of the original image is . The scaling ratios of the high-level feature layer and the original image in height and width are calculated as and , and the calculation formula is as follows:

[0123] (6)

[0124] (7)

[0125] Let the top-left corner coordinates of the real frame be , and the bottom-right corner coordinates be . For the high-level feature layer and the fusion feature layer, the foreground position information on the original image is respectively mapped to the corresponding feature layer. The mapping calculation formula of the high-level feature layer is as follows:

[0126] (8)

[0127] (9)

[0128] (10)

[0129] (11)

[0130] wherein is a floor function, is the top-left coordinate of the mapped high-level feature layer, is the bottom-right coordinate of the mapped high-level feature layer;

[0131] The mapping calculation formula of the fusion feature layer is as follows:

[0132] (12)

[0133] (13)

[0134] (14)

[0135] (15)

[0136] wherein is a floor function, is the top-left coordinate of the mapped fusion feature layer, is the bottom-right coordinate of the mapped fusion feature layer;

[0137] The weight maps of the high-level and the fusion layer in S440 are defined respectively and , and the pixel coordinates are , and the value rules are as follows:

[0138] (16)

[0139] (17)

[0140] In S450, and are expanded according to the channel dimension to obtain weight matrices and , which are respectively multiplied with the corresponding fusion feature layer and the high-level feature layer element by element to obtain the fusion feature layer and the high-level feature layer after foreground perception processing, and the specific process is as follows:

[0141] (18)

[0142] (19)

[0143] in, This indicates an element-wise multiplication operation.

[0144] Specifically, a comparison of the output characteristic heatmaps of the fault detection network before and after the improvement is shown in the figure below. Figure 3 As shown, the improved network can focus more attention on the foreground region while reducing attention to the background region, significantly improving the recognition accuracy of fault targets. The feature fusion and foreground perception target detection method for transmission line fault scenarios proposed in this invention can significantly improve the detection accuracy of different target detection models for various types of transmission line faults, providing line maintenance personnel with accurate fault location and type information, assisting in timely repair measures, and effectively ensuring the safe and stable operation of transmission lines. In transmission line fault detection, single feature information is insufficient to fully reflect fault characteristics. This invention selects low-level feature maps from the feature pyramid network output and performs preliminary feature fusion in the frequency domain, fully integrating multi-scale feature information, enhancing the ability to express complex fault features, and improving detection accuracy. The scaling ratio of the fused feature layer and the high-level feature layer output of the feature pyramid network to the original image is obtained, and the foreground position information obtained based on the ground truth bounding boxes is mapped onto the feature map to create a weight map to achieve foreground perception. A larger weight is assigned to the foreground and a smaller weight to the background. This approach highlights the importance of the foreground fault target in the feature map, enabling the model to focus more on the fault region and effectively improve the detection capability of fault targets.

[0145] In one embodiment, S500 includes:

[0146] S510: High-level feature map after foreground perception processing Perform bilinear interpolation upsampling to make it the same size as the fused feature map after foreground perception;

[0147] S520: Design the total loss function, including the classification loss function, regression loss function, and alignment loss function. Based on the alignment loss function, calculate the alignment loss between the high-level feature maps processed by foreground perception and the fused feature maps processed by foreground perception of the same size. Specifically:

[0148] (20)

[0149] (twenty one)

[0150] in, For alignment loss, For the target probability map, As the normalization factor, For batch size, For feature map height, is a feature map width, is a channel sum variable, is a total channel number of the feature map;

[0151] S530: According to the classification loss function, the regression loss function, the classification and regression results are combined to calculate the classification loss and the regression loss, and the total loss is obtained according to the classification loss, the regression loss and the alignment loss, specifically:

[0152] (22)

[0153] wherein, , and are weight coefficients of the classification loss , the bounding box regression loss and the feature alignment loss , which are adjusted and optimized through experiments;

[0154] S540: Based on the total loss function, the fault detection network is trained, when the preset training end condition is reached, the trained fault detection network is obtained, the test set is input to the trained fault detection network for feature extraction, feature fusion and classification and regression, the detection result is obtained, and the detection result is evaluated.

[0155] Specifically, the alignment loss function designed by the application innovatively considers the difference between the high-level feature map of the feature pyramid network and the fusion feature map. By minimizing this difference, the model is better aligned in the training process. The feature representation of different levels. This loss function not only balances the information transmission between features, but also optimizes the detection performance of the model for power line faults, improving the adaptability and accuracy of the model;

[0156] The learning rate, batch size, training round number and other training required hyperparameters are set in advance, and the training set is input into the fault detection model for multiple iteration optimization. Each iteration, the model calculates the loss function according to the input training data, updates the parameters through the back propagation algorithm, reduces the loss value, and finds the local optimal solution of the hyperparameters to obtain the optimal network model under the current data set. The test set is input into the trained fault detection model to obtain the detection result, and the result is evaluated according to the preset evaluation indicators, such as average precision mean (mAP), recall rate, etc., to intuitively reflect the detection and generalization ability of the model in actual application.

[0157] The power transmission line fault detection method first uses the inspection unmanned aerial vehicle to shoot and collect power transmission line fault images and make a detection dataset for model training and testing, and then a fault detection network extracts features from the main feature extraction network, selects low-level feature maps in the frequency domain for preliminary fusion and fixes the gradient, then obtains the scaling ratio of the fused feature layer and the high-level feature layer output by the feature pyramid network and the original image, obtains the foreground position information according to the real box and maps it to the feature map, and creates a weight map to realize foreground perception. After that, the training set is input into the model, and the high-level feature map after foreground perception learns from the fused feature map after foreground perception, and the model parameters are updated according to the set alignment loss, classification loss and regression loss function. Finally, the optimized model is repeatedly trained, the test set is input into the trained model to obtain the detection result and evaluate it. Through these steps, the detection of the power transmission line fault is realized, and this method is expected to improve the efficiency and accuracy of different target detection models for power transmission line fault detection, thereby helping the intelligent transformation of power transmission line operation and maintenance.

[0158] In one embodiment, a power transmission line fault detection system is also provided, comprising a dataset acquisition module, a fault detection network building module, a feature extraction and fusion module, a foreground perception processing module, a training and testing module;

[0159] The dataset acquisition module is used to collect images of different types of faults of the power transmission line by aerial shooting of the unmanned aerial vehicle, make a detection dataset of the power transmission line fault scene containing labeling information, and divide it into a training set and a test set;

[0160] The fault detection network building module is used to build a fault detection network, including a main feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module and a foreground and background weight map creation module;

[0161] The feature extraction and fusion module is used to input the images in the training set after convolution processing into the main feature extraction network to extract the features of the images, and generate multi-level feature maps; input the different level feature maps into the feature pyramid network for feature fusion to obtain a feature map set of different scales;

[0162] The foreground perception processing module is used to input the feature map set of different scales into the classification and regression subnetwork to obtain classification and regression results, wherein the low-level feature layer in the feature map set of different scales is also input into the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused feature is fixed; the high-level feature layer and the fused feature output by the frequency domain feature fusion module are input into the foreground and background weight map creation module for foreground perception processing to obtain the high-level feature map and the fused feature map after foreground perception processing;

[0163] The training and testing module is configured to design a loss function based on the classification and regression results, the high-level feature map after foreground perception processing, and the fusion feature map, train the fault detection network, and obtain a trained fault detection network when a preset training end condition is reached. The test set is input to the trained fault detection network for feature extraction, feature fusion, classification, and regression, and a detection result is obtained. The detection result is evaluated.

[0164] The specific limitations of the power line fault detection system can refer to the limitations of the power line fault detection method described above, which will not be repeated here. Each module in the above power line fault detection system can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0165] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the power line fault detection method when executing the computer program.

[0166] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the power line fault detection method.

[0167] Those skilled in the art can understand that all or part of the above-mentioned embodiment methods can be completed by a computer program to instruct related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database, or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0168] The power transmission line fault detection method, system, computer device and storage medium provided by the present application are described in detail above. The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above description of the examples is only used to help understand the core idea of the present application. It should be pointed out that, for ordinary skilled persons in the technical field, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method of transmission line fault detection, characterized by, The method comprises the following steps: S100: Collect images of different types of faults of the power transmission line by aerial shooting of the unmanned aerial vehicle, make a detection data set of the power transmission line fault scene containing labeling information, and divide it into a training set and a test set; S200: Build a fault detection network, including a backbone feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module, and a foreground-background weight map creation module; S300: After the images in the training set are subjected to convolution processing, the features of the images are extracted by the backbone feature extraction network to generate multi-level feature maps; the feature maps of different levels are input into the feature pyramid network for feature fusion to obtain a feature map set of different scales; S400: The feature map set of different scales is input into the classification and regression subnetwork to obtain classification and regression results, wherein the low-level feature layers in the feature map set of different scales are also input into the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed; the high-level feature layers and the fused features output by the frequency domain feature fusion module are respectively input into the foreground-background weight map creation module for foreground perception processing to obtain high-level feature maps and fused feature maps after foreground perception processing; S500: Based on the classification and regression results, the high-level feature maps and the fused feature maps after foreground perception processing, a loss function is designed, the fault detection network is trained, and when a preset training end condition is reached, the trained fault detection network is obtained; the test set is input into the trained fault detection network for feature extraction, feature fusion, and classification and regression to obtain detection results, and the detection results are evaluated.

2. The method of claim 1, wherein, S100 comprises: S110: A patrol unmanned aerial vehicle with a high-definition camera and a stable flight control system is used to patrol the power transmission line according to a preset route, collect various fault images, and record the shooting time, location, and angle information; S120: The image data is labeled according to 8 detection types of ceramic insulators, glass insulators, damaged insulators, connection components, bird nests, suspension clamps, good shock absorbers, and damaged shock absorbers; S130: The data set is randomly divided into a training set, a validation set, and a test set to ensure that various fault images are contained and the proportion is consistent with the original data set, and the image files and labeling information are saved respectively.

3. The method of claim 2, wherein, In S400, the low-level feature layers in the feature map set of different scales are also input into the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed, including: S410: From feature map sets of different scales In the middle, the low-level feature map is selected and denoted as An upsampling operation is performed on the low-level feature maps so that all low-level feature maps have the same characteristics as the first low-level feature map. Same size; among which, It has the highest spatial resolution. It has the richest semantic information. Indicates the selected first A low-level feature map, The number of low-level feature maps selected, and ; S420: After the feature maps with uniform size are converted from the spatial domain to the frequency domain, feature fusion is performed, and the gradient of the fused features is fixed.

4. The method of claim 3, wherein, S410 specifically comprises: The target size is set as The up-sampling method of bilinear interpolation is adopted, and for the feature map The pixel value at any position The pixel value after up-sampling is calculated as shown in formula (1): (1) wherein, is a height of the target feature map, is a width of the target feature map, is a bilinear interpolation up-sampling function; S420 specifically: each up-sampled low-level feature map From the spatial domain to the frequency domain, a two-dimensional discrete Fourier transform is adopted, and the calculation process is shown in formula (2): (2) in, for In the frequency domain representation, Spatial domain coordinates, For frequency domain coordinates, The coordinates of the upsampled feature map in the spatial domain Pixel value at that location, The height of the feature map, The width of the feature map. This is the kernel function for the two-dimensional discrete Fourier transform; The weighted summation method is used for fusion, and the fusion formula is shown in formula (3): (3) wherein, is the fused frequency domain feature map, is the weight of the first up-sampled low-level feature map, and satisfies , the weight is determined through experimental training.

5. The method of claim 4, wherein, In S400, the high-level feature layers and the fused features output by the frequency domain feature fusion module are respectively input into the foreground-background weight map creation module for foreground perception processing to obtain high-level feature maps and fused feature maps after foreground perception processing, including: S430: The scaling ratio of the fused feature layers and the high-level feature layers output by the feature pyramid network to the original image is calculated, According to the labeled true box information, a position of the foreground on the original image is obtained; S440: According to the mapped foreground position information, a weight map is created for the fused feature layer and each high-level feature layer, which has the same size as the feature layer. In the weight map, the foreground region and the background region are respectively assigned weights and weights wherein ; S450: extend the weight map to the same size as the corresponding fused feature layer and the high-level feature layer and the number of channels The size of the extended weight matrix is Element-wise multiply the extended weight matrix with the corresponding fused feature layer and the high-level feature layer respectively to obtain the foreground-aware processed fused feature layer and the high-level feature layer.

6. The method of claim 5, wherein, S430 is specifically: The size of the fused feature layer is , the size of the original image is , the scaling ratio of the height and the width of the original image is calculated and , and the calculation formula is as follows: (4) (5) The size of the high-level feature layer is , the original size is , the scaling ratio of the high-level feature layer in height and width is calculated and , and the calculation formula is as follows: (6) (7) Let the coordinates of the upper left corner of the real frame be , and the coordinates of the lower right corner be , for the high-level feature layer and the fusion feature layer, respectively map the foreground position information on the original image to the corresponding feature layer, The high-level feature layer mapping calculation formula is as follows: (8) (9) (10) (11) wherein is a floor function, is a top-left corner coordinate of the mapped high-level feature layer, is a bottom-right corner coordinate of the mapped high-level feature layer; The fusion feature layer mapping calculation formula is as follows: (12) (13) (14) (15) wherein is a floor function, is the top-left corner coordinate of the mapped fusion feature layer, is the bottom-right corner coordinate of the mapped fusion feature layer. S440 defines the weight map for high layer and fusion layer respectively and , let the pixel coordinates be , the value rules are as follows: (16) (17) In S450 With Expanding by channel dimension to obtain a weight matrix With , respectively, and the corresponding fusion feature layer and high-level feature layer element-wise multiplication, get through the foreground perception processing after the fusion feature layer And high-level feature layer Specifically: (18) (19) wherein represents an element-wise multiplication operation.

7. The method of claim 6, wherein, S500 includes: S510: high-level feature map after foreground perception processing The upsampling operation of bilinear interpolation is performed to have the same size as the fusion feature map after foreground perception. S520: a total loss function is designed, including a classification loss function, a regression loss function and an alignment loss function, an alignment loss between the high-level feature map after foreground perception processing and the fusion feature map after foreground perception processing of the same size is calculated based on the alignment loss function, and the alignment loss is specifically: (20) (21) wherein, is an alignment loss, is a target probability map, is a normalization factor, is a batch size, is a feature map height, is a feature map width, is a channel sum variable, is a total number of channels of the feature map; S530: a classification loss and a regression loss are calculated according to the classification loss function and the regression loss function in combination with the classification and regression results, and a total loss is obtained according to the classification loss, the regression loss and the alignment loss, and the total loss is specifically: (22) wherein, , and are the weight coefficients of the classification loss , the bounding box regression loss and the feature alignment loss , respectively, which are adjusted and optimized through experiments; S540: the fault detection network is trained based on the total loss function, when a preset training end condition is reached, a trained fault detection network is obtained, a test set is input to the trained fault detection network for feature extraction, feature fusion and classification and regression, a detection result is obtained, and the detection result is evaluated.

8. A power line fault detection system characterized by, The method includes a data set acquisition module, a fault detection network building module, a feature extraction and fusion module, a foreground perception processing module, and a training and testing module. The data set acquisition module is configured to collect images of different types of faults of a power transmission line by aerial photography of a UAV, make a detection data set of a power transmission line fault scene containing labeling information, and divide the detection data set into a training set and a test set. The fault detection network building module is configured to build a fault detection network, including a backbone feature extraction network, a feature pyramid network, a classification and regression subnetwork, a frequency domain feature fusion module and a foreground-background weight map creation module. The feature extraction and fusion module is configured to input the images in the training set after convolution processing to the backbone feature extraction network to extract features of the images, and generate multi-level feature maps. The feature extraction and fusion module is configured to input the images in the training set after convolution processing to the backbone feature extraction network to extract features of the images, and generate multi-level feature maps. The foreground perception processing module is configured to input the feature map set of different scales to the classification and regression subnetwork to obtain classification and regression results, wherein the low-level feature layers in the feature map set of different scales are also input to the frequency domain feature fusion module for preliminary feature fusion in the frequency domain, and the gradient of the fused features is fixed; the high-level feature layers and the fused features output by the frequency domain feature fusion module are input to the foreground-background weight map creation module for foreground perception processing, to obtain the high-level feature map and the fused feature map after foreground perception processing. The training and testing module is configured to design a loss function based on the classification and regression results, the high-level feature map and the fused feature map after foreground perception processing, train the fault detection network, obtain a trained fault detection network when a preset training end condition is reached, input the test set to the trained fault detection network for feature extraction, feature fusion, classification and regression, obtain a detection result, and evaluate the detection result. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images