Power transmission line fitting damage detection method and system based on thermal imaging

Abstract

The invention discloses a power transmission line fitting damage detection method based on thermal imaging. The method comprises the steps that an unmanned aerial vehicle thermal imaging detection system of a power transmission line fitting is built; carrying out thermal imaging on the power transmission line fitting by adopting a detection system to obtain an original data set; performing data processing on the original data set to construct a training data set; constructing a power transmission line fitting damage detection initial model based on thermal imaging, and training to obtain a power transmission line fitting damage detection model; and carrying out actual damage detection on the power transmission line hardware by adopting the power transmission line hardware damage detection model to complete damage detection based on thermal imaging. The invention also discloses a system for realizing the thermal imaging-based power transmission line fitting damage detection method. The thermal imaging-based power transmission line fitting damage detection method can realize thermal imaging-based power transmission line fitting damage detection, and is higher in reliability, better in accuracy and higher in efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of electrical automation, and in particular relates to a method and system for detecting damage to hardware of power transmission lines based on thermal imaging. Background Art

[0002] With the development of economy and technology and the improvement of people's living standards, electricity has become an indispensable secondary energy source in people's production and life, bringing endless convenience to people's production and life. Therefore, ensuring the stable and reliable supply of electricity has become one of the most important tasks of the power system.

[0003] As the core component of power transmission in the power system, the stability of the transmission line is of great significance to the power system. Transmission line hardware is the core component of the transmission line, which has the functions of fixing, protecting, connecting and maintaining the stable operation of the entire line. However, the working environment of transmission line hardware is usually extremely harsh, so it is very easy to have various problems such as rust, deformation, and breakage. Therefore, damage detection of transmission line hardware is of great significance.

[0004] At present, the damage detection of transmission line fittings is mainly divided into manual detection solutions and drone detection solutions. The manual detection solution requires power system maintenance personnel to climb the transmission line towers and manually detect the transmission line fittings. However, this type of manual detection solution not only has poor safety and reliability, but also has low detection efficiency. The drone detection solution generally uses the camera carried by the drone to take pictures of the transmission line fittings, and then detects the damage of the fittings through feature description algorithms or machine learning algorithms. However, this type of drone detection solution is also subject to the quality of the captured images, and its reliability and accuracy are still poor. Summary of the invention

[0005] One of the purposes of the present invention is to provide a method for detecting damage to power transmission line fittings based on thermal imaging with high reliability, good accuracy and high efficiency.

[0006] A second objective of the present invention is to provide a system for implementing the power transmission line fittings damage detection method based on thermal imaging.

[0007] The method for detecting damage to power transmission line hardware based on thermal imaging provided by the present invention comprises the following steps:

[0008] S1. Build a UAV thermal imaging detection system for transmission line hardware;

[0009] S2. Using the detection system built in step S1, thermal imaging of the transmission line fittings is performed to obtain an original data set;

[0010] S3. Processing the original data set obtained in step S2 to construct a training data set;

[0011] S4. Based on the YOLOv8 network and the deformable convolutional network, an initial model for transmission line hardware damage detection based on thermal imaging is constructed;

[0012] S5. Using the training data set obtained in step S3, the model constructed in step S4 is trained to obtain a transmission line hardware damage detection model;

[0013] S6. Using the power transmission line fitting damage detection model obtained in step S5, actual power transmission line fitting damage detection is performed to complete thermal imaging-based damage detection.

[0014] The construction of the UAV thermal imaging detection system for power transmission line fittings described in step S1 specifically includes the following steps:

[0015] An infrared thermal imager and a laser heater are carried on the UAV; the laser heater is used to heat the transmission line fittings, and the infrared thermal imager is used to obtain infrared thermal imaging data of the transmission line fittings.

[0016] Step S2 uses the detection system built in step S1 to perform thermal imaging on the transmission line fittings to obtain an original data set, which specifically includes the following steps:

[0017] Control the drone to fly within the set range near the power transmission line fittings;

[0018] First, the output power of the laser heater is set to a set low power value, and the laser heater is controlled to focus on the surface of the transmission line fittings; then, the output power of the laser heater is set to a set high power value, and the drone is controlled to fly around the transmission line fittings according to the set route and speed, and the laser heater is used to heat the surface of the transmission line fittings;

[0019] When the surface temperature of the transmission line fittings reaches the set requirements, the laser heater is turned off; an infrared thermal imager is used to obtain image data of the transmission line fittings from a set angle and a set direction to obtain an original data set; the image data includes infrared image data and visible light image data.

[0020] The step S3 of processing the original data set obtained in step S2 to construct a training data set specifically includes the following steps:

[0021] Performing data processing on the original data set obtained in step S2;

[0022] The data processing includes: image deletion, image annotation, and image enhancement;

[0023] Image deletion: delete redundant images and images that cannot be labeled;

[0024] Image annotation: Defect determination is performed on the segmented image, and the determined defects are annotated using annotation tools;

[0025] Image enhancement: A data enhancement scheme is used to enhance the labeled image; the data enhancement scheme includes rotation, flipping and contrast enhancement.

[0026] In step S4, based on the YOLOv8 network and the deformable convolutional network, an initial model for transmission line hardware damage detection based on thermal imaging is constructed, which specifically includes the following steps:

[0027] The initial model of power transmission line hardware damage detection based on thermal imaging was constructed, including defect detection module, defect segmentation module and area calculation module;

[0028] The defect detection module, the defect segmentation module and the area calculation module are connected in series in sequence;

[0029] Based on the YOLOv8 network and the variable convolutional network, a defect detection module is constructed; the defect detection module is used to judge the input image of the transmission line hardware to obtain the defect judgment result of the transmission line hardware;

[0030] A defect segmentation module is constructed based on the threshold method; the defect segmentation module is used to perform image segmentation and pixel statistics on the defective parts of the hardware in the transmission line hardware image based on the determination result output by the defect detection module;

[0031] Based on the parameters of the camera and the geometric relationship between the image and the real world, an area calculation module is constructed; the area calculation module calculates the actual defect area of ​​the transmission line fittings according to the pixels of the defective part of the fittings counted by the defect segmentation module, and completes the damage detection of the transmission line fittings based on thermal imaging.

[0032] The defect detection module is constructed based on the YOLOv8 network and the variable convolutional network, and specifically includes the following steps:

[0033] Build a defect detection module based on the YOLOv8 network;

[0034] The constructed defect detection module includes backbone layer, neck layer and head layer;

[0035] The structure and function of the neck layer and head layer are the same as those of the YOLOv8 network;

[0036] The backbone layer includes a first convolutional layer, a second convolutional layer, a first C2f layer, a third convolutional layer, a first DCN-C2f layer, a fourth convolutional layer, a second DCN-C2f layer, a fifth convolutional layer, a second C2f layer and an SPPF layer which are connected in series in sequence; wherein the structures and functions of the first convolutional layer, the second convolutional layer, the first C2f layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the second C2f layer and the SPPF layer are the same as those of the YOLOv8 network; the output of the first DCN-C2f layer, the output of the second DCN-C2f layer and the output of the SPPF layer are used as the output of the backbone layer;

[0037] The first DCN-C2f layer and the second DCN-C2f layer have the same structure and are both used to expand the receptive field of the feature map, adjust the regression parameters of the prediction box, and improve the model's ability to represent features;

[0038] The DCN-C2f layer includes the first sub-convolution layer, the first sub-split layer, the first sub-DCN layer to the Nth sub-DCN layer, the first sub-connection layer and the second sub-convolution layer;

[0039] The output of the first sub-convolutional layer is divided into two paths, one path is directly output to the first sub-split layer, and the other path is output to the first sub-connection layer; the first sub-split layer is used to divide the input features into two paths, one path is output to the first sub-connection layer, and the other path is output to the first sub-DCN layer; in the first sub-DCN layer to the Nth sub-DCN layer, the output of the i-th sub-DCN layer is used as the input of the i+1th sub-DCN layer, and the output of the i-th sub-DCN layer is also directly connected to the first sub-connection layer; the first sub-connection layer connects all the output features and outputs them to the second sub-convolutional layer; the output of the second sub-convolutional layer is used as the output of the corresponding DCN-C2f layer;

[0040] The first sub-convolution layer is used to perform convolution processing on the input features to capture the preliminary feature information in the input image; the first sub-split layer is used to segment the input features, thereby allocating the data stream to different deformable convolution layers; the first sub-DCN layer to the Nth sub-DCN layer are used to adjust the convolution kernel position to align and capture the feature information in the input features; the first sub-connection layer is used to connect the input features; the second sub-convolution layer further convolutions the connected features to enhance the association between different features and optimize the final output;

[0041] The structures of the first to Nth sub-DCN layers are the same; the i-th sub-DCN layer includes the first DCN layer, the second DCN layer and the summation layer; the input features are processed by the first DCN layer and the second DCN layer in sequence, and then output to the summation layer, which sums the output of the second DCN layer and the input features as the output of the i-th sub-DCN layer; the first DCN layer is used to perform spatial deformation processing on the features, and adjusts the shape and position of the convolution kernel by learning the local changes of the input features, so as to capture the geometric features of the input image; the second DCN layer is used to further enhance the adaptability of the model to the input image and provide more accurate feature adjustment;

[0042] The first DCN layer has the same structure as the second DCN layer, with a convolution kernel of 3×3 and a dilation rate of 1;

[0043] For the DCN layer with a convolution kernel of 3×3 and a dilation rate of 1, there is

[0044] R=(-1,1),(-1,0),(-1,-1),(0,1),(0,0),(0,-1),(1,1),(1,0),(1,-1)

[0045] Where R is the regular network of the convolution kernel, which represents the offset of the convolution kernel relative to the center point;

[0046] The DCN layer with a convolution kernel of 3×3 and a dilation rate of 1 has an output feature matrix expressed as

[0047]

[0048] In the formula, y(p 0 ) is p 0 The convolution output result of the DCN layer at position p 0 is the pixel position in the feature map; w(p n ) is p n The weight of the position; x(p 0 +p n +Δp n ) indicates that the input feature map is in p 0 +p n +Δp n The pixel value of the position;

[0049] The DCN layer has a convolution kernel of 3×3 and a dilation rate of 1, and uses a bilinear interpolation algorithm to achieve offset.

[0050] The defect segmentation module constructed based on the threshold method specifically includes the following steps:

[0051] The Otsu threshold method based on Gaussian filtering is used to segment the defective parts of the transmission line hardware images.

[0052] According to the obtained image segmentation results, the number of pixels contained in the defective part of the hardware is calculated.

[0053] The area calculation module is constructed based on the geometric relationship between the camera parameters and the image and the real world, and specifically includes the following steps:

[0054] Obtain the diagonal field of view angle α of the infrared thermal imager, the distance d between the infrared thermal imager and the line fittings, and the resolution M×N of the infrared thermal imager;

[0055] According to the resolution M×N of the infrared thermal imager, the number of pixels on the diagonal of the image is calculated as

[0056]

[0057] Assume the actual length L of the diagonal at distance d is Then the corresponding relationship between the pixel grid and the actual length is expressed as Where δ is the actual length of 1 pixel, and P is the number of pixels contained in the image diagonal at distance d;

[0058] According to δ and the number of pixels contained in the defective part of the hardware, the actual defect area of ​​the transmission line hardware is calculated.

[0059] The transmission line fitting damage detection model obtained in step S5 is used in step S6 to perform actual transmission line fitting damage detection, thereby completing thermal imaging-based damage detection, which specifically includes the following steps:

[0060] When actually performing the inspection, firstly, the UAV thermal imaging inspection system for the power transmission line fittings built in step S1 is used to obtain infrared thermal imaging data of the target power transmission line fittings;

[0061] The acquired infrared thermal imaging data of the target transmission line fittings is input into the transmission line fittings damage detection model obtained in step S5 to obtain the actual defect area of ​​the target transmission line fittings, thereby completing the thermal imaging-based damage detection of the target transmission line fittings.

[0062] The present invention also provides a system for realizing the power transmission line fittings damage detection method based on thermal imaging, comprising a system building module, a data acquisition module, a data processing module, a model building module, a model training module and a damage detection module; the system building module, the data acquisition module, the data processing module, the model building module, the model training module and the damage detection module are connected in series in sequence; the system building module is used to build a UAV thermal imaging detection system for power transmission line fittings, and upload the data information to the data acquisition module; the data acquisition module is used to perform thermal imaging on the power transmission line fittings according to the received data information by using the built detection system to obtain the original data set, and upload the data information to the data processing module; the data processing module is used to process the obtained data according to the received data information The original data set taken is processed to construct a training data set, and the data information is uploaded to the model construction module; the model construction module is used to construct an initial model for transmission line hardware damage detection based on thermal imaging based on the YOLOv8 network and the variable convolutional network according to the received data information, and upload the data information to the model training module; the model training module is used to train the model constructed in step S4 according to the received data information and the obtained training data set, so as to obtain a transmission line hardware damage detection model, and upload the data information to the damage detection module; the damage detection module is used to perform actual transmission line hardware damage detection according to the received data information and the obtained transmission line hardware damage detection model, and complete damage detection based on thermal imaging.

[0063] The power transmission line fittings damage detection method and system based on thermal imaging provided by the present invention heats the target power transmission line fittings by a drone to obtain thermal imaging data, and constructs a damage detection model based on a YOLOv8 network and a variable convolutional network to perform damage detection; therefore, the present invention can not only realize power transmission line fittings damage detection based on thermal imaging, but also has higher reliability, better accuracy and higher efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 The figure is a schematic diagram of the method flow of the present invention.

[0065] Figure 2 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION

[0066] like Figure 1 The method flow chart of the method of the present invention is shown as follows: The method for detecting damage to power transmission line hardware based on thermal imaging disclosed in the present invention comprises the following steps:

[0067] S1. Build a UAV thermal imaging detection system for transmission line fittings; specifically include the following steps:

[0068] An infrared thermal imager and a laser heater are mounted on the drone; the laser heater is used to heat the transmission line fittings, and the infrared thermal imager is used to obtain infrared thermal imaging data of the transmission line fittings;

[0069] S2. Using the detection system built in step S1, thermal imaging of the transmission line fittings is performed to obtain an original data set; specifically, the steps include:

[0070] Control the drone to fly to a set range near the power transmission line fittings, preferably 1 to 1.5 meters away from the power transmission line fittings;

[0071] First, the output power of the laser heater is set to a set low power value, and the laser heater is controlled to focus on the surface of the transmission line fittings; then, the output power of the laser heater is set to a set high power value, and the drone is controlled to fly around the transmission line fittings according to the set route and speed, and the laser heater is used to heat the surface of the transmission line fittings;

[0072] The laser heating method used has the following advantages:

[0073] a) Low heat loss and high efficiency: Laser heating is a non-contact heating method that avoids a large amount of heat loss caused by unnecessary heat conduction and heat convection; b) Green and energy-saving, fast heating: The excitation principle of laser determines that it has the characteristics of low noise and zero pollution during the heating process; at the same time, because laser can provide extremely high power density to act on the heated object, and has high thermal efficiency and low heat loss, its heating speed is fast, and the heating process can be completed in a short time, reducing energy consumption loss to a minimum, thereby extending the flight time of the drone in the air;

[0074] When the surface temperature of the transmission line fittings reaches the set requirement, the laser heater is turned off; an infrared thermal imager is used to obtain image data of the transmission line fittings from a set angle and a set direction to obtain an original data set; the image data includes infrared image data and visible light image data;

[0075] In the specific implementation, it is preferred to use an H20T airborne infrared thermal imager, which has an operating temperature of -20°C-50°C, can detect thermal radiation wavelengths in the range of 8-14 μm, has an infrared camera resolution of 640×512, and a visible light camera resolution of 1920×1080;

[0076] S3. Processing the original data set obtained in step S2 to construct a training data set; specifically comprising the following steps:

[0077] Performing data processing on the original data set obtained in step S2;

[0078] The data processing includes: image deletion, image annotation, and image enhancement;

[0079] Image deletion: delete redundant images and images that cannot be labeled;

[0080] Image annotation: Defect determination is performed on the segmented image, and the determined defects are annotated using an annotation tool (preferably the LableImage open source tool is installed in a python environment), and different defects in the image are annotated to generate an XML file; the defects include rust, deformation, surface damage, internal water injection, etc.;

[0081] Image enhancement: The annotated image is enhanced by using a data enhancement scheme; the data enhancement scheme includes rotation, flipping and contrast enhancement;

[0082] S4. Based on the YOLOv8 network and the deformable convolutional network, an initial model for transmission line hardware damage detection based on thermal imaging is constructed; specifically, the following steps are included:

[0083] The initial model of power transmission line hardware damage detection based on thermal imaging was constructed, including defect detection module, defect segmentation module and area calculation module;

[0084] The defect detection module, the defect segmentation module and the area calculation module are connected in series in sequence;

[0085] Based on the YOLOv8 network and the variable convolutional network, a defect detection module is constructed; the defect detection module is used to judge the input image of the transmission line fittings to obtain the defect judgment result of the transmission line fittings; specifically, the steps are as follows:

[0086] Build a defect detection module based on the YOLOv8 network;

[0087] The constructed defect detection module includes backbone layer, neck layer and head layer;

[0088] The structure and function of the neck layer and head layer are the same as those of the YOLOv8 network;

[0089] The backbone layer includes a first convolutional layer, a second convolutional layer, a first C2f layer, a third convolutional layer, a first DCN-C2f layer, a fourth convolutional layer, a second DCN-C2f layer, a fifth convolutional layer, a second C2f layer and an SPPF layer which are connected in series in sequence; wherein the structures and functions of the first convolutional layer, the second convolutional layer, the first C2f layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the second C2f layer and the SPPF layer are the same as those of the YOLOv8 network; the output of the first DCN-C2f layer, the output of the SPPF layer and the output of the second DCN-C2f layer are used as the output of the backbone layer;

[0090] The first DCN-C2f layer and the second DCN-C2f layer have the same structure and are both used to expand the receptive field of the feature map, adjust the regression parameters of the prediction box, and improve the model's ability to represent features;

[0091] The DCN-C2f layer includes the first sub-convolution layer, the first sub-split layer, the first sub-DCN layer to the Nth sub-DCN layer, the first sub-connection layer and the second sub-convolution layer;

[0092] The output of the first sub-convolutional layer is divided into two paths, one path is directly output to the first sub-split layer, and the other path is output to the first sub-connection layer; the first sub-split layer is used to divide the input features into two paths, one path is output to the first sub-connection layer, and the other path is output to the first sub-DCN layer; in the first sub-DCN layer to the Nth sub-DCN layer, the output of the i-th sub-DCN layer is used as the input of the i+1th sub-DCN layer, and the output of the i-th sub-DCN layer is also directly connected to the first sub-connection layer; the first sub-connection layer connects all the output features and outputs them to the second sub-convolutional layer; the output of the second sub-convolutional layer is used as the output of the corresponding DCN-C2f layer;

[0093] The first sub-convolution layer is used to perform convolution processing on the input features to capture the preliminary feature information in the input image and provide the original feature basis for subsequent more complex operations; the first sub-split layer is used to segment the input features, which helps to distribute the data streams to different deformation convolution layers. After segmentation, each stream can independently process different aspects of the data, making feature processing more accurate and effective; the first sub-DCN layer to the Nth sub-DCN layer are used to adaptively adjust the convolution kernel position to better align and capture important local features in the input features. The stacking of these layers improves the network's ability to process deformations and non-standardized features in images; the first sub-connection layer is used to connect the input features; the second sub-convolution layer further convolves the connected features to enhance the association between different features and optimize the final output.

[0094] The structures of the first to Nth sub-DCN layers are the same; the i-th sub-DCN layer includes the first DCN layer, the second DCN layer and the summation layer; the input features are processed by the first DCN layer and the second DCN layer in sequence, and then output to the summation layer, which sums the output of the second DCN layer and the input features as the output of the i-th sub-DCN layer; the first DCN layer is used to perform preliminary spatial deformation processing on the features, and adjusts the shape and position of the convolution kernel by learning the local changes of the input features to better capture the important geometric features of the input image; the second DCN layer is used to further enhance the adaptability of the model to the input image, and provides more accurate feature adjustment by refining the processing results of the first DCN layer;

[0095] The first DCN layer has the same structure as the second DCN layer, with a convolution kernel of 3×3 and a dilation rate of 1;

[0096] For the DCN layer with a convolution kernel of 3×3 and a dilation rate of 1, there is

[0097] R=(-1,1),(-1,0),(-1,-1),(0,1),(0,0),(0,-1),(1,1),(1,0),(1,-1)

[0098] Where R is the regular network of the convolution kernel, which represents the offset of the convolution kernel relative to the center point;

[0099] The DCN layer with a convolution kernel of 3×3 and a dilation rate of 1 has an output feature matrix expressed as

[0100]

[0101] In the formula, y(p 0 ) is p 0 The convolution output result of the DCN layer at position p 0 is the pixel position in the feature map; w(p n ) is p n The weight of the position; x(p 0 +p n +Δp n ) indicates that the input feature map is in p 0 +p n +Δp n The pixel value of the position;

[0102] The DCN layer has a convolution kernel of 3×3 and a dilation rate of 1, and uses a bilinear interpolation algorithm to achieve the offset;

[0103] In the original YOLOv8 backbone network, the C2f standard convolution kernel lacks flexibility, resulting in a small receptive field of the network and can only obtain local information, which makes the network prone to missed detection when processing multi-target and multi-scale targets; in addition, due to the interference of noise signals, it also has serious problems of false detection and missed detection of small targets;

[0104] The deformable convolution DCN can adjust the regression parameters of the prediction box and improve the model's ability to characterize occluded targets, multi-scale targets and small targets. At the same time, the purpose of adding the deformable convolution network DCN is to improve the model's ability to extract invariant features. DCN enables each sampling point in the convolution kernel to obtain an offset through learning to achieve adaptation to the target geometry. DCN can also learn different optimal convolution kernel structures according to different target data, thereby enhancing the ability to extract features of infrared targets of different scales.

[0105] A defect segmentation module is constructed based on the threshold method; the defect segmentation module is used to perform image segmentation and pixel statistics on the defective parts of the hardware in the transmission line hardware image based on the determination result output by the defect detection module; the specific steps include:

[0106] The Otsu threshold method based on Gaussian filtering is used to segment the defective parts of the hardware in the transmission line hardware image; the Otsu threshold method based on Gaussian filtering can retain relatively smooth image edges and relatively complete defect information;

[0107] According to the obtained image segmentation result, the number of pixels contained in the defective part of the hardware is calculated; in specific implementation, the number of pixels contained in the defect can be calculated by Photoshop software;

[0108] Based on the parameters of the camera and the geometric relationship between the image and the real world, an area calculation module is constructed; the area calculation module calculates the actual defect area of ​​the power transmission line fittings according to the pixels of the defective part of the fittings counted by the defect segmentation module, and completes the damage detection of the power transmission line fittings based on thermal imaging; specifically, the following steps are included:

[0109] Obtain the diagonal field of view angle α of the infrared thermal imager, the distance d between the infrared thermal imager and the line fittings, and the resolution M×N of the infrared thermal imager;

[0110] According to the resolution M×N of the infrared thermal imager, the number of pixels on the diagonal of the image is calculated as

[0111]

[0112] Assume the actual length L of the diagonal at distance d is Then the corresponding relationship between the pixel grid and the actual length is expressed as Where δ is the actual length of 1 pixel, and P is the number of pixels contained in the diagonal line of the image at distance d;

[0113] According to δ and the number of pixels contained in the defective part of the hardware, the actual defect area of ​​the transmission line hardware is calculated;

[0114] S5. Using the training data set obtained in step S3, the model constructed in step S4 is trained to obtain a transmission line hardware damage detection model;

[0115] After the training is completed, the defects are predicted, and the mean average precision (mAP) is calculated for each prediction result. Since there may be errors or deviations in the data annotation process, the initially trained network model has poor results in defect prediction. In order to offset the impact of data set annotation deviation (including wrong annotation, missing annotation, and inconsistent annotation) on the prediction effect, during the training process, the network prediction results are reviewed and the predicted results are used as new data sets, and the wrong annotations are corrected. Afterwards, the corrected data is retrained until the model's prediction effect meets the relevant accuracy requirements.

[0116] S6. Using the power transmission line fitting damage detection model obtained in step S5, actual power transmission line fitting damage detection is performed to complete thermal imaging-based damage detection; specifically, the following steps are included:

[0117] When actually performing the inspection, firstly, the UAV thermal imaging inspection system for the power transmission line fittings built in step S1 is used to obtain infrared thermal imaging data of the target power transmission line fittings;

[0118] The acquired infrared thermal imaging data of the target transmission line fittings is input into the transmission line fittings damage detection model obtained in step S5 to obtain the actual defect area of ​​the target transmission line fittings, thereby completing the thermal imaging-based damage detection of the target transmission line fittings.

[0119] like Figure 2The figure shows a schematic diagram of the functional modules of the system of the present invention: the system disclosed in the present invention for realizing the thermal imaging-based damage detection method for power transmission line fittings comprises a system building module, a data acquisition module, a data processing module, a model building module, a model training module and a damage detection module; the system building module, the data acquisition module, the data processing module, the model building module, the model training module and the damage detection module are connected in series in sequence; the system building module is used to build a UAV thermal imaging detection system for power transmission line fittings, and upload the data information to the data acquisition module; the data acquisition module is used to perform thermal imaging of the power transmission line fittings using the built detection system according to the received data information, so as to obtain the original data set, and upload the data information to the data processing module; the data processing module is used to The data information is used to process the original data set obtained to construct a training data set, and upload the data information to the model construction module; the model construction module is used to construct an initial model for damage detection of power transmission line fittings based on thermal imaging based on the YOLOv8 network and the variable convolutional network according to the received data information, and upload the data information to the model training module; the model training module is used to train the model constructed in step S4 using the obtained training data set according to the received data information, obtain a transmission line fitting damage detection model, and upload the data information to the damage detection module; the damage detection module is used to perform actual damage detection of transmission line fittings according to the received data information using the obtained transmission line fitting damage detection model, and complete damage detection based on thermal imaging.

Claims

1. A method for detecting damage to transmission line hardware based on thermal imaging, comprising the following steps: S1. Build a UAV thermal imaging detection system for transmission line hardware; S2. Using the detection system built in step S1, thermal imaging of the transmission line fittings is performed to obtain an original data set; S3. Processing the original data set obtained in step S2 to construct a training data set; S4. Based on the YOLOv8 network and the deformable convolutional network, an initial model for transmission line hardware damage detection based on thermal imaging is constructed; S5. Using the training data set obtained in step S3, the model constructed in step S4 is trained to obtain a transmission line hardware damage detection model; S6. Using the power transmission line fitting damage detection model obtained in step S5, actual power transmission line fitting damage detection is performed to complete thermal imaging-based damage detection.

2. The method for detecting damage to power transmission line fittings based on thermal imaging according to claim 1 is characterized in that The construction of the UAV thermal imaging detection system for power transmission line fittings described in step S1 specifically includes the following steps: An infrared thermal imager and a laser heater are carried on the UAV; the laser heater is used to heat the transmission line fittings, and the infrared thermal imager is used to obtain infrared thermal imaging data of the transmission line fittings.

3. The method for detecting damage to power transmission line fittings based on thermal imaging according to claim 2 is characterized in that Step S2 uses the detection system built in step S1 to perform thermal imaging on the transmission line fittings to obtain an original data set, which specifically includes the following steps: Control the drone to fly within the set range near the power transmission line fittings; First, the output power of the laser heater is set to a set low power value, and the laser heater is controlled to focus on the surface of the transmission line fittings; then, the output power of the laser heater is set to a set high power value, and the drone is controlled to fly around the transmission line fittings according to the set route and speed, and the laser heater is used to heat the surface of the transmission line fittings; When the surface temperature of the transmission line fittings reaches the set requirements, the laser heater is turned off; an infrared thermal imager is used to obtain image data of the transmission line fittings from a set angle and a set direction to obtain an original data set; the image data includes infrared image data and visible light image data.

4. The method for detecting damage to power transmission line fittings based on thermal imaging according to claim 3 is characterized in that The step S3 of processing the original data set obtained in step S2 to construct a training data set specifically includes the following steps: Performing data processing on the original data set obtained in step S2; The data processing includes: image deletion, image annotation, and image enhancement; Image deletion: delete redundant images and images that cannot be labeled; Image annotation: Defect determination is performed on the segmented image, and the determined defects are annotated using annotation tools; Image enhancement: A data enhancement scheme is used to enhance the labeled image; the data enhancement scheme includes rotation, flipping and contrast enhancement.

5. The method for detecting damage to power transmission line hardware based on thermal imaging according to claim 4 is characterized in that The initial model for power transmission line hardware damage detection based on thermal imaging is constructed based on the YOLOv8 network and the deformable convolutional network described in step S4, which specifically includes the following steps: The initial model of power transmission line hardware damage detection based on thermal imaging was constructed, including defect detection module, defect segmentation module and area calculation module; The defect detection module, the defect segmentation module and the area calculation module are connected in series in sequence; Build a defect detection module based on the YOLOv8 network and the deformable convolutional network; The defect detection module is used to judge the input image of the transmission line fittings to obtain the defect judgment result of the transmission line fittings; Construct defect segmentation module based on threshold method; The defect segmentation module is used to perform image segmentation and pixel statistics on the defective parts of the power transmission line hardware images based on the determination results output by the defect detection module; Based on the camera parameters and the geometric relationship between the image and the real world, an area calculation module is constructed; The area calculation module calculates the actual defect area of ​​the transmission line fittings based on the pixels of the defective parts of the fittings counted by the defect segmentation module, and completes the damage detection of the transmission line fittings based on thermal imaging.

6. The method for detecting damage to power transmission line hardware based on thermal imaging according to claim 5 is characterized in that The defect detection module is constructed based on the YOLOv8 network and the variable convolutional network, and specifically includes the following steps: Build a defect detection module based on the YOLOv8 network; The constructed defect detection module includes backbone layer, neck layer and head layer; The structure and function of the neck layer and head layer are the same as those of the YOLOv8 network; The backbone layer includes a first convolutional layer, a second convolutional layer, a first C2f layer, a third convolutional layer, a first DCN-C2f layer, a fourth convolutional layer, a second DCN-C2f layer, a fifth convolutional layer, a second C2f layer and an SPPF layer which are connected in series in sequence; wherein the structures and functions of the first convolutional layer, the second convolutional layer, the first C2f layer, the third convolutional layer, the fourth convolutional layer, the fifth convolutional layer, the second C2f layer and the SPPF layer are the same as those of the YOLOv8 network; the output of the first DCN-C2f layer, the output of the second DCN-C2f layer and the output of the SPPF layer are used as the output of the backbone layer; The first DCN-C2f layer and the second DCN-C2f layer have the same structure and are both used to expand the receptive field of the feature map, adjust the regression parameters of the prediction box, and improve the model's ability to represent features; The DCN-C2f layer includes the first sub-convolution layer, the first sub-split layer, the first sub-DCN layer to the Nth sub-DCN layer, the first sub-connection layer and the second sub-convolution layer; The output of the first sub-convolutional layer is divided into two paths, one path is directly output to the first sub-split layer, and the other path is output to the first sub-connection layer; the first sub-split layer is used to divide the input features into two paths, one path is output to the first sub-connection layer, and the other path is output to the first sub-DCN layer; in the first sub-DCN layer to the Nth sub-DCN layer, the output of the i-th sub-DCN layer is used as the input of the i+1th sub-DCN layer, and the output of the i-th sub-DCN layer is also directly connected to the first sub-connection layer; the first sub-connection layer connects all the output features and outputs them to the second sub-convolutional layer; the output of the second sub-convolutional layer is used as the output of the corresponding DCN-C2f layer; The first sub-convolution layer is used to perform convolution processing on the input features to capture the preliminary feature information in the input image; the first sub-split layer is used to segment the input features, thereby allocating the data stream to different deformable convolution layers; the first sub-DCN layer to the Nth sub-DCN layer are used to adjust the convolution kernel position to align and capture the feature information in the input features; the first sub-connection layer is used to connect the input features; the second sub-convolution layer further convolutions the connected features to enhance the association between different features and optimize the final output; The structures of the first to Nth sub-DCN layers are the same; the i-th sub-DCN layer includes the first DCN layer, the second DCN layer and the summation layer; the input features are processed by the first DCN layer and the second DCN layer in sequence, and then output to the summation layer, which sums the output of the second DCN layer and the input features as the output of the i-th sub-DCN layer; the first DCN layer is used to perform spatial deformation processing on the features, and adjusts the shape and position of the convolution kernel by learning the local changes of the input features, so as to capture the geometric features of the input image; the second DCN layer is used to further enhance the adaptability of the model to the input image and provide more accurate feature adjustment; The first DCN layer has the same structure as the second DCN layer, with a convolution kernel of 3×3 and a dilation rate of 1; For the DCN layer with a convolution kernel of 3×3 and a dilation rate of 1, there is R＝(-1,1),(-1,0),(-1,-1),(0,1),(0,0),(0,-1),(1,1),(1,0),(1,-1) Where R is the regular network of the convolution kernel, which represents the offset of the convolution kernel relative to the center point; The DCN layer with a convolution kernel of 3×3 and a dilation rate of 1 has an output feature matrix expressed as Where y(p0) is the convolution output of the DCN layer at position p0; p0 is the pixel position in the feature map; w(p n ) is p n The weight of the position; x(p0+p n +Δp n ) indicates that the input feature map is at p0+p n +Δp n The pixel value of the position; The DCN layer has a convolution kernel of 3×3 and a dilation rate of 1, and uses a bilinear interpolation algorithm to achieve offset.

7. The method for detecting damage to power transmission line hardware based on thermal imaging according to claim 6 is characterized in that The defect segmentation module constructed based on the threshold method specifically includes the following steps: The Otsu threshold method based on Gaussian filtering is used to segment the defective parts of the transmission line hardware images. According to the obtained image segmentation results, the number of pixels contained in the defective part of the hardware is calculated.

8. The method for detecting damage to power transmission line hardware based on thermal imaging according to claim 7 is characterized in that The area calculation module is constructed based on the geometric relationship between the camera parameters and the image and the real world, and specifically includes the following steps: Obtain the diagonal field of view angle α of the infrared thermal imager, the distance d between the infrared thermal imager and the line fittings, and the resolution M×N of the infrared thermal imager; According to the resolution M×N of the infrared thermal imager, the number of pixels on the diagonal of the image is calculated as Assume the actual length L of the diagonal at distance d is Then the corresponding relationship between the pixel grid and the actual length is expressed as Where δ is the actual length of 1 pixel, and P is the number of pixels contained in the image diagonal at distance d; According to δ and the number of pixels contained in the defective part of the hardware, the actual defect area of ​​the transmission line hardware is calculated.

9. The method for detecting damage to power transmission line hardware based on thermal imaging according to claim 8, characterized in that The power transmission line fitting damage detection model obtained in step S5 is used in step S6 to perform actual power transmission line fitting damage detection, thereby completing thermal imaging-based damage detection, which specifically includes the following steps: When actually performing the inspection, firstly, the UAV thermal imaging inspection system for the power transmission line fittings built in step S1 is used to obtain infrared thermal imaging data of the target power transmission line fittings; The acquired infrared thermal imaging data of the target transmission line fittings is input into the transmission line fittings damage detection model obtained in step S5 to obtain the actual defect area of ​​the target transmission line fittings, thereby completing the thermal imaging-based damage detection of the target transmission line fittings.

10. A system for implementing the power transmission line hardware damage detection method based on thermal imaging as claimed in any one of claims 1 to 9, characterized in that It includes a system building module, a data acquisition module, a data processing module, a model building module, a model training module and a damage detection module; the system building module, the data acquisition module, the data processing module, the model building module, the model training module and the damage detection module are connected in series in sequence; the system building module is used to build a UAV thermal imaging detection system for transmission line hardware, and upload the data information to the data acquisition module; the data acquisition module is used to perform thermal imaging of the transmission line hardware according to the received data information using the built detection system to obtain the original data set, and upload the data information to the data processing module; The data processing module is used to process the original data set obtained according to the received data information to construct a training data set, and upload the data information to the model construction module; The model building module is used to build an initial model for power transmission line hardware damage detection based on thermal imaging based on the YOLOv8 network and the variable convolutional network according to the received data information, and upload the data information to the model training module; The model training module is used to train the model constructed in step S4 according to the received data information and the obtained training data set, so as to obtain a power transmission line hardware damage detection model, and upload the data information to the damage detection module; The damage detection module is used to perform actual damage detection of transmission line fittings based on the received data information and the obtained transmission line fitting damage detection model, thereby completing damage detection based on thermal imaging.

Images