Insulator defect recognition method for uav inspection image of extra-high voltage transmission line

Abstract

The application discloses an insulator defect identification method for an unmanned aerial vehicle (UAV) inspection image of an extra-high voltage transmission line, and belongs to the technical field of transmission line inspection. The method comprises the following steps: transmitting visible light images and infrared images of insulators captured by the UAV during the inspection process to an edge computing platform on the UAV; constructing a double-branch feature extraction network architecture to extract an insulator visible light image feature map and an insulator infrared image feature map; constructing a space-time feature interaction module to output a deformed visible light feature map and an infrared time sequence feature map; fusing the deformed visible light feature map and the infrared time sequence feature map to obtain a fused feature map; inputting the fused feature map into a network model based on an encoder-decoder architecture to output a high-resolution feature map; and identifying defects based on the high-resolution feature map. The application realizes autonomous identification of insulator defects during the UAV inspection process, and improves the inspection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to a method for identifying insulator defects from images of UHV transmission line inspections by drones, belonging to the field of transmission line inspection technology. Background Technology

[0002] In ultra-high voltage transmission lines of the power system, insulators are important components for insulation, so it is necessary to inspect the insulators of ultra-high voltage transmission lines.

[0003] Traditional inspection methods often rely on manual climbing for inspection or the use of tower cranes for localized checks. This not only significantly increases maintenance costs but also poses certain safety hazards. Furthermore, their effectiveness is easily affected by weather conditions, external environment, and personnel safety. While drone inspection technology has emerged with the development of inspection methods, there is currently a lack of technology for using drones for insulator defect identification. Moreover, for insulator defects in power grid faults, the images taken by power grid drones often suffer from complex backgrounds, low contrast, and significant background variations across different regions and seasons. This makes it difficult to accurately identify insulator defects during image analysis, resulting in low accuracy in insulator defect identification.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] This invention provides a method for identifying insulator defects in UHV transmission line UAV inspection images. It combines a dual-branch feature extraction network architecture, a spatiotemporal feature interaction module, a network model based on an encoder-decoder architecture, and edge computing technology to achieve autonomous identification of insulator defects during UAV inspection, thereby improving inspection efficiency and accuracy.

[0006] The technical solution of this invention is:

[0007] A method for identifying insulator defects from UAV inspection images of ultra-high voltage transmission lines includes:

[0008] S1. Acquire visible light and infrared images of insulators captured by the drone during the inspection process, and transmit the visible light and infrared images of insulators to the edge computing platform on the drone.

[0009] S2. Construct a dual-branch feature extraction network architecture on the edge computing platform. The dual-branch feature extraction network architecture includes an improved MobileNetV3 network architecture and a lightweight ResNet variant network architecture. The improved MobileNetV3 network architecture is used to process visible light images and extract visible light image feature maps of insulators. The lightweight ResNet variant network architecture is used to process infrared images and extract infrared image feature maps of insulators.

[0010] S3. Construct a spatiotemporal feature interaction module on the edge computing platform to output deformation visible light feature map and infrared temporal feature map; fuse the deformation visible light feature map and infrared temporal feature map to obtain a fused feature map;

[0011] S4. Input the fused feature map into the network model based on the encoder-decoder architecture to output a high-resolution feature map; perform defect identification based on the high-resolution feature map.

[0012] Furthermore, the improved MobileNetV3 network architecture uses the traditional MobileNetV3 network architecture as a framework, adding an improved deformable convolutional layer after the depthwise separable convolutional layer of the traditional MobileNetV3 network architecture. The improved deformable convolutional layer introduces a dynamic offset in the offset calculation; the channel attention (SE) module is replaced with a CBAM hybrid attention module that includes channel attention and spatial attention; wherein, the dynamic offset is predicted from the input feature map of the improved deformable convolutional layer through convolution operations.

[0013] Furthermore, the lightweight ResNet variant network architecture uses the traditional ResNet18 network architecture as a framework, and replaces the standard convolutional residual blocks in the first three stages of the traditional ResNet18 network architecture with multi-scale depth-separable convolutional stacked residual blocks; and removes the last standard convolutional residual block in the traditional ResNet18 network architecture.

[0014] Furthermore, the spatiotemporal feature interaction module includes a deformable convolution with temperature gradient constraints and a bidirectional LSTM; the visible light image feature map and the infrared image feature map of the insulator are used as inputs to the deformable convolution with temperature gradient constraints to output a deformable visible light feature map; the infrared image feature map of the insulator is used as inputs to the bidirectional LSTM to output an infrared temporal feature map; wherein, the deformable convolution with temperature gradient constraints uses the visible light image feature map and the infrared image feature map of the insulator as inputs, and predicts the offset through convolution operation; temperature gradient constraints are introduced to correct the predicted offset to obtain the corrected offset; based on the corrected offset, the deformable visible light feature map is output.

[0015] Furthermore, the process of fusing the deformed visible light feature map and the infrared temporal feature map to obtain a fused feature map specifically involves:

[0016] Set range constraints for visible light weights and infrared weights; within the constraints, adaptively adjust the visible light weights and infrared weights using the signal-to-noise ratio.

[0017] The visible light feature map and the infrared time-series feature map of deformation are fused according to the visible light weight and the infrared weight to obtain a fused feature map.

[0018] The visible light weight range is [0.6, 0.8], and the infrared weight range is [0.2, 0.4]; and the sum of the visible light weight and the infrared weight is 1.

[0019] Furthermore, the network model based on the encoder-decoder architecture is U-Net.

[0020] Furthermore, the defect identification based on high-resolution feature maps includes:

[0021] The segmentation head is accessed, and the probability of each pixel in the fused feature map belonging to a hot spot is predicted based on the segmentation head; a binary mask is generated based on the probability of the hot spot; and the area of ​​the hot spot is obtained based on the binary mask.

[0022] The regression head is accessed, and the temperature value of each pixel in the fused feature map is predicted based on the regression head to obtain the average temperature of the hot spot.

[0023] Access the classification header and predict the defect category based on the classification header.

[0024] Furthermore, it also includes: calculating the insulator defect severity index based on the predicted hot spot area and average hot spot temperature.

[0025] The beneficial effects of this invention are as follows: By combining a dual-branch feature extraction network architecture, a spatiotemporal feature interaction module, a network model based on an encoder-decoder architecture, and edge computing technology, this invention enables real-time processing of image data during UHV transmission line UAV inspections. This significantly improves the efficiency and accuracy of defect identification while reducing computational resource consumption. Compared with traditional manual inspection methods, this invention offers advantages in automation, efficiency, and intelligence, and can be effectively used for intelligent inspection and closed-loop defect management of UHV transmission lines, greatly improving the operation and maintenance efficiency and safety of UHV transmission lines. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0027] Figure 2 This is a schematic diagram of the dual-branch feature extraction network architecture in this invention.

[0028] Figure 3 These are example images of visible light and infrared images collected by a drone according to an embodiment.

[0029] Figure 4 It is a deformable convolution sampling region offset map with temperature gradient constraints provided in the embodiment.

[0030] Figure 5 It is a curve showing the change of local temperature of the insulator over time. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0032] Example 1: As Figures 1-5 As shown, a method for identifying insulator defects from UAV inspection images of ultra-high voltage transmission lines includes:

[0033] S1. Acquire visible light and infrared images of insulators taken by the drone during the inspection process, and transmit the visible light and infrared images of insulators to the edge computing platform on the drone.

[0034] Furthermore, the acquisition of visible light and infrared images of insulators captured by the UAV during the inspection process specifically involves the following: During the inspection along the designated route, the UAV simultaneously acquires visible light and infrared images of the insulators using its onboard multispectral imaging system. The pre-stored image data is then transmitted to the edge computing platform via a dual-channel gigabit Ethernet interface. The multispectral imaging system employs an adaptive sampling strategy to control the image acquisition frequency. Specifically, the sampling frequency (0.5~5Hz) is dynamically adjusted based on the UAV's flight speed. For example, when the UAV is flying at high speed (e.g., exceeding 10m / s), the sampling frequency increases to ensure image continuity; when the UAV is hovering or flying at low speed (e.g., below 2m / s), the sampling frequency decreases to avoid data redundancy. Additionally, when the edge computing platform temperature exceeds 65℃, the system automatically reduces the image acquisition frequency to decrease the computational load and prevent overheating.

[0035] For example: the multispectral imaging system carried by the UAV includes a high-resolution visible light camera and a mid-wave infrared thermal imager.

[0036] S2. Construct a dual-branch feature extraction network architecture on the edge computing platform. This dual-branch feature extraction network architecture includes an improved MobileNetV3 network architecture and a lightweight ResNet variant network architecture. The improved MobileNetV3 network architecture is used to process visible light images and extract visible light image feature maps of insulators. The lightweight ResNet variant network architecture is used to process infrared images and extract infrared image feature maps of insulators. Specifically, this includes:

[0037] Construct a dual-branch feature extraction network architecture on an edge computing platform, such as Figure 2 As shown.

[0038] The improved MobileNetV3 network architecture is used to extract visible light image features, optimize computational efficiency, and is suitable for real-time processing of high-resolution visible light images.

[0039] The traditional MobileNetV3 network architecture mainly includes a first convolutional layer, a linear bottleneck layer (i.e., a Bneck structure), a depthwise separable convolutional layer (composed of "depthwise convolution" and "pointwise convolution"), a channel attention (SE) module, and pointwise convolutional layers. Among these, the depthwise separable convolution is the core component of MobileNetV3, decomposing the traditional standard convolution into two independent steps: first, the depthwise convolution independently extracts spatial features for each input channel; subsequently, the pointwise convolution (i.e., 1x1 convolution) handles row linear combination and dimensionality reduction fusion. This decomposition strategy significantly reduces redundant computation and the number of parameters, thereby greatly reducing the computational cost and parameter volume of the model. To better suit edge computing platforms, this invention improves the traditional MobileNetV3 network architecture in two aspects, obtaining an improved MobileNetV3 network architecture, the specific architecture of which is as follows: Figure 2 As shown, this is to further improve its performance in insulator defect identification.

[0040] The improved MobileNetV3 network architecture is mainly reflected in two aspects: (1) An improved deformable convolutional layer is added after the depthwise separable convolutional layer of the traditional MobileNetV3 network architecture. The improved deformable convolutional layer introduces a dynamic offset in the offset calculation. (2) Replace the channel attention SE module with a CBAM hybrid attention module that includes channel attention and spatial attention. The CBAM hybrid attention module performs channel dimension compression and spatial dimension feature recalibration on the input features in sequence.

[0041] The improved deformable convolutional layer specifically involves introducing dynamic offsets through offsets. This allows the convolutional kernel to adaptively adjust its receptive field shape based on the input feature map, resulting in an improved deformable convolutional layer. Output feature map at the location It can be represented as:

[0042] ;

[0043] in, It is the input feature map of the improved deformable convolutional layer; Is the convolution kernel at position Weight at each location; It is a basic reference location for the input feature map of the improved deformable convolutional layer; It is the offset corresponding to the k-th position of the convolution kernel; It is the size of the convolution kernel; It is the dynamic offset predicted from the input feature map X of the improved deformable convolutional layer through convolution operations.

[0044] Introducing improved deformable convolution at this stage enables more precise dynamic adjustment of the receptive field, integration of global and local information, thereby enhancing feature representation capabilities and improving pixel classification accuracy.

[0045] To further enhance the model's feature extraction capabilities, this invention replaces the original channel attention (SE) module with a CBAM (Convolutional Block Attention Module) hybrid attention module, which incorporates both channel and spatial attention. Compared to the SE module, which focuses solely on channel-level attention, the CBAM hybrid attention module also integrates spatial attention, enabling adaptive weighting of input features across both channel and spatial dimensions. This more effectively strengthens important features while suppressing irrelevant ones. The processing flow of the CBAM hybrid attention module can be represented in two steps:

[0046] ;

[0047] ;

[0048] in, This is the input feature map for the CBAM hybrid attention module; The intermediate feature map is used as input to the CBAM hybrid attention module; and These are the channel attention module and the spatial attention module, respectively. This is the final output feature map of the CBAM hybrid attention module; This indicates element-wise multiplication.

[0049] Then the feature map output by the CBAM hybrid attention module is used. Input point convolutional layer, through Convolution kernel pair The channel information is linearly combined and dimensionality-reduced to obtain the final visible light image feature map of the insulator. Furthermore, a lightweight ResNet variant network architecture is proposed to adapt to low-resolution feature extraction from infrared images.

[0050] It should be noted that the traditional ResNet18 network architecture mainly includes an initial feature extraction module and four stages of standard convolutional residual block stacking (the initial feature extraction module contains a 7x7 standard convolutional layer and a max pooling layer). This invention modifies the traditional ResNet18 network architecture by: ① replacing the standard convolutional residual blocks (e.g., ConvBlock_1, ConvBlock_2, and ConvBlock_3) in the first three stages of the original network with multi-scale depthwise separable convolutional stacked residual blocks; ② deleting the last standard convolutional residual block in the original network, further reducing model depth and computational cost.

[0051] The multi-scale depthwise separable convolution stacked residual block in the lightweight ResNet variant network architecture is composed of multiple multi-scale depthwise separable convolutions, which include two parts: multi-scale depthwise convolutions and pointwise convolutions.

[0052] ① Multi-scale depth convolution: Using convolution kernels of different sizes (3×3, 5×5, 7×7, etc.) to extract spatial features from the input feature map independently, and then concatenating the results of these depth convolutions at different scales along the channel dimension.

[0053] ② Pointwise convolution: A 1×1 convolution kernel is used to linearly fuse the outputs of multi-scale depthwise convolution convolutions. Specifically, the number of channels after concatenation is... Perform pointwise convolution operations on the feature maps, using a size of The pointwise convolutional kernel linearly fuses feature information at different scales and increases the number of channels in the feature map from... Adjusted to ;in, The number of convolution kernels for multi-scale depthwise convolutions. The number of channels in the input feature map. This represents the number of channels in the output feature map.

[0054] The lightweight ResNet variant architecture of this invention significantly reduces the computational cost and number of parameters of the model through multi-scale depthwise separable convolution operations. For the input feature map... After this multi-scale depthwise separable convolution process, the output feature map is obtained. Then, the expression for multi-scale depthwise separable convolution can be functionally represented as:

[0055] ;

[0056] in, The input feature map is a multi-scale depthwise separable convolution. This indicates that the input feature map Y for multi-scale depthwise separable convolution uses a size of depthwise convolution kernels Perform depthwise convolution operations ( This operation performs convolution independently on each input channel while keeping the number of channels constant (i.e., the number of channels in the output feature map is the same as the number of channels in the input feature map). equal); This represents the concatenation operation performed along the channel dimension. After concatenating the outputs of multi-scale depthwise convolution, the number of channels becomes... ( This refers to the number of convolution kernels in multi-scale depthwise convolution, which is taken as 3 in this embodiment of the invention, corresponding to 3×3, 5×5, and 7×7 convolution kernels. This indicates that a pointwise convolution (1x1 convolution) operation is performed on the concatenated feature map, using... Pointwise convolution kernel The number of channels will be reduced from Adjusted to .

[0057] Applying the above scheme, the lightweight ResNet variant network architecture processes the input raw infrared image through an initial feature extraction module, followed by three cascaded multi-scale depthwise separable convolutional stacked residual blocks. Multi-scale depthwise separable convolutional stacked residual block 1 receives the initial features extracted by the initial feature extraction module and outputs F1; multi-scale depthwise separable convolutional stacked residual block 2 receives F1 and outputs F2; ​​and multi-scale depthwise separable convolutional stacked residual block 3 receives F2 and outputs F3. Through these multi-scale depthwise separable convolutional stacked residual blocks, not only is the size of the feature map gradually reduced, but the feature map becomes more refined and abstract by increasing the channel dimension and capturing more complex semantic information. Finally, the highest-level feature map F3 is used as the feature map of the insulator infrared image, which is represented in the subsequent spatiotemporal feature interaction module as follows: .

[0058] S3. Construct a spatiotemporal feature interaction module, which includes a deformable convolution with temperature gradient constraints and a bidirectional LSTM. The visible light image feature map and the infrared image feature map of the insulator are used as inputs to the deformable convolution with temperature gradient constraints to output a deformable visible light feature map. The infrared image feature map of the insulator is used as inputs to the bidirectional LSTM to output an infrared temporal feature map. An adaptive weighting mechanism is designed to fuse the deformable visible light feature map and the infrared temporal feature map to obtain a fused feature map.

[0059] Specifically:

[0060] The receptive field is dynamically adjusted using deformable convolution with temperature gradient constraints, and its input is the visible light image feature map of the insulator. Infrared image feature map of insulator Predicting offsets through convolution operations :

[0061] ;

[0062] in, This indicates channel concatenation, and the convolution kernel weights are optimized through backpropagation.

[0063] By introducing a temperature gradient constraint, the predicted offset is corrected to obtain the corrected offset. :

[0064] ;

[0065] ;

[0066] in, For local temperature, It is a sigmoid function. For heat sensitivity coefficient, As a dynamic environmental reference temperature, This represents the natural constant. Temperature gradient constraints enable the sampling points of the convolution kernel to be more densely distributed in regions with large temperature gradients (such as the edges of hot spots), thereby more accurately capturing the geometric correspondence between hot spots and visible light defects and improving the accuracy of feature matching.

[0067] By applying the corrected offset, the final output is a deformed visible light feature map:

[0068] ;

[0069] in, It is a deformable convolution with temperature gradient constraints at position Output feature map at that location, It is the input visible light image feature map of the insulator. Is the convolution kernel at position The weight of the position, It is the first convolution kernel The offset corresponding to each position It is the first convolution kernel The offset after position correction It is the size of the convolution kernel.

[0070] The infrared image feature map of the insulator was analyzed using a bidirectional LSTM, specifically as follows:

[0071] The input insulator infrared image feature map sequence is represented as follows: ,in This indicates the maximum number of input sequences. For time step Infrared image feature map of insulator. Temporal dependencies are extracted using bidirectional LSTM, expressed as:

[0072] ;

[0073] in, Indicates at time step At this point, the forward LSTM unit receives the currently input infrared image feature map. The hidden state of the previous time step Output the current hidden state. As the output of the forward LSTM layer; Indicates at time step At this point, the inverse LSTM unit receives the feature map of the currently input infrared image. and the hidden state of the next time step Output the current hidden state. As the output of the inverse LSTM layer; Indicates time step The output infrared timing characteristic map.

[0074] Next, this invention designs an adaptive weighting mechanism for feature fusion: visible light weighting. and infrared weight Adaptive adjustment via signal-to-noise ratio (SNR).

[0075] ;

[0076] in, For the Sigmoid function, and The signal-to-noise ratios of the visible light feature map and the infrared temporal feature map, respectively, are used to calculate the deformation. Given a 1×2 row vector, It is a constant.

[0077] Set weight range constraints; visible light weights At this time, it can achieve the purpose of prioritizing the use of high-resolution textures for defect judgment; infrared weighting At the same time, it can further assist in judging internal defects.

[0078] Weight fusion is performed using the following formula:

[0079] ;

[0080] in, To fuse feature maps; Right now , Right now .

[0081] S4. Input the fused feature map into the network model based on the encoder-decoder architecture to output a high-resolution feature map; perform defect identification based on the high-resolution feature map.

[0082] Specifically: merging feature maps The input is a U-Net based on an encoder-decoder architecture. The U-Net encoder first extracts high-level features and downsamples them, then the decoder recovers the spatial resolution of the feature map through upsampling and skip connections. Finally, the U-Net decoder outputs a high-resolution feature map. Based on high-resolution feature maps Defect identification:

[0083] ① Predict the probability that each pixel in the fused feature map belongs to a hotspot based on the segmentation head; generate a binary mask based on the probability of the hotspot; obtain the hotspot area based on the binary mask;

[0084] exist Based on this, a segmentation head is added to predict the probability that each pixel belongs to a "hotspot". The expression is:

[0085] ;

[0086] in, For pixels Category The probability of; Use the Softmax activation function; This indicates the convolution operation in the segmentation head.

[0087] Binary mask generation:

[0088] ;

[0089] in, For pixels Whether it belongs to the binary mask of the hotspot (1 for hotspot, 0 for background); The probability threshold for segmentation tasks; : Index corresponding to the hot spot category.

[0090] Hot spot area calculation:

[0091] ;

[0092] in, This represents the actual hot spot area; This represents the actual physical size corresponding to a single pixel.

[0093] ②The average temperature of the hot spot is obtained by predicting the temperature value of each pixel in the fused feature map based on the regression head;

[0094] exist Based on this, a regression head is connected to predict the temperature value of each pixel. The expression is:

[0095] ;

[0096] in, This indicates the convolution operation in the regression head.

[0097] ③ Predict the defect category based on the classification header;

[0098] exist A separate classification head is then built on top of this to predict the specific category of the defect. This classification head typically consists of a global pooling layer and a fully connected layer.

[0099] ;

[0100] ;

[0101] in, To represent categorical features, it is a feature vector processed by global average pooling; This is a global average pooling layer; One or more fully connected layers in the classification head map global features to the log odds of defect categories; Predicting defects belongs to category The probability of.

[0102] Final Defect Categories :

[0103] ;

[0104] in, Iterate through all possible categories Find it The category that reaches the maximum value .

[0105] ④ Calculate the insulator defect severity index based on the predicted hot spot area and average hot spot temperature:

[0106] ;

[0107] in, This is a defect severity index. For material property coefficients, As a dynamic environmental reference temperature, This is the measurement error term; This indicates the average temperature of the hot spot; Indicates the predicted hotspot area .

[0108] In summary, this invention can comprehensively and quantitatively assess the defects in insulators: the types of defects are determined by... The severity of the defect is predicted to be determined by... Value quantization is used, while the location of the defect is determined by a hotspot segmentation mask. It provides precise indication and can further extract the coordinates of its centroid or bounding box.

[0109] Furthermore, it also includes S5, which feeds back the diagnostic results (defect category, defect severity index, defect location) obtained in S4 to the inspection personnel or the operation and maintenance management system of the transmission line to achieve closed-loop management of defects.

[0110] Example 2: The following describes an optional embodiment of the insulator defect identification method for UHV transmission line UAV inspection images provided by the present invention, with reference to the accompanying drawings:

[0111] I. Inspection Scenarios and Data Collection

[0112] This example demonstrates an autonomous UAV inspection of an ultra-high-voltage transmission line. The inspection target is a section of transmission line located in a suburban area that contains various types of insulators. The UAV is equipped with a multispectral imaging system, which simultaneously acquires visible light and infrared images of the insulators during the inspection.

[0113] 1.1 UAV Inspection Parameters

[0114] The drone flies autonomously according to the preset inspection route. The parameters of the drone at different flight stages are shown in Table 1.

[0115] Table 1. Relevant parameters of UAVs

[0116]

[0117] 1.2 Visible and Infrared Images

[0118] like Figure 3 The image shown is a visible light image and an infrared image obtained from a drone inspection. Figure 3The infrared image shows a hot spot with an abnormal temperature in a localized area.

[0119] 1.3 Data Preprocessing and Transmission

[0120] The acquired visible light and infrared images were formatted (PNG format) and annotated with metadata (timestamp, GPS coordinates, camera parameters, etc.) on the drone. The image data was then compressed using H.265 and transmitted to the edge computing platform via a dual-channel gigabit Ethernet interface. Considering real-time requirements, infrared channel data was prioritized for transmission. A circular buffer pre-stored the most recent 30 seconds of data to ensure no data loss during transmission congestion. The edge computing platform temperature was maintained at a stable 55°C, below the 65°C threshold, and the image acquisition frequency was kept at a normal level (3Hz).

[0121] II. Dual-branch feature extraction network architecture and spatiotemporal feature interaction module

[0122] After receiving the image data, the edge computing platform immediately starts the dual-branch feature extraction network architecture and the spatiotemporal feature interaction module for processing.

[0123] 2.1 Dual-branch feature extraction network architecture

[0124] The edge computing platform has been loaded and initialized as follows: Figure 2 The dual-branch feature extraction network architecture is shown. For visible light images, an improved MobileNetV3 network architecture is used; for infrared images, a lightweight ResNet variant network architecture is used.

[0125] Subsequently, the visible light image feature map and the infrared image feature map of the insulator are fused through the spatiotemporal feature interaction module.

[0126] 2.2 Spatial Dimension Analysis: Deformable Convolution with Temperature Gradient Constraints

[0127] To accurately align a location in a visible light image with a hotspot region in an infrared image, this invention introduces deformable convolution with temperature gradient constraints, such as... Figure 4 This demonstrates the process of shifting the deformable convolution sampling region with temperature gradient constraints, from... Figure 4 It can be seen that the original sampling region is modified by deformable convolution with temperature gradient constraint. It can be seen that at the edge of the hot spot region (high temperature, and the gradient may also be high), the sampling region is guided to the region with more drastic temperature change. This helps the model to more accurately capture the geometric correspondence between the hot spot and visible light defects.

[0128] 2.3 Time Dimension Analysis: Bidirectional LSTM

[0129] To capture the timing patterns of insulator degradation, especially slow-developing defects (such as persistent heating caused by flashover), a bidirectional LSTM was used to analyze the infrared timing characteristics of 5-8 consecutive frames. The temperature change at a point on the insulator over 8 consecutive frames (approximately 2.6 seconds, sampling frequency 3Hz) was used to demonstrate how LSTM captures timing patterns. Figure 5 The curves showing the local temperature change of the insulator over time are presented. Figure 5 It can be seen that the local temperature of the insulator initially showed a slight upward trend, then stabilized, but remained significantly higher than the ambient reference temperature. The bidirectional LSTM was able to learn this temperature dependence before and after, determining that this was a persistent anomalous hotspot rather than a transient disturbance.

[0130] 2.4 Adaptive Weight Fusion

[0131] After feature extraction and spatiotemporal interaction are completed, an adaptive weighting mechanism is used to fuse the deformed visible light feature map and the infrared temporal feature map for feature extraction. The weights are adaptively adjusted through the signal-to-noise ratio (SNR), and the SNR and calculated fusion weights are shown in Table 2.

[0132] Table 2 Signal-to-noise ratio and calculated fusion weights

[0133]

[0134] III. Defect Diagnosis and Quantification

[0135] The fused feature map is input into U-Net, which then executes the segmentation head, regression head, and classification head in parallel, and finally calculates the defect severity index.

[0136] 3.1. Based on the segmentation head, predict the probability that each pixel in the fused feature map belongs to a hotspot; generate a binary mask based on the hotspot probability; the hotspot attribute parameters obtained from the binary mask are shown in Table 3:

[0137] Table 3 Hot Spot Properties

[0138]

[0139] 3.2. Based on the regression head, predict the temperature value of each pixel in the fused feature map, and calculate the average temperature as shown in Table 4:

[0140] Table 4 Average Temperature

[0141]

[0142] 3.3 The defect category results obtained based on the classification head prediction are shown in Table 5. As can be seen from Table 5, the probability of cracks in the insulator is the highest, and the defect category identified is crack.

[0143] Table 5 Defect Categories

[0144]

[0145] 3.4 Defect Severity Index Calculation: The calculated insulator defect severity index is shown in Table 6.

[0146] Table 6 Defect Severity Index Calculation

[0147]

[0148] 3.5 Insulator defect diagnosis: The results of the insulator defect diagnosis are shown in Table 7.

[0149] Table 7 Insulator Defect Diagnosis Results

[0150]

[0151] As can be seen from the above technical solution, the present invention realizes the autonomous identification of insulator defects during UAV inspection, thereby improving inspection efficiency and accuracy, and is applicable to intelligent inspection and defect closed-loop management scenarios of transmission lines.

[0152] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for identifying insulator defects from UAV inspection images of ultra-high voltage transmission lines, characterized in that, The method comprises the following steps: S1, acquiring visible light images and infrared images of insulators taken by a UAV during inspection, and transmitting the visible light images and infrared images of insulators to an edge computing platform on the UAV; S2, constructing a double-branch feature extraction network architecture on the edge computing platform, the double-branch feature extraction network architecture comprising an improved MobileNetV3 network architecture and a lightweight ResNet variant network architecture; processing the visible light images by the improved MobileNetV3 network architecture to extract visible light image feature maps of insulators; processing the infrared images by the lightweight ResNet variant network architecture to extract infrared image feature maps of insulators; S3, constructing a space-time feature interaction module on the edge computing platform to output a deformed visible light feature map and an infrared time-series feature map; and performing feature fusion on the deformed visible light feature map and the infrared time-series feature map to obtain a fused feature map; S4, inputting the fused feature map into a network model based on an encoder-decoder architecture to output a high-resolution feature map; performing defect recognition based on the high-resolution feature map; The improved MobileNetV3 network architecture takes a conventional MobileNetV3 network architecture as a framework, and an improved deformable convolution layer is added after a depth separable convolution layer of the conventional MobileNetV3 network architecture, wherein the improved deformable convolution layer introduces a dynamic offset for offset calculation; The channel attention SE module is replaced by a CBAM hybrid attention module comprising channel attention and spatial attention; wherein the dynamic offset is predicted from the input feature map of the improved deformable convolution layer through convolution operation; The lightweight ResNet variant network architecture takes a conventional ResNet18 network architecture as a framework, and a multi-scale depth separable convolution stacking residual block is used to replace the standard convolution residual blocks in the first three stages of the conventional ResNet18 network architecture; and the last standard convolution residual block in the conventional ResNet18 network architecture is deleted; The space-time feature interaction module comprises a deformable convolution with temperature gradient constraint and a bidirectional LSTM; the visible light image feature map of insulators and the infrared image feature map of insulators are taken as inputs of the deformable convolution with temperature gradient constraint to output a deformed visible light feature map; and the infrared image feature map of insulators is taken as an input of the bidirectional LSTM to output an infrared time-series feature map; wherein the deformable convolution with temperature gradient constraint takes the visible light image feature map of insulators and the infrared image feature map of insulators as inputs, and predicts an offset through convolution operation; a temperature gradient constraint is introduced to correct the predicted offset to obtain a corrected offset; and the deformed visible light feature map is output according to the corrected offset; The feature fusion of the deformed visible light feature map and the infrared time-series feature map to obtain the fused feature map is specifically as follows: The range constraint of visible light weight and infrared weight is set; and the visible light weight and the infrared weight are adaptively adjusted through signal-to-noise ratio within the constraint range. The visible light deformation feature map and the infrared time sequence feature map are fused according to a visible light weight and an infrared weight to obtain a fused feature map; The defect recognition based on the high-resolution feature map comprises: An access segmentation head is connected, and the probability of each pixel of the fused feature map belonging to the hot spot is predicted according to the segmentation head; a binary mask is generated according to the probability of the hot spot; and the hot spot area is obtained according to the binary mask; An access regression head is connected, and the temperature value of each pixel of the fused feature map is predicted according to the regression head to obtain the average temperature of the hot spot; An access classification head is connected, and the defect category is predicted according to the classification head.

2. The method of claim 1, wherein the method further comprises: The visible light weight ranges from 0.6 to 0.8, the infrared weight ranges from 0.2 to 0.4, and the sum of the visible light weight and the infrared weight is 1.

3. The method of claim 1, wherein the method further comprises: The network model based on the encoder-decoder architecture is a U-Net.

4. The insulator defect identification method for UHV transmission line unmanned aerial vehicle inspection images according to claim 1, characterized in that, Further comprising: According to the predicted hot spot area and the average temperature of the hot spot, the insulator defect severity index is calculated.

Images