Intelligent diagnosis method for substation equipment infrared image thermal fault based on deep learning

Abstract

The application discloses a kind of based on deep learning's substation equipment infrared image thermal fault intelligent diagnosis method, provide a kind of based on deep learning's substation equipment infrared image thermal fault intelligent diagnosis method, for the defects that existing substation infrared image exists picture quality blur, shield small target recognition is poor, small heating fault segmentation precision is low, diagnosis relies on artificial experience, design three-layer progressive lightweight deep learning network, realize low-quality infrared image self-adaptive enhancement, substation equipment multi-target accurate identification, suspected thermal fault area pixel-level segmentation in turn, supporting standardized relative temperature difference fault grading algorithm, build full-automatic integrated diagnosis process.Can embedded deployment, replace artificial inspection, greatly reduce misdiagnosis, for 110kV and above substation intelligent operation and maintenance.

Description

Technical Field

[0001] This invention belongs to the field of intelligent power inspection technology, specifically relating to an intelligent diagnostic method for thermal faults in substation equipment based on deep learning using infrared images. Background Technology

[0002] Substations are a critical component of power systems, and the safe operation of their equipment directly affects the stability of the power system. Many substation equipment operate continuously under harsh natural environments and high current and voltage conditions, making them prone to various faults such as insulation aging, poor contact, and insulator damage. Research shows that substation equipment faults are usually accompanied by abnormal heating of the entire equipment or specific areas; therefore, fault analysis and judgment can be performed based on the thermal state of the equipment.

[0003] Infrared thermal imaging technology, as a non-contact condition sensing technology, can accurately acquire temperature information of equipment components without background interference, and has been widely used in the daily inspection of substation equipment. However, existing technologies still have the following shortcomings: Firstly, there is the problem of low infrared image quality. During the acquisition of infrared images for substation equipment, factors such as shooting angle, shooting distance, atmospheric transmittance, and insufficient exposure time can easily lead to low-quality issues like low contrast and blurred details. Existing image enhancement methods, such as histogram equalization, Retinex algorithm, and guided filtering, are prone to introducing noise, have weak generalization ability, and are not effective in enhancing detailed features and image edge information. Deep learning-based enhancement methods, on the other hand, lack sufficient paired infrared image datasets, resulting in limited enhancement effectiveness. Although some research has attempted to apply CycleGAN to infrared image enhancement, the original CycleGAN model has insufficient ability to extract small target features and texture information in the encoding network, and the residual structure in the transcoding network has limited ability to map global information of the infrared image, leading to blurred details and unsatisfactory enhancement results in the generated image.

[0004] Secondly, there is the problem of low equipment recognition accuracy in complex backgrounds. Infrared images of substation equipment often contain multiple targets, small targets, and occluded targets, making it difficult for existing target recognition algorithms to accurately identify and locate all equipment. Two-stage detection models such as Faster R-CNN, while highly accurate, suffer from high computational cost and slow inference speed; one-stage detection models such as the YOLO series, while fast, lack sufficient accuracy in recognizing small and occluded targets. Although CenterNet-based algorithms do not rely on anchor boxes, their feature extraction capabilities for multiple, small, and occluded targets in complex backgrounds remain limited.

[0005] Third, there is the challenge of balancing accuracy and efficiency in segmenting suspected fault areas. Suspected fault areas in infrared images of substation equipment are characterized by irregular shapes, small size, and indistinct edge information. Existing segmentation methods, such as clustering, thresholding, and superpixel segmentation, suffer from low segmentation accuracy and poor generalization. While deep learning methods offer higher accuracy, they involve a large number of model parameters and high computational complexity, making it difficult to achieve both high segmentation accuracy and lightweight model design.

[0006] Fourth, there is a lack of end-to-end automated diagnostic solutions. In existing technologies, image enhancement, device identification, fault area segmentation, and fault diagnosis are usually implemented as independent modules, lacking a complete technical solution that integrates them organically. This makes it difficult to achieve fully automated diagnosis from low-quality infrared image input to fault level output. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing a deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment. This method aims to solve the following technical problems: low contrast and blurred details in infrared images of low-quality substation equipment; low recognition accuracy of multiple targets, small targets, and occluded targets in complex backgrounds; difficulty in balancing the segmentation accuracy of suspected fault areas with model lightweighting; and the lack of an end-to-end automated thermal fault diagnosis scheme.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for intelligent diagnosis of thermal faults in substation equipment based on infrared images using deep learning, the method comprising the following steps: S1. Dataset Construction and Multi-Network Transfer Learning Training: Collect infrared images of various types of equipment in the substation, divide them into training set, validation set, and test set, and train the image enhancement network, equipment recognition network, and fault area segmentation network respectively. S2. Adaptive Infrared Image Enhancement: The low-quality original infrared images of the substation collected on-site are input into the trained improved self-attention deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images, CycleGAN, and the image enhancement network outputs enhanced infrared images with high contrast and clear details. S3. Precise multi-target identification of substation equipment: The improved deep learning-based intelligent diagnosis method for thermal faults in substation equipment based on infrared images is trained by CenterNet, an equipment identification network based on deep learning for infrared images of substation equipment. It identifies all substation equipment categories in the image and outputs independently cropped images of each equipment. S4. Lightweight pixel segmentation of suspected thermal fault areas: Input the cropped image of a single device into the trained lightweight improved deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment YOLOv8, and extract all suspected thermal fault areas on the surface of the equipment at the pixel level. S5. Infrared image temperature value extraction: Based on the infrared image grayscale and temperature mapping standard, extract the highest temperature of the suspected fault area, the reference temperature of the normal equipment area, and the actual measured temperature of the on-site environment. S6. Automatic thermal fault classification and diagnosis: The relative temperature difference method is used to calculate the abnormal temperature rise difference of the equipment, and the fault level of the equipment is automatically determined by combining the infrared diagnostic standards for live equipment in the power industry, and a standardized operation and maintenance diagnosis report is generated.

[0009] The aforementioned intelligent diagnosis method for thermal faults in substation equipment based on infrared images using deep learning, specifically the improved self-attention method based on deep learning for intelligent diagnosis of thermal faults in substation equipment using CycleGAN, includes the following improved structure for the image enhancement network: an efficient local attention module embedded within the generator, a fusion structure of the self-attention module and the feature pyramid; a dual-channel parallel feature extraction network used in the discriminator; and simultaneous optimization of the overall network loss function using the square term, eliminating logarithmic operations to accelerate model convergence.

[0010] The aforementioned intelligent diagnostic method for thermal faults in substation equipment based on infrared images using deep learning involves an efficient local attention module that extracts long-range features in the horizontal and vertical directions of the infrared image through one-dimensional horizontal global pooling and one-dimensional vertical global pooling, respectively. Spatial attention weights are generated through one-dimensional convolution, group normalization, and activation functions, and then multiplied element-wise with the original feature map to enhance detailed features.

[0011] The aforementioned intelligent diagnosis method for thermal faults in substation equipment based on deep learning infrared images, specifically the improved structure of the equipment identification network in the CenterNet method, includes: an adaptive spatial feature fusion module and a feature pyramid multi-scale fusion structure integrated at the back end of the backbone network; a global attention module embedded in the feature fusion network; lightweighting by replacing all standard convolutions in the network with depthwise separable convolutions; and replacing the original network loss function with a combination of distributed focal loss and smoothing loss.

[0012] The aforementioned intelligent diagnostic method for thermal faults in substation equipment based on infrared images using deep learning includes a global attention module divided into a channel attention branch and a spatial attention branch. The channel attention branch uses three-dimensional feature permutation and a multilayer perceptron to capture cross-channel dependencies, while the spatial attention branch enhances the spatial saliency of the equipment target within the image. The dual branches jointly suppress invalid feature weights in complex backgrounds.

[0013] The aforementioned intelligent diagnosis method for infrared image thermal faults in substation equipment based on deep learning, specifically the improved structure of the segmentation network in the lightweight improved method based on deep learning (YOLOv8) for infrared image thermal fault diagnosis of substation equipment, includes: replacing the original YOLOv8 method's main segmentation network with a lightweight inverse residual network based on MobileNetV3. Dry network; introduce semantic detail injection module into network neck; replace the original deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment (C2f) with an upsampled inverse bottleneck fusion structure (C2f_UIB); replace the original mask loss and classification loss with a combination of focus loss and similarity coefficient loss (Dice).

[0014] The aforementioned intelligent diagnostic method for thermal faults in substation equipment based on infrared images, which is based on deep learning, involves a semantic detail injection module that performs multi-scale interactive operations such as downsampling, identity mapping, and upsampling on features at different levels. At the same time, it embeds dual channel and spatial attention at the front end of each level of features to achieve complete cross-scale feature extraction of irregular micro-fault regions.

[0015] The above-mentioned intelligent diagnostic method for thermal faults in substation equipment based on deep learning infrared images, in step S5 of the intelligent diagnostic method for thermal faults in substation equipment based on deep learning infrared images, adopts the infrared image grayscale and temperature mapping standard GB / T 44006-2024, which is based on the method for intelligent diagnostic method for thermal faults in substation equipment based on deep learning infrared images. The method uses the RGB temperature representation rules of the infrared image based on the image grayscale level, the highest temperature in the image temperature measurement range, and the lowest temperature to calculate the actual temperature value corresponding to each pixel.

[0016] The aforementioned intelligent diagnostic method for thermal faults in substation equipment based on deep learning, specifically step S6, involves the following grading logic for the relative temperature difference method: If the relative temperature difference is less than 35%, the equipment is deemed fault-free; if the relative temperature difference is greater than or equal to 35% and less than the infrared image thermal fault value of the substation equipment, the equipment is deemed fault-free. The intelligent diagnostic method for thermal faults in substation equipment based on infrared images, which is used in 80% of cases, is classified as a general fault. A relative temperature difference greater than or equal to 80% and less than 95% is classified as a serious fault. A relative temperature difference greater than or equal to 95% is classified as an emergency fault.

[0017] The aforementioned intelligent diagnostic method for thermal faults in substation equipment based on infrared images, which is based on deep learning, is deployed on a substation inspection terminal, an airborne edge computing module of a UAV, or a back-end server. After batch input of infrared images, it automatically executes the entire process of image enhancement, equipment identification, fault segmentation, temperature conversion, and fault classification in sequence. It does not require manual annotation or interpretation and directly outputs a structured diagnostic report containing equipment type, fault area location, relative temperature difference value, fault level, and maintenance suggestions.

[0018] Compared with the prior art, the present invention has the following advantages: 1. In terms of infrared image enhancement, it significantly improves the quality of low-quality images.

[0019] This invention enhances the extraction capability of infrared image texture information and edge detail features by introducing an efficient local attention mechanism (ELA) into the encoding network of the CycleGAN generator; it achieves cross-layer feature fusion of shallow device contour information and deep device texture information by constructing a feature pyramid structure (FPN) between the encoding and decoding networks, effectively solving the problem of missing content in the generated image; it improves the network's ability to map global features of infrared images and improves image contrast by introducing a self-attention mechanism to the transcoding network instead of the traditional residual structure, while reducing the number of model parameters; it enhances the discriminator's ability to extract image difference information by constructing a dual-channel discriminator; and it accelerates model convergence by optimizing the loss function to a squared form. Experimental results show that the low-quality infrared images processed by the algorithm of this invention show an average improvement of chroma, saturation, and feature similarity indices of 10.91%, 18.89%, and 29.82%, respectively, while the number of model parameters is reduced by 37.89% compared to the original algorithm.

[0020] 2. In terms of device recognition, it significantly improves the recognition accuracy of multiple targets, small targets, and occluded targets in complex backgrounds.

[0021] This invention constructs an adaptive feature fusion network by combining an adaptive spatial feature fusion module (ASFF) and a feature pyramid structure (FPN), enhancing the model's ability to fuse features across scales for multiple and small targets, and effectively eliminating background information interference. By embedding a global attention mechanism (GAM) into the feature fusion network, the feature saliency of occluded and small targets is enhanced. The introduction of depthwise separable convolution (DSC) significantly reduces the number of model parameters while maintaining recognition accuracy. Furthermore, the use of distributed focus loss (DFL) instead of traditional focus loss improves the model's sensitivity to occluded targets and its convergence speed. Experimental results show that the algorithm of this invention achieves a recognition accuracy of 95.19%, a 3.55% improvement over the original CenterNet algorithm, with only 32.52M model parameters, outperforming existing mainstream target recognition algorithms in both recognition accuracy and algorithm complexity.

[0022] 3. In terms of segmentation of suspected fault regions, a balance is achieved between segmentation accuracy and model lightweighting.

[0023] This invention introduces MobileNetV3 as the backbone network, leveraging its lightweight design to effectively reduce the number of model parameters and computational complexity. By introducing a Semantic Details Injection (SDI) module, it enhances the model's ability to extract cross-scale features from irregular, suspected fault regions. Furthermore, by introducing an upsampled inverse bottleneck feature fusion structure (C2f_UIB), it enhances the model's ability to extract edge information from suspected fault regions, effectively solving the oversegmentation problem caused by indistinct edge information in infrared images. Finally, by replacing the binary cross-entropy loss with FocalLoss and Dice Loss, it improves the model's robustness under imbalanced sample conditions. Experimental results show that the proposed algorithm achieves a segmentation accuracy of 88.81%, a 2.38% improvement over the original YOLOv8 algorithm, while the number of model parameters is only 9.17M, a 26.46% reduction compared to the original model.

[0024] 4. In terms of the overall technical solution, end-to-end automated thermal fault diagnosis has been achieved.

[0025] This invention organically integrates four aspects—image enhancement, equipment identification, suspected fault area segmentation, and thermal fault diagnosis—into a complete technical solution. It realizes a fully automated diagnostic process from low-quality infrared image input to fault level output, requiring no manual intervention. This effectively improves the efficiency and accuracy of substation equipment fault diagnosis, providing a strong guarantee for the safe and stable operation of substations.

[0026] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0027] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a diagram of the improved generator structure of the present invention; Figure 3A This is a schematic diagram of the residual structure of the present invention; Figure 3B This is a schematic diagram of the self-attention mechanism of the present invention; Figure 4 This is a schematic diagram illustrating the principle of the efficient local attention mechanism of this invention. Figure 5 This is a structural diagram of the improved discriminator of the present invention; Figure 6 This is a visualization of the enhanced effects of the present invention; Figure 7 This is a visualization of the image recognition results of the present invention; Figure 8 This is a diagram of the improved CenterNet architecture of the present invention; Figure 9 This is a schematic diagram of the ASFF principle of the present invention; Figure 10 This is a schematic diagram of the working principle of GAM in this invention; Figure 11 This is a diagram illustrating the working principle of depthwise separable convolution. Figure 12 For the purposes of this invention λ size and λ off The accuracy of the value recognition; Figure 13 This is a graph showing the changes in the evaluation indicators of this invention. Figure 14 These are images showing the recognition results of different algorithms of this invention in different scenarios; Figure 15 This is a schematic diagram of the lightweight YOLOv8 model of this invention; Figure 16 This is a flowchart of the substation equipment fault diagnosis process of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0029] like Figure 1 As shown, a deep learning-based intelligent diagnostic method for thermal faults in substation equipment using infrared images includes the following steps: S1. Dataset Construction and Multi-Network Transfer Learning Training: Collect infrared images of various types of equipment in the substation, divide them into training set, validation set, and test set, and train the image enhancement network, equipment recognition network, and fault area segmentation network respectively. S2. Adaptive Infrared Image Enhancement: The low-quality original infrared images of the substation collected on-site are input into the trained improved self-attention deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images, CycleGAN, and the image enhancement network outputs enhanced infrared images with high contrast and clear details. S3. Precise multi-target identification of substation equipment: The improved deep learning-based intelligent diagnosis method for thermal faults in substation equipment based on infrared images is trained by CenterNet, an equipment identification network based on deep learning for infrared images of substation equipment. It identifies all substation equipment categories in the image and outputs independently cropped images of each equipment. S4. Lightweight pixel segmentation of suspected thermal fault areas: Input the cropped image of a single device into the trained lightweight improved deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment YOLOv8, and extract all suspected thermal fault areas on the surface of the equipment at the pixel level. S5. Infrared image temperature value extraction: Based on the infrared image grayscale and temperature mapping standard, extract the highest temperature of the suspected fault area, the reference temperature of the normal equipment area, and the actual measured temperature of the on-site environment. S6. Automatic thermal fault classification and diagnosis: The relative temperature difference method is used to calculate the abnormal temperature rise difference of the equipment, and the fault level of the equipment is automatically determined by combining the infrared diagnostic standards for live equipment in the power industry, and a standardized operation and maintenance diagnosis report is generated.

[0030] In this embodiment, the improved structure of the image enhancement network of the CycleGAN deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment includes: embedding an efficient local attention module inside the generator, a fusion structure of the self-attention module and the feature pyramid; the discriminator adopts a dual-channel parallel feature extraction network; and the overall loss function of the network is optimized by squared terms, and logarithmic operations are eliminated to accelerate the convergence speed of the model.

[0031] The backbone network of the original CycleGAN generator easily loses fine feature information such as the edges and textures of substation equipment during forward propagation. Furthermore, the feature information propagation channel is singular, failing to effectively fuse shallow equipment contour information with deep equipment texture information, resulting in severe loss of detail in the generated infrared image. To address these issues, this invention proposes a generator network based on feature fusion and self-attention mechanisms, improving the generator structure as follows: Figure 2 As shown.

[0032] By constructing a feature pyramid structure between the encoding and decoding networks, multi-channel propagation and effective fusion of feature information are achieved. An efficient localization attention (ELA) mechanism is added to the encoding network, further optimizing the model's feature focusing ability during infrared image generation without significantly increasing the number of parameters or inference time. Furthermore, to further enhance the generator's ability to model device contours, a self-attention mechanism is introduced into the transcoding network.

[0033] Self-attention is a fundamental component of the Transformer. It transforms the interactions between features in the input infrared image into corresponding weights, updating each component of the sequence by integrating global information from the complete infrared image sequence. The original CycleGAN model uses a residual structure in the transcoding network to map image style features, but this suffers from insufficient transmission of infrared detail information, resulting in an inability to effectively improve image contrast. For example... Figure 3A As shown, the residual structure contains two convolution operations. Its advantage is that it can transfer information through multiple stacked structures and prevent overfitting. However, low-quality infrared images of substation equipment, which are the subject of this invention, have low contrast and color saturation, making it difficult to fully extract such information through convolution operations. Therefore, this invention introduces a self-attention mechanism to provide an effective modeling method for capturing global information from infrared images, such as... Figure 3B As shown.

[0034] First, it maps the input X∈Rn×c to a query Q∈Rn×d, a key K∈Rn×d, and a value V∈Rn×d, where d is the dimension and Wq, Wk, and Wv are trainable corresponding weights. This process can be expressed by the formula:

[0035] Query Q represents the query vector, indicating the location information the model is currently focusing on. The introduction of the query vector Q allows for greater attention to detail in the infrared images of substation equipment, while the key K helps the model calculate its importance relative to the query vector. The correlation between query Q and the corresponding key K defines the attention matrix A. The output Z of the self-attention mechanism is the dot product of the value V and A. The self-attention mechanism is better at capturing the internal correlations of features and has a stronger ability to extract detail and texture features from the infrared images of equipment.

[0036] Furthermore, self-attention mechanisms require fewer parameters compared to traditional convolution operations. In this invention, the number of parameters for a single self-attention operation is 3. d c+d d, a convolution operation with a kernel of 3 and 3 parameters. 3 C_in C_out. Therefore, compared to residual structures, self-attention mechanisms can significantly reduce the number of model parameters.

[0037] In this embodiment, the efficient local attention module extracts long-range features in the horizontal and vertical directions of the infrared image through one-dimensional horizontal global pooling and one-dimensional vertical global pooling, respectively. Spatial attention weights are generated by one-dimensional convolution, group normalization and activation function, and then multiplied element-wise with the original feature map to complete the enhancement of detail features.

[0038] Because low-quality substation equipment infrared images contain rich texture and edge details, when using CycleGAN for image style conversion, the first step is to extract feature information from the infrared image through an encoding network. However, during downsampling, the receptive field of the convolution operation is too large, making it impossible to fully extract the texture and edge details of the substation equipment infrared image. Therefore, this invention introduces an efficient local attention mechanism into the encoding network, the principle of which is as follows: Figure 4 As shown, this module further optimizes the model's ability to focus on detailed features during image generation without significantly increasing the number of parameters or inference time, thus improving the accuracy of texture reconstruction and edge preservation.

[0039] like Figure 4 As shown, X Avg Pool and Y Avg Pool represent one-dimensional horizontal global pooling and one-dimensional vertical global pooling, respectively. Their mathematical expressions are shown in equations (2.3) and (2.4):

[0040] In the formula This represents the horizontal pooling result of channel c at position h. (h,i) represents the feature value in the c-th channel, at the h-th row and i-th column; This represents the vertical pooling result of channel c at position w. (j, w) represents the feature value in the e-th channel, at row j and column w; H and W represent the horizontal and vertical dimensions of the feature map, respectively. ELA performs average pooling on each channel in both spatial ranges to obtain the horizontal and vertical feature vectors of the substation equipment infrared image, capturing long-range dependencies. The pooled result is then fed into a one-dimensional convolution F and a group normalization Gn, and activated by the Sigmoid function σ, as shown in equations (2.5) and (2.6). The output of the ELA module can be represented as Y, as shown in equation (2.7).

[0041]

[0042] In the formula, and This represents the attention weights calculated in the horizontal and vertical directions; and This refers to one-dimensional convolution operations in the horizontal and vertical directions; Indicates the original feature map at location The feature values ​​are Y, which is obtained by element-wise multiplying the original feature map with attention in the horizontal and vertical directions.

[0043] To address the issue of missing content in generated images, this invention constructs a feature pyramid structure between the encoding and decoding networks, effectively guiding the alignment and fusion of shallow and deep features. This enables the generator to have stronger adaptability and generalization ability for complex scenes, improves the generator's ability to generate global information, and overcomes the problem of lost feature information of small targets in infrared images.

[0044] The original CycleGAN discriminator extracts model features through three convolution operations and judges the generated image based on the extraction results. Infrared images of substation equipment have low contrast, and the numerous devices make them difficult to distinguish. For example, current transformers, voltage transformers, and bushings have similar characteristics, which is not conducive to discriminator identification. Single-channel feature extraction networks cannot fully extract image feature information, leading to misjudgments. To improve the discriminator network's ability to distinguish generated equipment infrared images, this invention adds a new infrared image information extraction channel to the initial discriminator, constructing a dual-channel discriminator, such as... Figure 5 As shown, the new discriminator architecture enhances the network's ability to extract device texture features against complex backgrounds in infrared images, effectively distinguishing differences between devices with similar features, thereby improving the discriminator's performance.

[0045] The initial loss function of CycleGAN is divided into two parts. One part is the adversarial loss function. CycleGAN has two discriminators, so there are two adversarial loss functions, as shown in equations (2.8) and (2.9):

[0046] In the formula Represents the generator G and the discriminator The adversarial loss between them is represented by Ey~pdata(y)[log(DY(y)], which is the score of the discriminator on the real data y, and Ex~pdata(x)[log(1-DY(G(x))], which is the loss of the fake image generated by the generator G. Let F represent the adversarial loss between the generator F and the discriminator DX, where Ex~pdata(x)[log(DX(x)] represents the discriminator score on the real data x; and Ey~pdata(y)[log(1-DX(F(y))] is the loss of the fake images generated by the generator F. The goal is to make the images generated by the generator able to fool the discriminator. .

[0047] The other part is the cycle-consistent loss, as shown in equation (2.10):

[0048] In the formula, Ex~pdata(x)[||F(G(x))-x||1] represents that after generator G transforms x from domain X to domain Y, generator F transforms G(x) back to X, and the difference between the restored image and the original image x is calculated, so that F(G(x)) is as close to x as possible. The other term is similar to the first term, except that the operation is performed in the other direction.

[0049] The overall loss function is shown in equation (2.11):

[0050] To further improve the quality of the generated images and the model's convergence speed, this invention optimizes the loss function, specifically the adversarial loss function, as shown in equations (2.12) and (2.13). The logarithmic operation is replaced with a square operation, reducing computational complexity and improving the model's convergence speed. The optimized overall loss function is shown in equation (2.14).

[0051]

[0052] To verify the performance of the algorithm of this invention, the experiment was conducted in the PyTorch framework, and the software and hardware configuration parameters are shown in Table 2.1.

[0053] Table 2.1 Software and Hardware Parameter Configuration Table

[0054] To objectively evaluate the performance of the algorithm of this invention, image entropy, image color intensity, saturation, and feature similarity are selected as evaluation indicators.

[0055] (1) Image entropy is defined as the maximum amount of image information in an image. Generally, the larger the value, the richer the information contained in the image. It can effectively evaluate the content information of infrared images. Its formula is shown in (2.15).

[0056]

[0057] (2) Image colorfulness can measure the richness of image colors. The greater the colorfulness, the richer the image colors. It can effectively evaluate the richness of infrared colors. Its formula is shown in Equation (2.16).

[0058]

[0059] In the formula and The calculation method is shown in equations (2.17) and (2.18):

[0060] In the formula , , and These are the standard deviation and mean of rg and yb, respectively.

[0061] (3) Saturation represents the brightness of the colors of objects in a photograph. The higher the value in an image, the more vivid the colors it contains. Saturation is calculated by averaging the saturation of all intensities in the image. Saturation can be used to effectively evaluate the contrast information of an infrared image. Its calculation formula is shown in Equation (2.19).

[0062]

[0063] Where R(u,v), G(u,v) and B(u,v) represent the average saturation values ​​of the red, green and blue channels, respectively.

[0064] (4) Feature Similarity (FSIM)

[57] uses phase consistency (PC) as the main feature and image gradient magnitude as the secondary feature. The two play a complementary role in characterizing image quality. The larger the FSIM value, the higher the image quality. Its calculation formula is shown in Equation (2.20). Its calculation stage includes two stages. The first stage calculates the local similarity map; the second stage merges the similarity map into a single similarity score. Using FSIM as an evaluation index can accurately measure the global feature similarity of enhanced infrared images.

[0065]

[0066] In addition to the above evaluation indicators, this invention also introduces the number of model parameters as an evaluation indicator.

[0067] In order to objectively and realistically evaluate the performance of the algorithm of the present invention, the present invention selects some traditional image enhancement methods such as histogram equalization, Retinex algorithm and guided filtering, and also selects some latest deep learning image enhancement methods such as DCGAN

[58] and diffusion model as comparison objects. Table 2.2 shows the relevant evaluation index results of different algorithms. Since the image enhancement effect of Retinex algorithm and guided filtering algorithm is related to the specific set parameters, the two algorithms have poor enhancement effect when processing infrared images of substation equipment. As can be seen from Table 2.2, the algorithm of the present invention has obvious advantages compared with the mainstream image enhancement algorithms. It achieves the best results in image color, saturation and feature similarity, with an average improvement of 10.91%, 18.89% and 29.82%, respectively. At the same time, due to the introduction of a self-attention mechanism with fewer parameters and an efficient local attention mechanism, the number of model parameters of the algorithm of the present invention is reduced by 37.89% compared with the original algorithm. This shows that the algorithm of the present invention has obvious advantages in processing low-quality infrared images of substation equipment.

[0068] Table 2.2 Comparison of results from different algorithms

[0069] To verify the effectiveness of the improved algorithm in this invention, ablation experiments were conducted, and the results are shown in Table 2.3. Scheme 1, with the addition of self-attention, significantly reduces the number of model parameters, achieving lightweight design with only 4.32M parameters. However, the FSIM value is low, resulting in poor image enhancement. Scheme 2, based on Scheme 1, incorporates an ELA attention mechanism, improving the generator's ability to extract detailed image features. Results show that it effectively improves image quality. Scheme 3, based on Scheme 2, adds a feature pyramid structure, enhancing the generator network's feature fusion capability and overcoming the problem of missing information in the generated image. Based on Scheme 3, this invention constructs a dual-channel feature extraction network in the discriminator, improving the discriminator's feature comparison capability. The FSIM value reaches a maximum of 0.61, indicating that this invention's algorithm effectively improves image quality compared to the original algorithm.

[0070]

[0071] To better demonstrate the superiority of the algorithm of this invention, this section showcases the enhancement effects of various algorithms, as shown in the following figures. Figure 6As shown in the figure, the algorithm of this invention effectively enhances low-quality images even under complex substation backgrounds and with a wide variety of equipment, significantly improving image contrast and saturation. Traditional methods, such as the Retinex algorithm and histogram equalization algorithm, perform poorly on infrared image enhancement of low-quality substation equipment. While deep learning image enhancement methods can improve image quality, they suffer from the loss of detailed features, resulting in incomplete images or altered original features. For example, in images enhanced by the diffusion model, the equalizing ring of the surge arrester shows features of a current transformer. Images enhanced by DCGAN show partial feature loss and blurring. The original CycleGAN-enhanced image still has low contrast, saturation, and color. Therefore, the algorithm of this invention has significant advantages in processing infrared image enhancement tasks for low-quality equipment.

[0072] The purpose of enhancing infrared images is to meet the requirements of subsequent advanced machine vision tasks such as object recognition and image segmentation. This section uses the CenterNet object recognition algorithm to evaluate the image enhancement effect. Experimental results are shown below. Figure 7 As shown, unenhanced low-quality infrared images have poor recognition performance, resulting in missed detections and false detections. However, the infrared images of substation equipment enhanced by the algorithm of this invention show a significant improvement in recognition performance. For example, surge arresters in the unenhanced image have low contrast and are blurry, failing to be identified, while the target equipment in the enhanced image is accurately identified. This demonstrates that the algorithm of this invention can effectively improve image quality while simultaneously increasing the recognition accuracy of the equipment identification model.

[0073] In this embodiment, the improved structure of the equipment identification network of the CenterNet deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images includes: an adaptive spatial feature fusion module and a feature pyramid multi-scale fusion structure integrated at the back end of the backbone network; a global attention module embedded in the feature fusion network; lightweighting by replacing all standard convolutions in the network with depthwise separable convolutions; and replacing the original network loss function with a combination of distributed focal loss and smoothing loss.

[0074] In this embodiment, the global attention module is divided into a channel attention branch and a spatial attention branch. The channel attention branch uses three-dimensional feature permutation and multilayer perceptron to capture cross-channel dependencies, while the spatial attention branch enhances the spatial saliency of the device target in the image. The two branches jointly suppress invalid feature weights in complex backgrounds.

[0075] While the CenterNet algorithm can identify infrared images of substation equipment, the actual acquisition of infrared images often encounters problems such as complex backgrounds and the coexistence and occlusion of multiple devices. This leads to poor performance of the original CenterNet model in identifying multiple infrared targets, small targets, and occluded targets in complex backgrounds. Especially when these conditions occur simultaneously, the model's recognition accuracy is significantly affected. To address these issues, this invention improves the CenterNet algorithm, designing an improved CenterNet substation equipment identification method that enhances the model's ability to extract feature information from equipment in complex backgrounds. Its structure is as follows: Figure 8 As shown.

[0076] In order to improve the recognition accuracy of small and multiple targets in complex backgrounds in infrared images of substation equipment, this invention integrates Feature Pyramid Networks (FPN)

[61] and Adaptively Spatial Feature Fusion (ASFF)

[62] into CenterNet to construct an adaptive feature fusion network. The network adaptively fuses low-level features (such as 128×128×256 and 64×64×512) and high-level detail features (such as 32×32×1048) from the backbone network through the ASFF module. Then, the FPN structure is used to connect these feature information with the decoding network, thereby enhancing the reusability of feature maps and optimizing the multi-scale feature fusion capability, thereby improving the model's recognition performance of small and multiple targets in infrared images.

[0077] ASFF first processes the low-level feature map Flow, the mid-level feature map Fmid, and the high-level feature map Fhigh in the backbone network using 1×1 convolution operations. Next, it adjusts the size of the resulting feature maps through downsampling, upsampling, and upsampling operations to ensure they are the same size. These three adjusted feature maps are then stacked dimensionally and processed again by 1×1 convolution to obtain the fused feature map. Its basic principle can be represented by equation (3.1).

[0078] In ASFF, α, β, and γ are weight coefficients representing the parameters of a 1×1 convolutional kernel. After training with samples, these three kernels can learn the important relationships between feature maps of different sizes. The traditional CenterNet model, during feature extraction, only obtains a 16×16×2048 feature map after four downsampling passes, leading to significant loss of feature information for small targets. Therefore, this invention introduces an ASFF module based on the traditional FPN structure. By leveraging an adaptive feature fusion network, it can effectively fuse cross-scale semantic information, thereby improving the recognition accuracy of small and multiple targets. The FPN structure is shown below. Figure 9 .

[0079]

[0080] To address the issues of missed detections and false detections caused by factors such as the lack of obvious features of small targets and occluded targets in complex backgrounds and the wide variety of equipment in infrared images of substation equipment, this invention introduces a Global Attention Mechanism (GAM).

[63] This strengthens the effective feature weights during feature fusion and suppresses the weights of ineffective features in the background information. GAM, by calculating attention weights for each input element and considering the relationships between elements, can capture salient features in three dimensions, thus significantly improving the model's performance in recognizing small infrared targets. GAM consists of two parts: a channel attention mechanism and a spatial attention mechanism, and its working principle is as follows: Figure 10 As shown.

[0081] Traditional channel attention mechanisms typically employ Global Average Pooling (GAP) to calculate the weights of each channel. However, the input features obtained through GAP lack sufficient diversity, making it difficult to effectively capture the rich feature information in an image. The channel attention mechanism proposed in this invention preserves three-dimensional information through 3D permutation and utilizes two layers of Multilayer Perceptrons (MLPs) to enhance cross-dimensional channel-spatial dependencies, thereby better capturing the diversity of input features. Its calculation formula is shown in Equation (3.2). Since channel attention mechanisms may lead to the loss of image positional information, to address this issue, this invention further introduces a spatial attention mechanism, the calculation formula of which is shown in Equation (3.3).

[0082]

[0083] In the formula, M c and M s These represent channel attention mechanism and spatial attention mechanism, respectively.

[0084] To achieve high recognition accuracy while minimizing the impact of adding feature fusion modules and attention mechanisms on recognition speed, this invention introduces Depthwise Separable Convolution (DSC).

[64] Replacing ordinary convolutions in the original network reduces the number of model parameters and improves inference speed. Depthwise separable convolutional structures, such as... Figure 11 As shown.

[0085] Figure 11 In this context, depthwise separable convolution consists of two parts: depthwise convolution and pointwise convolution. The ratio of depthwise separable convolution to standard convolution is:

[0086] In the formula H , W Given the height and width of the input feature map, M The number of input feature channels, K The kernel size is the convolution kernel size. N The number of channels in the output feature map. As can be seen from equation (3.4), depthwise separable convolution can greatly reduce the computational cost and parameter count of the model.

[0087] The original loss function of CenterNet consists of three parts: heatmap loss. L k Target width and height prediction results regression loss L size Regression loss of target center point offset prediction results L off The overall loss function is shown in equation (3.5):

[0088] in, L k This is known as Focal Loss. L size and L off For L1 Loss, λ size and λ off For the weights, respectively represent L size and L off The magnitude of its contribution to the total loss. Focal Loss is shown in equation (3.6): For predicted values, For the true value,α and β Using the weights, this loss function effectively solves the problem of model non-convergence caused by an imbalance in the number of positive and negative samples.

[0089] Distribution Focal Loss (DFL) optimizes the determination of bounding box regression values ​​in Focal Loss by changing the single Dirac distribution to an arbitrary probability distribution within a certain range. This allows the network to quickly focus on values ​​near the label, thereby increasing the network's sensitivity to infrared boundary targets and maximizing the probability density at the label. Its formula is shown in equation (3.7):

[0090] In the formula y With the target center point, y i+1 and y i These are the two points closest to the center of the target. S i+1 and S i These are the probability distributions of the two points closest to the center of the target.

[0091] L1 Loss is highly sensitive to outliers when calculating the differences between bounding boxes and is easily affected by noise. Smooth L1 Loss effectively balances the sensitivity to outliers and improves the robustness of the model by introducing a smoothing function, as shown in equation (3.8).

[0092] This invention replaces Focal Loss with DFL and L1 Loss with Smooth L1 Loss to improve the sensitivity of the loss function to infrared occluded targets and enhance the model's convergence speed and recognition accuracy.

[0093] The infrared image dataset of this invention comes from the 2024 provincial inspection infrared thermography spectrum compilation of 110kV substations of a State Grid Corporation of a certain province. It includes 4125 images of seven types of substation equipment, such as surge arresters, circuit breakers, and disconnectors. LabelImg is used to annotate 744 surge arresters, 498 high-voltage bushings, 2472 insulators, 600 current transformers, 281 voltage transformers, 934 circuit breakers, and 723 disconnectors. To better verify the model's performance, the dataset was randomly divided into training, validation, and test sets in a 7:2:1 ratio.

[0094] The experiment used mAP0.5, precision (P), recall (R) and frames per second (FPS) as objective evaluation indicators, and the definitions of each indicator are shown in Equations (3.9) and (3.10).

[0095]

[0096] in, T P To predict the correct number of positive samples, F P To predict the number of positive samples that are incorrect, F N This represents the number of negative samples with prediction errors. mAP0.5 is a commonly used performance evaluation metric, which means the average AP for each class when IOU is 0.5.

[0097] In order to determine the equation (3.5) λ size and λ off The present invention has made different optimal values. λ size and λ off Comparison experiment of values. Through transfer learning

[65] The method uses the weights trained on the VOC dataset as the initial weights of the model to accelerate model convergence. Experimental parameters were set as follows: input image size 512×512×3, 300 training epochs (first 50 epochs frozen training, last 250 epochs unfrozen training). An adaptive moment estimation optimizer was used for training with an initial learning rate of 1×10⁻⁶. -4 The experimental results are as follows: Figure 12 As shown.

[0098] Figure 12 In the diagram, the two values ​​on the horizontal axis are respectively λ size and λ off Experimental results show that the model has the highest recognition accuracy when both are set to 0.1. Therefore, in the algorithm of this invention, both are set to 0.1.

[0099] The training iteration process of the algorithm of this invention is as follows: Figure 13 As shown. By Figure 13As can be seen, compared with the original CenterNet algorithm, the convergence speed of this algorithm is significantly faster, and the loss value is smaller and the robustness is stronger in both the freezing and thawing phases. At the same time, this algorithm has higher recognition accuracy, reaching over 90% accuracy by the 90th round. This demonstrates that the algorithm of this invention has a strong ability to recognize multiple targets, small targets, and occluded targets in complex backgrounds of infrared images, showcasing the superiority of this algorithm.

[0100] Table 3.1 shows the recognition results of the model on infrared images of different types of substation equipment. The results show that the algorithm of the present invention has accurate recognition effect on substation equipment under various complex backgrounds, with an average recognition accuracy of 95.19%.

[0101] Table 3.1 Recognition results of different devices (%)

[0102] To verify the effectiveness of the model, this section compares the effects of different improvement methods through ablation experiments. The experimental results are shown in Table 3.2. Scheme 1 adds an ASFF+FPN structure to the original CenterNet model, improving the model's ability to extract target features and perform multi-scale fusion in complex backgrounds, resulting in a 0.79% improvement in recognition accuracy compared to the original algorithm. Scheme 2 introduces a GAM attention mechanism based on Scheme 1, enabling the network to adaptively focus on small and occluded targets in complex backgrounds, improving recognition accuracy by 0.67% compared to Scheme 1. Scheme 3 adds depthwise separable convolution to Scheme 2, but has little impact on improving model accuracy. Furthermore, by improving the loss function based on Scheme 3, the model accuracy is improved by 1.87%. These results indicate that improving the loss function can effectively enhance the model's ability to recognize occluded and small targets, thereby improving recognition accuracy.

[0103]

[0104] To further verify the superiority of the model, this invention introduces the number of model parameters to evaluate the model's space complexity and floating-point operations (FLOPs) to measure the model's time complexity. Simultaneously, comparative experiments were conducted with the algorithm of this invention and selected mainstream target recognition algorithms. The experimental results are shown in Table 3.3.

[0105] As shown in Table 3.3, the algorithm of this invention outperforms the comparative algorithms in terms of recognition accuracy, number of model parameters, and number of floating-point operations. This indicates that the algorithm of this invention has significant advantages in both recognition accuracy and algorithm complexity, and also shows a strong improvement in inference speed. A comparison with the original CenterNet shows that, after incorporating various improvements, the algorithm of this invention not only improves recognition accuracy but also further reduces the number of model parameters, achieving model lightweighting. This has significant practical implications and application value for subsequent model deployment.

[0106]

[0107] To verify the generalization ability of the model in different scenarios, aerial images of faulty and normal insulators under complex backgrounds in visible light scenes were selected as the recognition objects. A dataset of 1800 aerial images of insulators was used, with a ratio of normal insulator strings to faulty insulator strings of 3:2. The specific training process was set up according to the relevant description in Section 4.3. The experimental results are shown in Table 3.4.

[0108] As shown in Table 3.4, in the generalization experiment, the algorithm of this invention has a higher recognition rate than the original CenterNet algorithm in identifying both normal and defective insulators, with a recognition rate of 3.67% higher across all categories. This indicates that in visible light scenarios, the algorithm of this invention has good recognition performance for multiple insulator targets in complex backgrounds and smaller faulty insulators, and also demonstrates that the model has strong cross-scale feature extraction and adaptive feature fusion capabilities.

[0109]

[0110] The recognition performance of different algorithms in different scenarios is as follows Figure 14 As shown: Scenario 1 presents a multi-target recognition scenario against a complex substation background. Experimental results show that only the algorithm proposed in this invention can accurately identify all infrared targets, proving that the ASFF+FPN structure can effectively extract features of different substation equipment in complex backgrounds. Other algorithms suffer from missed detections; for example, while Faster R-CNN identified most of the substation equipment, it failed to identify the insulator in the upper left corner, and other algorithms even missed several targets.

[0111] Scenario 2 presents a complex substation environment with multiple and small targets coexisting. Experimental results show that the algorithm of this invention can accurately identify various substation equipment in complex environments and successfully detect small, distant transformers, indicating that by introducing the GAM attention mechanism, the model can adaptively extract effective features of the equipment. Other algorithms perform poorly in identifying small, distant targets, exhibiting instances of missed detections or failures to identify them.

[0112] Scenario 3 presents a complex substation environment with multiple targets and obstructed targets coexisting. In this scenario, surge arresters obstruct most bushings, and other algorithms besides the one described in this invention failed to identify the obstructed bushings. Experimental results show that the algorithm described in this invention significantly improves the model's sensitivity to obstructed targets by optimizing the loss function and the probability distribution of bounding box regression, thus enabling it to identify obstructed equipment more effectively.

[0113] In this embodiment, the improved structure of the segmentation network of the lightweight, improved deep learning-based intelligent diagnosis method for infrared thermal faults in substation equipment (YOLOv8) includes: replacing the backbone network of the original deep learning-based intelligent diagnosis method for infrared thermal faults in substation equipment (YOLOv8) with a lightweight inverse residual network from the deep learning-based intelligent diagnosis method for infrared thermal faults in substation equipment (MobileNetV3); and introducing a network neck. The semantic detail injection module replaces the original feature fusion structure of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (C2f) with the upsampling inverse bottleneck fusion structure of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (C2f_UIB); and replaces the original mask loss and classification loss with a combination of focus loss and similarity coefficient loss of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (Dice).

[0114] In this embodiment, the semantic detail injection module performs downsampling, identity mapping, and upsampling multi-scale interactive operations on features at different levels, and embeds channel and spatial dual attention at the front end of each feature level to achieve complete cross-scale feature extraction of irregular small fault regions.

[0115] Suspected fault regions in infrared images of substation equipment are characterized by irregular shapes, small size, and indistinct details. These factors make it difficult for deep learning models to accurately capture the edge shapes of these regions. Feature extraction from smaller regions requires higher computational complexity, but to maintain the algorithm's practicality in engineering, lightweight deep learning models must also be considered. Therefore, this invention proposes a lightweight YOLOv8-based method for segmenting suspected fault regions in infrared images of substation equipment. This method aims to ensure accurate extraction of suspected fault regions from substation equipment infrared images while minimizing the number of model parameters and improving extraction efficiency. Its network structure is as follows: Figure 15 As shown.

[0116] To address the model redundancy issue arising from the extraction of detailed features of suspected fault areas in infrared images of substation equipment, this invention introduces a lightweight MobileNetV3 network as the backbone. This significantly reduces the number of model parameters and complexity while enhancing the network's ability to extract features from suspected fault areas in infrared images. MobileNetV3 employs the h-swish activation function, approximating the sigmoid component of the swish activation function to improve computational efficiency, which in turn significantly enhances model performance. MobileNetV3 can optionally incorporate a squeeze-and-excite (SE) attention mechanism, adaptively adjusting the weights of channel features to allow the network to focus on diagnostically significant thermal anomaly features.

[0117] To address the issue of inconsistent sizes of suspected fault areas in infrared images of substation equipment and the problem of cross-scale features, the YOLOv8 model uses simple connection operations in the neck region to achieve feature interaction between different network levels. While this method is simple and fast, it suffers from insufficient feature interaction. Therefore, this invention introduces a Semantics and Detail Infusion (SDI) module into the neck network to achieve spatial interaction of feature information across different levels.

[0118] To address the issue of indistinct edge information in suspected fault areas of substation equipment infrared images, which can easily lead to oversegmentation, the YOLOv8 model adds several compact 2-level feature integration (C2f) structures to the neck region. The core idea of ​​the C2f structure is to fuse features through multiple branches, aiming to enhance feature information at different scales in the image. However, this structure also has certain limitations. Because the C2f structure uses branch merging for feature fusion, when processing subtle edge information in suspected fault area images, the branch fusion process may lose some local details, especially for some complex-shaped suspected fault areas, potentially leading to oversegmentation.

[0119] To improve the model's accuracy and robustness in extracting suspected fault areas in substation equipment, this invention introduces a compact 2-level feature integration with upsample-inverted bottleneck (C2f_UIB) structure. C2f_UIB integrates an efficient multi-branch fusion structure while introducing multiple compact inverted bottleneck (CIB) modules. The CIB module is an improved bottleneck structure designed to improve computational efficiency and performance, especially when applied in deep neural networks, effectively reducing the number of model parameters and computational load.

[0120] Extracting suspected faults from infrared images of substation equipment presents unique challenges, including small target size in fault areas, class imbalance, and potential overlap of suspected fault areas. The original YOLOv8 loss function consists of bounding box regression loss. Classification loss Confidence loss and mask loss composition.

[0121]

[0122] In the formula and The weights for confidence and masking losses are denoted as . Both classification and masking losses employ Binary CrossEntropy Loss (BCE Loss). These designs perform well in general segmentation tasks, but they may be insufficient for the complexity of infrared image suspected fault segmentation tasks. This is because suspected fault regions in infrared images typically have irregular areas and unclear target boundaries. These characteristics may cause the bounding box-based and classification-based loss functions to fail to fully capture the detailed features of the infrared fault region, thus affecting the model's segmentation performance. To address these issues, this invention respectively and Replacing BCE Loss with FocalLoss and Dice Loss significantly improves segmentation performance.

[0123] The improved loss function introduces Dice Loss into the mask loss. Dice Loss directly optimizes the overlap between the predicted and ground truth masks, measuring the target matching degree in the form of Intersection over Union (IoU). Its formula is as follows:

[0124] inP To predict the mask, G For the real mask, To prevent constants with zero denominators. Traditional loss functions (such as cross-entropy) typically focus on the correct classification of each pixel, while Dice Loss places more emphasis on pixel-level overlap, making it particularly suitable for scenarios with small fault regions and unevenly distributed targets. Its emphasis on pixel overlap significantly improves the segmentation performance of small target regions.

[0125] In this embodiment, in step S5 of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment using infrared images, the infrared image grayscale and temperature mapping standard adopts the GB / T 44006-2024 method for intelligent diagnosis of thermal faults in substation equipment using infrared images, which is based on the temperature representation rules of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment using infrared images. The actual temperature value corresponding to each pixel is calculated based on the image grayscale level, the highest temperature in the image temperature measurement range, and the lowest temperature.

[0126] In this embodiment, the hierarchical judgment logic of the relative temperature difference method in step S6 of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images is as follows: if the relative temperature difference value is less than 35%, the equipment is determined to be fault-free; if the relative temperature difference value is greater than or equal to 35% and less than the value of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images, the equipment is determined to be fault-free. The method for intelligent diagnosis of thermal faults in substation equipment based on deep learning infrared images determines the following faults: 80% of the faults are classified as general faults; relative temperature differences greater than or equal to 80% and less than 95% are classified as severe faults; and relative temperature differences greater than or equal to 95% are classified as emergency faults.

[0127] In this embodiment, the method is deployed on a substation inspection terminal, an airborne edge computing module of a drone, or a back-end server. After batch input of infrared images, it automatically executes the entire process of image enhancement, equipment identification, fault segmentation, temperature conversion, and fault classification in sequence. It does not require manual annotation or manual interpretation and directly outputs a structured diagnostic report containing equipment type, fault area location, relative temperature difference value, fault level, and maintenance suggestions.

[0128] The substation equipment fault diagnosis process proposed in this invention is as follows: Figure 16 As shown.

[0129] The specific diagnostic steps are as follows: (1) Infrared image enhancement of substation equipment. Low-quality infrared images are input into the improved CycleGAN model, and the image quality is significantly enhanced through style transfer.

[0130] (2) Equipment type identification. After image enhancement processing, the infrared image is input into the CenterNet classification model for equipment type identification. Through the model's output, we can accurately identify the equipment type and locate the target equipment, providing a basis for subsequent fault diagnosis.

[0131] (3) Extraction of suspected fault areas. By using the improved YOLOv8 region segmentation method, the temperature abnormality areas of the equipment, i.e., the suspected fault parts, are extracted.

[0132] (4) Obtain relevant temperature information of the equipment. Through the above steps, the normal and abnormal temperature areas of the target equipment can be obtained. In order to obtain the specific temperature information of the corresponding area, the image of the suspected fault area needs to be converted into a grayscale image, and the latest image grayscale value and temperature value conversion standard should be referred to—"Infrared Image Temperature Representation Rules RGB Method / GB T44006-2024".

[73] This allows for the acquisition of temperature values ​​in suspected fault areas. The standard specifies the mapping relationship between temperature and grayscale values ​​in infrared images as follows:

[0133] In the formula S The grayscale value of a pixel in a grayscale image; T The temperature value of the pixel to be determined is in degrees Celsius. This is the highest temperature value within the temperature range, corresponding to the highest temperature value on the infrared image colorimetric card, in degrees Celsius. T min This is the lowest temperature value within the temperature range, corresponding to the lowest temperature value on the infrared image colorimetric card, in degrees Celsius. N is the gray level of the grayscale image; in this study, its value is 256.

[0134] (5) Calculate the relative temperature difference. Before performing thermal fault diagnosis, the relative temperature difference method needs to be used to calculate the relative temperature difference value. In order to more accurately reflect the fault status of the equipment, this project selects the highest temperature value of the suspected fault area as the temperature of the abnormal condition area; in order to most accurately reflect the temperature of the normal operating state of the equipment, this project selects the temperature corresponding to the gray value that appears most frequently in the normal area of ​​the equipment; and the actual ambient temperature of the equipment can be used as the standard by the ambient temperature measured by the thermometer during the inspection. Finally, the relative temperature difference is calculated using formula (5.2).

[0135] (6) Equipment thermal fault diagnosis. Based on the data calculated using the relative temperature difference method, and in conjunction with the fault criteria in Table 5.1, the equipment fault status is determined. If the equipment is faulty, the severity of the fault is further determined. Based on the equipment diagnosis results, corresponding measures are taken to handle the situation.

[0136] If the calculated relative temperature difference is less than 35%, it means that the target equipment is not faulty and is operating normally; if it is greater than or equal to 35% but less than 80%, the equipment is operating abnormally and is considered a general fault; if the relative temperature difference is greater than or equal to 80% but less than 95%, the equipment is in a serious fault state; if the value is greater than or equal to 95%, the equipment has a critical fault and needs to be shut down immediately for repair.

[0137] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for intelligent diagnosis of thermal faults in substation equipment based on infrared images using deep learning, characterized in that, The method includes the following steps: S1. Dataset Construction and Multi-Network Transfer Learning Training: Collect infrared images of various types of equipment in the substation, divide them into training set, validation set, and test set, and train the image enhancement network, equipment recognition network, and fault area segmentation network respectively. S2. Adaptive Infrared Image Enhancement: The low-quality original infrared images of the substation collected on-site are input into the trained improved self-attention deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images, CycleGAN, and the image enhancement network outputs enhanced infrared images with high contrast and clear details. S3. Precise multi-target identification of substation equipment: The improved deep learning-based intelligent diagnosis method for thermal faults in substation equipment based on infrared images is trained by CenterNet, an equipment identification network based on deep learning for infrared images of substation equipment. It identifies all substation equipment categories in the image and outputs independently cropped images of each equipment. S4. Lightweight pixel segmentation of suspected thermal fault areas: Input the cropped image of a single device into the trained lightweight improved deep learning-based intelligent diagnosis method for thermal faults in infrared images of substation equipment YOLOv8, and extract all suspected thermal fault areas on the surface of the equipment at the pixel level. S5. Infrared image temperature value extraction: Based on the infrared image grayscale and temperature mapping standard, extract the highest temperature of the suspected fault area, the reference temperature of the normal equipment area, and the actual measured temperature of the on-site environment. S6. Automatic thermal fault classification and diagnosis: The relative temperature difference method is used to calculate the abnormal temperature rise difference of the equipment, and the fault level of the equipment is automatically determined by combining the infrared diagnostic standards for live equipment in the power industry, and a standardized operation and maintenance diagnosis report is generated.

2. The intelligent diagnostic method for thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: The improved self-attention-based deep learning-based intelligent diagnosis method for thermal faults in substation equipment using infrared images (CycleGAN) features an improved image enhancement network structure including: an efficient local attention module embedded within the generator; a fusion structure of the self-attention module and the feature pyramid; a dual-channel parallel feature extraction network for the discriminator; and optimization of the overall network loss function using the square term, eliminating logarithmic operations to accelerate model convergence.

3. The intelligent diagnosis method for infrared image thermal faults of substation equipment based on deep learning according to claim 2, characterized in that: The efficient local attention module extracts long-range features in the horizontal and vertical directions of the infrared image through one-dimensional horizontal global pooling and one-dimensional vertical global pooling, respectively. It generates spatial attention weights through one-dimensional convolution, group normalization and activation function, and then multiplies them element-wise with the original feature map to complete the enhancement of detailed features.

4. The intelligent diagnostic method for thermal faults in substation equipment based on infrared images according to claim 1, characterized in that: The improved structure of the equipment identification network in the CenterNet deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images includes: an adaptive spatial feature fusion module and a feature pyramid multi-scale fusion structure integrated in the back end of the backbone network; a global attention module embedded in the feature fusion network; lightweighting by replacing all standard convolutions in the network with depthwise separable convolutions; and replacing the original network loss function with a combination of distributed focal loss and smoothing loss.

5. The intelligent diagnostic method for infrared image thermal faults in substation equipment based on deep learning according to claim 4, characterized in that: The global attention module is divided into a channel attention branch and a spatial attention branch. The channel attention branch uses three-dimensional feature permutation and multilayer perceptron to capture cross-channel dependencies, while the spatial attention branch enhances the spatial saliency of the device target in the image. The two branches work together to suppress invalid feature weights in complex backgrounds.

6. The intelligent diagnostic method for infrared image thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: The improved structure of the segmentation network in the lightweight, improved deep learning-based intelligent diagnosis method for infrared thermal faults in substation equipment (YOLOv8) includes: replacing the original backbone network with a lightweight inverse residual network from the MobileNetV3 deep learning-based intelligent diagnosis method for infrared thermal faults in substation equipment; and introducing semantic details into the network neck. The injection module replaces the original feature fusion structure of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (C2f) with the upsampling inverse bottleneck fusion structure of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (C2f_UIB); and replaces the original mask loss and classification loss with a combination of focus loss and similarity coefficient loss of the deep learning-based intelligent diagnosis method for thermal faults in substation equipment infrared images (Dice).

7. The intelligent diagnostic method for infrared image thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: The semantic detail injection module performs multi-scale interactive operations such as downsampling, identity mapping, and upsampling on features at different levels, and embeds dual channel and spatial attention at the front end of each feature level to achieve complete cross-scale feature extraction of irregular small fault regions.

8. The intelligent diagnostic method for infrared image thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: Step S5: In the deep learning-based intelligent diagnosis method for thermal faults in substation equipment using infrared images, the infrared image grayscale and temperature mapping standard adopts the GB / T 44006-2024 method for intelligent diagnosis of thermal faults in substation equipment using infrared images. The method uses RGB temperature representation rules to calculate the actual temperature value corresponding to each pixel based on the image grayscale level, the highest temperature in the image temperature measurement range, and the lowest temperature.

9. The intelligent diagnostic method for infrared image thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: Step S6: The relative temperature difference method in the intelligent diagnosis method for thermal faults of substation equipment based on deep learning is as follows: if the relative temperature difference value is less than 35%, the equipment is determined to be fault-free. If the relative temperature difference is greater than or equal to 35% and less than 80%, it is considered a general fault. A relative temperature difference greater than or equal to 80% and less than 95% of the value of the infrared image thermal fault diagnosis method for substation equipment based on deep learning is considered a serious fault. A relative temperature difference greater than or equal to 95% of the values ​​determined by the deep learning-based intelligent diagnosis method for thermal faults in substation equipment using infrared images are considered an emergency fault.

10. The intelligent diagnostic method for thermal faults in substation equipment based on deep learning according to claim 1, characterized in that: The method is deployed on a substation inspection terminal, an airborne edge computing module of a drone, or a back-end server. After batch input of infrared images, it automatically executes the entire process of image enhancement, equipment identification, fault segmentation, temperature conversion, and fault classification in sequence. It does not require manual annotation or interpretation and directly outputs a structured diagnostic report containing equipment type, fault area location, relative temperature difference value, fault level, and maintenance suggestions.

Images