Real-time identification and early warning system for substation equipment defects based on image fusion technology

Abstract

This invention discloses a real-time substation equipment defect identification and early warning system based on image fusion technology. The system comprises a data acquisition module that deploys mobile data acquisition equipment, infrared sensors, and visible light sensors to capture infrared and visible light images of substation equipment; a data fusion module that utilizes a scene-device decoupling fusion algorithm to obtain a fused image of the substation equipment's infrared and visible light images; a defect identification module that uses a target detection model to identify equipment defects based on the fused image and output the results; and an early warning module that, if the output result exceeds a risk threshold, initiates an audible and visual early warning based on the equipment defect type and simultaneously sends an early warning signal to the operation and maintenance terminal and platform. This invention uses multimodal images to improve the accuracy of identifying substation equipment defects, thereby ensuring safe production in substations.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment failure early warning, specifically to power equipment safety early warning, and more particularly to a real-time identification and early warning system for substation equipment defects based on image fusion technology. Background Art

[0002] Substations are a vital component of the power system. The rapid advancement of urbanization and industrialization has significantly increased the scale and complexity of substation equipment. Accurately identifying substation equipment defects is fundamental to inspection and fault diagnosis, and plays a vital role in ensuring the safety and stability of the power system. Traditional methods for identifying substation equipment defects rely primarily on single-modal images, which often fail to fully reflect the equipment's condition. Visible light images can provide rich texture information by capturing structural features, but their quality is highly susceptible to environmental factors, degrading in low light conditions, fog, or when the equipment is partially obscured. Infrared images, on the other hand, provide valuable temperature data, enable all-weather inspections, and are highly resistant to environmental interference. However, infrared images are also limited by factors such as thermal equilibrium, wavelength limitations, and transmission distance, which can result in low contrast and blurred visual quality.

[0003] Therefore, the substation equipment defect warning method based on single-modal images is limited by the inherent perception limitations of single-modal images, and has problems with defect recognition failure and insufficient accuracy. Specifically, it is difficult to effectively warn of local overheating, mechanical damage and leakage defects in key equipment such as voltage transformers, current transformers, insulators, and bushings, and is prone to false alarms, which leads to serious waste of operation and maintenance resources, abnormal substation operation status or even system paralysis, and derives fire hazards and personal safety risks, ultimately causing unpredictable major economic losses and safety accidents. Summary of the Invention

[0004] In order to improve the accuracy of substation equipment defect identification, this application integrates the shortcomings of existing single-modal images in substation equipment defect warning, designs a substation equipment defect real-time identification and warning system based on multimodal image fusion technology, improves the accuracy of substation equipment defect identification, and issues warnings based on high-accuracy defect identification to ensure the safe operation of the substation.

[0005] The present invention provides a real-time identification and early warning system for substation equipment defects based on image fusion technology, which is used to solve the problem of poor early warning and monitoring effect of substation equipment defects in the technology, including:

[0006] The data acquisition module uses deployed mobile acquisition devices, infrared sensors, and visible light sensors to collect infrared and visible light images of substation equipment, including current transformers, voltage transformers, insulators, and bushings.

[0007] A data fusion module is used to obtain a fused image of the infrared image and the visible light image of the substation equipment using a scene-device decoupling fusion algorithm based on the infrared image and the visible light image;

[0008] The defect recognition module is used to identify equipment defects based on the fused image and use the target detection model to output the results;

[0009] The early warning module is used to judge the output result. If the output result is greater than the risk threshold, an audible and visual alarm is activated according to the defect type of the equipment, and an early warning signal is sent to the operation and maintenance terminal and operation and maintenance platform.

[0010] The data fusion module specifically includes:

[0011] The source images to be fused are acquired using a mobile acquisition device, including infrared images of substation equipment acquired by an infrared image sensor and visible light images of substation equipment acquired by a visible light sensor. All images are paired;

[0012] Based on the paired infrared image and visible light image pairs, the scene-device decoupling fusion algorithm is used to obtain the fused image of the substation equipment infrared image and the visible light image;

[0013] The scene-device decoupling fusion algorithm is used to obtain a fusion image of the infrared image and visible light image of the substation equipment, including:

[0014] The pre-designed scene-device feature decoupling network SEFDNet is used to extract features from the fused source image to obtain infrared scene features. Features with infrared devices , and visible light scene characteristics Visible light device characteristics , in order to realize the decoupling of the source image scene features and the device features, and use the decoder to reconstruct the source image to obtain the reconstructed infrared image and reconstructing visible light images ;

[0015] Using the pre-designed visible light scene enhancement fusion network VSEFNet, the visible light scene features are obtained through the scene-device feature decoupling network SEFDNet. and infrared scene characteristics and visible light device characteristics Features with infrared devices and use the decoder to generate a fused image .

[0016] The beneficial effects of the present invention are:

[0017] By applying the above scheme, the present invention comprehensively considers the shortcomings of existing single-modal images in substation equipment defect warning, designs a substation equipment defect warning system based on image fusion technology of infrared images and visible light images, and improves the accuracy of identifying substation equipment defects based on multi-modal images, thereby ensuring safe production of substations.

[0018] Specifically, by accurately identifying and rapidly warning of high-temperature defects in voltage transformers, current transformers, insulators, and bushings, the system effectively avoids equipment failures caused by overheating, thereby eliminating the resulting abnormal substation operation, fire hazards, and personal safety risks. By accurately identifying and rapidly warning of mechanical damage defects in voltage transformers, current transformers, insulators, and bushings, the system effectively avoids equipment downtime and chain reactions due to structural failure. By accurately identifying and rapidly warning of leakage defects in voltage transformers, current transformers, insulators, and bushings, the system effectively prevents fires and environmental pollution caused by oil leaks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is an overall structural diagram of a real-time identification and early warning system for substation equipment defects based on image fusion technology in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0022] The inventors have discovered that multimodal fusion methods have attracted much attention in recent years due to their ability to integrate information from multiple modalities. These methods are able to learn more comprehensive and rich representations, significantly improving the information processing and comprehension capabilities of deep learning models. As a widely studied method in multimodal fusion, the infrared and visible light image fusion method (IVIF) retains detailed scene information and highlights key object features, providing more information-rich visual representations for downstream tasks in fields such as healthcare, remote sensing, and power equipment inspection. In particular, in substation equipment inspections, the IVIF algorithm seamlessly combines the fine texture and structural details captured by visible light images with the temperature information provided by infrared images. This fusion enables all-weather equipment recognition, enhances defect detection and fault diagnosis, and significantly improves the accuracy and reliability of intelligent inspection systems.

[0023] However, the existing IVIF algorithm still faces the following challenges in the task of fusing infrared and visible light images of substation equipment:

[0024] 1) Weak semantic discrimination of extracted features: Existing fusion methods can be divided into two categories: single-branch feature extraction and fusion methods, in which the encoder extracts features that are mostly global image features; and dual-branch feature decoupling and fusion methods. These algorithms generally use a dual-branch encoder to extract features from different domains in the source image for subsequent fusion of features across these domains. However, the features extracted by these algorithms are not suitable for substation equipment image fusion tasks, with weak semantic discrimination and inappropriate application scenarios. This can easily lead to feature loss or redundancy during the fusion process, thus compromising image fusion quality.

[0025] 2) Visible light scene information is masked: Visible light scene information plays a key role in substation image fusion and equipment detection tasks. However, the visible light scene information in the image obtained by the existing fusion method is masked by excessive infrared scene information, resulting in reduced fusion image quality. At the same time, it affects the downstream task detection model's ability to capture the relationship between equipment and scene, reducing the detection model's sensitivity to the infrared characteristics of the equipment, thereby affecting the detection results.

[0026] It can be said that existing image fusion methods have limitations such as weak semantic differentiation of features and the masking of valid information in the source image, such as visible light scene information. For example, the features extracted by the general fusion method are not suitable for the substation equipment image fusion task. The feature semantic differentiation is weak and does not conform to the substation application scenario. This can easily lead to feature loss or redundancy during the fusion process, thus affecting the image fusion quality. At the same time, the visible light scene information in the fused image obtained by the general algorithm is masked by excessive infrared scene information, resulting in reduced fused image quality. This also affects the downstream task detection model's ability to capture the relationship between equipment and scene, reducing the model's sensitivity to infrared features in the equipment, thereby affecting the accuracy of defect identification results.

[0027] To address the aforementioned issues, this application provides a real-time substation equipment defect identification and early warning system based on image fusion technology. This system addresses the inadequacy of common fusion methods for substation equipment image fusion tasks. By designing a scene-device feature decoupling network (SEFDNet), the system decouples source image features into scene features and device features. This addresses the weak semantic differentiation of features in common algorithms, making the semantic information more relevant to substation applications. An equipment temperature-texture constrained loss function (ETTCLoss) is proposed to enhance the model's feature decoupling capabilities. By designing a visible light scene enhancement fusion network (VSEFNet) to fuse image features from different modalities and generate a fused image, this system addresses the issue of visible light scene information being masked in fused images from common algorithms. An equipment thermal attention mechanism (ETBAM) is proposed to suppress infrared features within the scene region to enhance the representation of visible light information within the scene. This application effectively improves fusion quality and generates fused images that are more suitable for substation applications. Through application in downstream inspection tasks, the generated fused images effectively enhance the accuracy of substation equipment defect identification, enabling alarms based on highly accurate defect identification, thereby providing strong support for the safe and stable operation of power systems.

[0028] The real-time identification and early warning system for substation equipment defects based on image fusion technology described in this application. Figure 1 As shown, including:

[0029] The data acquisition module uses deployed mobile data acquisition devices, infrared sensors, and visible light sensors to collect infrared and visible light images of substation equipment, including current transformers, voltage transformers, insulators, and bushings. The data fusion module uses the scene-device decoupling fusion algorithm (DSEFusion) to generate fused infrared and visible light images of substation equipment based on the infrared and visible light images captured by the sensors. The defect identification module uses the target detection model to identify equipment defects based on the fused infrared and visible light images obtained by the data fusion module and output the results. The early warning module evaluates the output of the defect identification module. If the output exceeds the risk threshold, it triggers an audible and visual alarm based on the equipment defect type and simultaneously sends a warning signal to the operation and maintenance terminal and platform.

[0030] In this embodiment, by configuring a mobile data acquisition device in the substation and carrying an infrared sensor and a visible light sensor, infrared and visible light images of the substation equipment are obtained. Through the data fusion module, the scene-device decoupling fusion algorithm (DSEFusion) is used to obtain the infrared and visible light fused images of the substation equipment. The defect recognition module monitors the infrared and visible light fused images of the substation equipment in real time, thereby obtaining the equipment defect type and issuing equipment defect warnings.

[0031] In some embodiments of the present application, the data acquisition module is used to deploy mobile acquisition devices, infrared sensors, and visible light sensors to acquire infrared images and visible light images of substation equipment, including:

[0032] The mobile collection device includes an aircraft or a robot vehicle equipped with a camera; the infrared sensor and the visible light sensor are configured on the mobile collection device, and follow the mobile collection device to collect infrared images and visible light images of the substation equipment.

[0033] In some embodiments of the present application, the data fusion module uses a scene-device decoupling fusion algorithm (DSEFusion) to obtain infrared and visible light fused images of substation equipment based on the infrared images and visible light images collected by the sensor, including:

[0034] The source images to be fused are obtained using a mobile acquisition device, including infrared images of substation equipment collected by an infrared image sensor and visible light images of the substation collected by a visible light sensor. All images are paired;

[0035] Based on the paired infrared and visible light image pairs, the scene-device decoupling fusion algorithm (DSEFusion) is used to obtain infrared and visible light fused images of substation equipment.

[0036] In some embodiments of the present application, the scene-device decoupling fusion algorithm (DSEFusion) acquires infrared and visible light fusion images of substation equipment, including:

[0037] The pre-designed scene-device feature decoupling network SEFDNet is used to extract features from the fused source image to obtain infrared scene features. Features with infrared devices , and visible light scene characteristics Visible light device characteristics , in order to realize the decoupling of the source image scene features and the device features, and use the decoder to reconstruct the source image to obtain the reconstructed infrared image and reconstructing visible light images .

[0038] The pre-designed scene-device feature decoupling network SEFDNet can solve the problem of weak semantic differentiation of features in general algorithms. The device features of infrared and visible light images are expressed as the key information of the device in infrared and visible light images, namely, the temperature information of the device in infrared images and the texture information of the device in visible light images. The scene features of infrared and visible light images are expressed as the scene-related information in infrared and visible light images. At the same time, the device is a part of the scene, and the relationship between the device and the scene, such as the positional relationship, is also included in the scene feature expression information. As supplementary information to the device in the scene features, for scene features, since the source images are taken in the same scene, the scene features of infrared images and visible light images should be similar to each other. For device features, since the device features are expressed as the key information of the device in infrared and visible light images, namely, the temperature information of the device in infrared images and the texture information of the device in visible light images, the device temperature information is extracted from the infrared image. , extracting device texture information from visible light images , taking the above two types of information as labels, the model is trained using the pre-designed device temperature-texture constraint loss function ETTCLoss to optimize the model's ability to extract device features.

[0039] Using the pre-designed visible light scene enhancement fusion network VSEFNet, the visible light scene features are obtained through the scene-device feature decoupling network SEFDNet. and infrared scene characteristics and visible light device characteristics Features with infrared devices and use the decoder to generate a fused image .

[0040] Among them, considering that the fused image should show more scene information from the visible light image in the scene area, it is necessary to suppress the expression of infrared scene features in the scene area. At the same time, the infrared scene features There is additional information about the device, which needs to be effectively preserved due to the characteristics of the infrared image device. The device area that can reflect the infrared image is designed based on the spatial attention mechanism to design the device thermal attention mechanism ETBAM for infrared scene features. Processing is performed to promote the characteristics of the processed infrared scene It focuses on the complementary information of the device and suppresses the scene-related information in the infrared scene features, thereby enhancing the expression of the visible light image information in the scene.

[0041] The present invention takes into account that in the substation image fusion task, the features extracted by the general algorithm in the encoding stage show inadaptability in the substation equipment image fusion task, the feature semantic differentiation is weak, and it does not conform to the substation application scenario. It is easy to cause feature loss or redundancy during the fusion process, thereby affecting the image fusion quality. A more accurate feature decomposition method that is more in line with the substation equipment scenario is needed. This method proposes a device-scene feature decoupling network SEFDNet, which decomposes the source image features into scene features and device features, and designs the device temperature-texture constraint loss function ETTCLoss to enhance the feature decoupling capability of the model. The scene-device feature decoupling network SEFDNet includes:

[0042] First, the shallow features in the source image are extracted through the shallow feature encoder, which are infrared shallow features Shallow features of visible light ;

[0043] Then, the scene encoder and device encoder are used to decouple the infrared shallow features. Shallow features of visible light , extract infrared scene features and device characteristics , and visible light scene characteristics and device characteristics Among them, the infrared and visible light image device features are the key information of the device in the infrared and visible light images, namely the temperature information of the device in the infrared image and the texture information of the device in the visible light image. The infrared and visible light image scene features are the scene-related information in the infrared and visible light images. At the same time, as the device is part of the scene, the relationship between the device and the scene, such as the positional relationship, is also included in the scene feature expression information as supplementary information for the device in the scene feature.

[0044] For scene features, since different modal images are all taken in the same scene, the scene features of different modal images should be similar, so the scene consistency loss function is used , enhance the correlation of scene features of different modal images in the same scene, scene consistency loss function The expression is as follows:

[0045] ;

[0046] in, Indicates smoothness Norm, constrained infrared scene features Visible light scene characteristics correlation and allow for differences in sparsity due to different modalities;

[0047] For device features, the device temperature-texture constraint loss function ETTCLoss is designed to guide the extraction of infrared and visible light device features, and optimize the overall feature decoupling capability of the model. The infrared image and visible light image are converted into grayscale images respectively, and the device temperature information is extracted using the Canny algorithm and setting the grayscale threshold. With texture information , respectively guide the infrared device characteristics Visible light device characteristics Extraction, device temperature-texture constraint loss function ETTCLoss The expression is as follows:

[0048] ;

[0049] ;

[0050] ;

[0051] in, and Infrared device characteristics Visible light device characteristics Through the projection of point convolution on a single channel dimension, and are the infrared constraint component and visible light constraint component of ETTCLoss respectively, is the weight parameter, which ensures the uniformity of the magnitude of the components. It is the L2 norm, ensuring that the device features can accurately express the corresponding device information;

[0052] Finally, the decoder is used to reconstruct the scene features and device features of the same modality to obtain the reconstructed infrared image and reconstructing visible light images , design image reconstruction loss function to constrain the reconstruction of infrared image and reconstructing visible light images Approximate the original infrared image Compared with the original visible light image , in order to optimize the decoder's image reconstruction capability, for infrared images, the image reconstruction loss function It can be expressed as:

[0053] ;

[0054] Similarly, for visible light images, the image reconstruction loss function It can be expressed as:

[0055] ;

[0056] in, is the weight parameter, is the structural similarity loss function, which is used to minimize the structural difference between the reconstructed image and the source image. is the L2 norm, ensuring that the reconstructed image is close to the original image;

[0057] Combining the above loss functions, the total training loss function of the scene-device feature decoupling network SEFDNet is It can be expressed as:

[0058] ;

[0059] in, is a weight parameter used to ensure the uniformity of the magnitude of the components. Optimize the feature decoupling and image reconstruction capabilities of the device-scene feature decoupling network SEFDNet.

[0060] The present invention takes into account the need to express texture information from visible light images and temperature information from infrared images in the device area in the fused image, while suppressing scene features in the infrared image while expressing more visible light information in the scene area, making the relationship between the scene and the device in the fused image clearer and facilitating downstream device detection tasks. The present invention proposes a visible light scene enhancement fusion network VSEFNet based on the device thermal attention mechanism ETBAM, which suppresses infrared features in the scene area to enhance the expression of visible light scene features, and then fuses the processed scene features from different modal images with device features to generate the final fused image. The visible light scene enhancement fusion network VSEFNet includes:

[0061] First, the source image features are extracted by the SEFDNet pre-trained encoder, which are represented as infrared scene features below. and device characteristics , and visible light scene characteristics and device characteristics ; Design the device thermal attention mechanism ETBAM to suppress infrared scene features Scene related information, processed infrared scene features It can be expressed as:

[0062] ;

[0063] ;

[0064] in, and For infrared device characteristics After channel average pooling and maximum pooling, is a 7×7 convolution, is the sigmoid function, is element-by-element multiplication. Since infrared device features can usually reflect the area of ​​the device, based on the spatial attention mechanism, the device thermal attention map is obtained , and the device thermal attention map Perform element-by-element multiplication with the infrared scene features to retain the complementary information of the infrared scene features to the device and suppress the information of the infrared scene features in the scene area, thereby enhancing the expression of the visible light scene information;

[0065] Then, the scene feature fusion module and the device feature fusion module are used to fuse the scene features and device features of different modalities to obtain the fused scene feature and fusion device features ;

[0066] Then, the decoder pre-trained in the scene-device feature decoupling network SEFDNet is used to reconstruct the fused scene and fused device features into a fused image. ;

[0067] Finally, we design the content distribution loss function and the gradient maximum retention loss function. For the content distribution loss function, we include the variables and ;

[0068] in, It can be expressed as:

[0069] ;

[0070] in, , , They are infrared image, fusion image and partial image of visible light image in the device area. is a weight parameter used to control the distribution of infrared and visible light information within the device area. is the image content constraint loss function, which can be expressed as:

[0071] ;

[0072] in, is the weight parameter, , Represent two different images respectively;

[0073] Similarly, It can be expressed as:

[0074] ;

[0075] , , They are the infrared image, fused image and partial image of the visible light image in the scene area. is a weight parameter used to control the distribution of infrared and visible light information within the scene area;

[0076] For the maximum gradient retention loss function , which can be expressed as:

[0077] ;

[0078] in, is the gradient operator, is the absolute value function, is the maximum value function, for norm;

[0079] Combining the above loss functions, it can be expressed as:

[0080] ;

[0081] in, is a weight parameter used to ensure the uniformity of the components in magnitude, and to optimize the feature fusion and image generation capabilities of the visible light scene enhancement fusion network VSEFNet through the overall loss function.

[0082] In some embodiments of the present application, the defect recognition module is configured to identify equipment defects using a target detection model based on the infrared and visible light fusion image acquired by the data fusion module and output the results, including:

[0083] Based on the fused image obtained by the data fusion module, mark the defects according to their types;

[0084] The labeled fused images are divided into training and validation sets to train the object detection model YOLOv9;

[0085] The trained object detection model YOLOv9 is applied to the defect recognition module to achieve real-time monitoring of equipment defects and output results.

[0086] The defect types include:

[0087] Heating defects in current transformers, voltage transformers, insulators, and bushings in substation equipment;

[0088] Damage defects in substation equipment current transformers, voltage transformers, insulators and bushings;

[0089] Oil leakage defects in current transformers, voltage transformers, insulators and bushings of substation equipment.

[0090] In some embodiments of the present application, the early warning module is configured to determine the output result of the defect identification module. If the output result is greater than a risk threshold, the early warning module activates an audible and visual alarm based on the defect type of the equipment and simultaneously sends an early warning signal to the operation and maintenance terminal and the operation and maintenance platform, including:

[0091] If the output result is a heating defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, a heating warning will be issued for the corresponding equipment, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform at the same time;

[0092] If the output result is a damage defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, a damage warning for the corresponding equipment will be issued, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform at the same time;

[0093] If the output result is an oil leakage defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, an oil leakage warning will be issued for the corresponding equipment, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform.

[0094] By applying the above scheme, the present invention comprehensively considers the shortcomings of existing single-modal imagery in substation equipment defect warning and designs a substation equipment defect warning system based on image fusion technology of infrared and visible light images. By decoupling the features of the source image into device features and scene features, a fusion strategy tailored to these features is devised to improve the quality of the fused image and its effectiveness in downstream substation equipment defect identification tasks. The present invention first proposes a scene-device feature decoupling network (SEFDNet) and innovatively designs a device temperature-texture constraint loss function (ETTCLoss) to guide model optimization and enhance the effect of feature decoupling. Furthermore, a visible light scene enhancement fusion network (VSEFNet) based on the device thermal attention mechanism (ETBAM) is designed to suppress the expression of infrared image features in the scene, allowing the fused image to express more visible light scene information. Based on this, substation equipment defects are identified, their accuracy is improved, and early warnings are issued based on accurate defect identification, thereby ensuring safe production in substations.

[0095] Specifically, by accurately identifying and rapidly warning of high-temperature defects in voltage transformers, current transformers, insulators, and bushings, the system effectively avoids equipment failures caused by overheating, thereby eliminating the resulting abnormal substation operation, fire hazards, and personal safety risks. By accurately identifying and rapidly warning of mechanical damage defects in voltage transformers, current transformers, insulators, and bushings, the system effectively avoids equipment downtime and chain reactions due to structural failure. By accurately identifying and rapidly warning of leakage defects in voltage transformers, current transformers, insulators, and bushings, the system effectively prevents fires and environmental pollution caused by oil leaks.

[0096] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented through hardware or by using software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) and includes a number of instructions for enabling a computer device (such as a personal computer, a server, or a network device) to execute the methods described in various implementation scenarios of the present invention.

[0097] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

[0098] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0099] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that the process, method, article, or apparatus comprising a series of elements inherent to the elements, or also including elements inherent to these processes, methods, articles, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0100] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A real-time identification and early warning system for substation equipment defects based on image fusion technology, characterized by: include: The data acquisition module uses deployed mobile acquisition devices, infrared sensors, and visible light sensors to collect infrared and visible light images of substation equipment, including current transformers, voltage transformers, insulators, and bushings. A data fusion module is used to obtain a fused image of the infrared image and the visible light image of the substation equipment using a scene-device decoupling fusion algorithm based on the infrared image and the visible light image; The defect recognition module is used to identify equipment defects based on the fused image and use the target detection model, and output the results; The early warning module is used to judge the output result of the defect identification module. If the output result is greater than the risk threshold, it will activate the sound and light alarm according to the defect type of the equipment, and send a warning signal to the operation and maintenance terminal and operation and maintenance platform at the same time; The data fusion module specifically includes: The source images to be fused are acquired using a mobile acquisition device, including infrared images of substation equipment acquired by an infrared image sensor and visible light images of substation equipment acquired by a visible light sensor. All images are paired; Based on the paired infrared image and visible light image pairs, the scene-device decoupling fusion algorithm is used to obtain the fused image of the substation equipment infrared image and the visible light image; The scene-device decoupling fusion algorithm is used to obtain a fusion image of the infrared image and visible light image of the substation equipment, including: The pre-designed scene-device feature decoupling network SEFDNet is used to extract features from the fused source image to obtain infrared scene features. Features with infrared devices , and visible light scene characteristics Visible light device characteristics , in order to realize the decoupling of the source image scene features and the device features, and use the decoder to reconstruct the source image to obtain the reconstructed infrared image and reconstructing visible light images ; Using the pre-designed visible light scene enhancement fusion network VSEFNet, the visible light scene features are obtained through the scene-device feature decoupling network SEFDNet. and infrared scene characteristics and visible light device characteristics Features with infrared devices and use the decoder to generate a fused image ; The scene-device feature decoupling network SEFDNet includes: First, the shallow features in the source image are extracted through the shallow feature encoder, which are infrared shallow features Shallow features of visible light ; Then, the scene encoder and device encoder are used to decouple the infrared shallow features. Shallow features of visible light , extract infrared scene features and device characteristics , and visible light scene characteristics and device characteristics Among them, the infrared and visible light image device features are the key information of the device in the infrared and visible light images, namely the temperature information of the device in the infrared image and the texture information of the device in the visible light image. The infrared and visible light image scene features are the scene-related information in the infrared and visible light images. At the same time, as the device is part of the scene, the positional relationship between the device and the scene is also included in the scene feature expression information, which serves as supplementary information for the device in the scene feature. For scene features, since different modal images are all taken in the same scene, the scene features of different modal images should be similar, so the scene consistency loss function is used , enhance the correlation of scene features of different modal images in the same scene, scene consistency loss function The expression is as follows: ； in, Indicates smoothness Norm, constrained infrared scene features Visible light scene characteristics correlation and allow for differences in sparsity due to different modalities; For device features, the device temperature-texture constraint loss function ETTCLoss is designed to guide the extraction of infrared and visible light device features, and optimize the overall feature decoupling capability of the model. The infrared image and visible light image are converted into grayscale images respectively, and the device temperature information is extracted using the Canny algorithm and setting the grayscale threshold. With texture information , respectively guide the infrared device characteristics Visible light device characteristics Extraction, device temperature-texture constraint loss function ETTCLoss The expression is as follows: ； ； ； in, and Infrared device characteristics Visible light device characteristics Through the projection of point convolution on a single channel dimension, and are the infrared constraint component and visible light constraint component of ETTCLoss respectively, is the weight parameter, which ensures the uniformity of the magnitude of the components. It is the L2 norm, ensuring that the device features can accurately express the corresponding device information; Finally, the decoder is used to reconstruct the scene features and device features of the same modality to obtain the reconstructed infrared image and reconstructing visible light images , design image reconstruction loss function to constrain the reconstruction of infrared image and reconstructing visible light images Approximate the original infrared image Compared with the original visible light image , in order to optimize the decoder's image reconstruction capability, for infrared images, the image reconstruction loss function It can be expressed as: ； Similarly, for visible light images, the image reconstruction loss function It can be expressed as: ； in, is the weight parameter, is the structural similarity loss function, which is used to minimize the structural difference between the reconstructed image and the source image. is the L2 norm, ensuring that the reconstructed image is close to the original image; Combining the above loss functions, the total training loss function of the scene-device feature decoupling network SEFDNet is It can be expressed as: ； in, is a weight parameter used to ensure the uniformity of the magnitude of the components. Optimize the feature decoupling and image reconstruction capabilities of the device-scene feature decoupling network SEFDNet; The visible light scene enhancement fusion network VSEFNet includes: First, the source image features are extracted by the SEFDNet pre-trained encoder, which are represented as infrared scene features below. and device characteristics , and visible light scene characteristics and device characteristics ; Design the device thermal attention mechanism ETBAM to suppress infrared scene features Scene related information, processed infrared scene features It can be expressed as: ； ； in, and For infrared device characteristics After channel average pooling and maximum pooling, is a 7×7 convolution, is the sigmoid function, It is an element-by-element multiplication. Since the infrared device features can usually reflect the area of ​​the device, the device thermal attention map is obtained based on the spatial attention mechanism. , and the device thermal attention map Perform element-by-element multiplication with the infrared scene features to retain the complementary information of the infrared scene features to the device and suppress the information of the infrared scene features in the scene area, thereby enhancing the expression of the visible light scene information; Then, the scene feature fusion module and the device feature fusion module are used to fuse the scene features and device features of different modalities to obtain the fused scene feature and fusion device features ; Then, the decoder pre-trained in the scene-device feature decoupling network SEFDNet is used to reconstruct the fused scene and fused device features into a fused image. ; Finally, we design the content distribution loss function and the gradient maximum retention loss function. For the content distribution loss function, we include the variables and ; in, It can be expressed as: ； in, , , They are infrared image, fusion image and partial image of visible light image in the device area. is a weight parameter used to control the distribution of infrared and visible light information within the device area. is the image content constraint loss function, which can be expressed as: ； in, is the weight parameter, , Represent two different images respectively; Similarly, It can be expressed as: ； , , They are the infrared image, fused image and partial image of the visible light image in the scene area. is a weight parameter used to control the distribution of infrared and visible light information within the scene area; For the maximum gradient retention loss function , which can be expressed as: ； in, is the gradient operator, is the absolute value function, is the maximum value function, for norm; Combining the above loss functions, it can be expressed as: ； in, is a weight parameter used to ensure the uniformity of the components in magnitude, and to optimize the feature fusion and image generation capabilities of the visible light scene enhancement fusion network VSEFNet through the overall loss function.

2. The real-time identification and early warning system for substation equipment defects based on image fusion technology according to claim 1 is characterized in that: The data acquisition module specifically includes: The mobile collection device includes an aircraft or a robot vehicle equipped with a camera; the infrared sensor and the visible light sensor are configured on the mobile collection device, and follow the mobile collection device to collect infrared images and visible light images of the substation equipment.

3. The real-time identification and early warning system for substation equipment defects based on image fusion technology according to claim 1 is characterized in that: The defect identification module specifically includes: Based on the fused image obtained by the data fusion module, mark the defects according to their types; The labeled fused images are divided into training and validation sets to train the object detection model YOLOv9; The trained object detection model YOLOv9 is applied to the defect recognition module to achieve real-time monitoring of equipment defects and output results.

4. The real-time identification and early warning system for substation equipment defects based on image fusion technology according to claim 3 is characterized in that: The defect types include: Heating defects in current transformers, voltage transformers, insulators, and bushings in substation equipment; Damage defects in substation equipment current transformers, voltage transformers, insulators and bushings; Oil leakage defects in current transformers, voltage transformers, insulators and bushings of substation equipment.

5. The real-time identification and early warning system for substation equipment defects based on image fusion technology according to claim 4 is characterized in that: The early warning module specifically includes: If the output result is a heating defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, a heating warning will be issued for the corresponding equipment, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform at the same time; If the output result is a damage defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, a damage warning for the corresponding equipment will be issued, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform at the same time; If the output result is an oil leakage defect in the substation equipment current transformer, voltage transformer, insulator or bushing, and the output result is greater than the risk threshold, an oil leakage warning will be issued for the corresponding equipment, and a warning signal will be sent to the operation and maintenance terminal and operation and maintenance platform.

Images