A method, system, device and medium for defect detection of a gas insulated switchgear
By extracting infrared thermal imaging and visible light image features of gas-insulated switchgear using the CAFF-DINO model and the Swin-Transformer network, a cross-modal correlation feature map is generated, which solves the limitations and robustness problems of existing detection methods and achieves high-precision defect detection and location.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MAINTENANCE BRANCH OF STATE GRID HEBEI ELECTRIC POWER
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-05
AI Technical Summary
Existing defect detection methods for gas-insulated switchgear suffer from limitations in single-mode information, poor data correlation, and insufficient robustness, making it difficult to meet the high standards of intelligent operation and maintenance.
The CAFF-DINO model is adopted, and feature maps of infrared thermal imaging and high-resolution visible light images are extracted through the Swin-Transformer backbone network to generate cross-modal correlation feature maps. Channel stitching and fusion are then performed, and defect detection is performed using the DINO detection head.
It achieves high-precision collaborative detection of internal and external defects in gas-insulated switchgear, improving detection accuracy and reliability, and can identify defects such as overheating, cracks and discharges and provide accurate location.
Smart Images
Figure CN122156117A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of high-voltage electrical equipment monitoring technology, and in particular relates to a defect detection method, system, equipment and medium for gas-insulated switchgear. Background Technology
[0002] With the development of smart grid and online monitoring technology for high-voltage electrical equipment, online defect detection of gas-insulated switchgear (GIS) has become crucial for ensuring the safe operation of the power grid. Visible light imaging and infrared thermal imaging, as two mainstream technologies, are widely used in GIS inspections because they excel at capturing surface details and temperature anomalies, respectively.
[0003] In traditional technologies, defect detection for GIS equipment typically employs separate detection methods or simple multi-source data overlay analysis. A common approach is for maintenance personnel to conduct independent inspections of the equipment using infrared thermal imagers and visible light cameras, then comprehensively assess the equipment's condition through manual comparison or simple image overlay. Another more advanced traditional method involves registering infrared and visible light images during preprocessing, then employing early feature fusion strategies, such as simple image stitching at the input layer, or channel concatenation or weighted summation of feature maps after feature extraction, before inputting them into a convolutional neural network-based detection model for identification.
[0004] However, current detection methods or traditional multispectral fusion approaches have the following significant shortcomings: First, they suffer from limitations in single-modal information. Visible light cannot detect internal thermal anomalies, and infrared imaging is insensitive to minute surface defects; no single modality can provide a complete defect characterization. Second, they exhibit poor data correlation. Traditional fusion methods are mostly linear and static feature combinations, failing to delve into the deep semantic relationships and complementary information between infrared thermal features and visible light texture features, and ignoring the dynamic interaction and collaborative enhancement mechanisms between cross-modal features. Third, they are susceptible to interference from complex environments. GIS equipment has complex internal structures and severe component obstruction, and the on-site environment contains multiple noise sources. Traditional geometry-based registration algorithms and shallow fusion models lack robustness, easily leading to feature mismatches and false detections in complex scenarios, making it difficult to meet the high standards of intelligent operation and maintenance in terms of detection accuracy and reliability. Summary of the Invention
[0005] Therefore, it is necessary to provide a defect detection method, system, equipment, and medium for gas-insulated switchgear to address the aforementioned technical problems.
[0006] In a first aspect, this application provides a defect detection method for gas-insulated switchgear, comprising:
[0007] The infrared thermal image and high-resolution visible light image of the GIS equipment to be inspected are acquired, and the infrared thermal image and high-resolution visible light image are preprocessed and registered to obtain the registered infrared thermal image and the registered high-resolution visible light image.
[0008] The registered infrared thermal image and the registered high-resolution visible light image are input into the CAFF-DINO model. The heat distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image are extracted through the Swin-Transformer backbone network.
[0009] Based on thermal distribution feature maps and surface texture feature maps, a cross-modal correlation feature map is generated.
[0010] Channel concatenation is performed on the thermal distribution feature map, surface texture feature map, and cross-modal correlation feature map to obtain a stacked feature map;
[0011] Adjust the channel dimension of the stacked feature maps to the preset dimension to obtain the fused feature map;
[0012] The fused feature map is input into the DINO detection head to obtain the defect detection results of the GIS equipment. The defect detection results include defect category and defect location information. The defect category includes at least one of overheating, cracks and discharge. The defect location information includes equipment unit number and coordinates.
[0013] In one embodiment, a cross-modal correlation feature map is generated based on the thermal distribution feature map and the surface texture feature map, including:
[0014] Perform hierarchical cross-attention operations on the thermal distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level;
[0015] Intermediate association features are obtained based on cross-attention feature maps;
[0016] A cross-attention mechanism is used to enhance the intermediate association features, resulting in enhanced association features;
[0017] Channel compression is performed on the enhanced correlation features to generate a cross-modal correlation feature map.
[0018] In one embodiment, a hierarchical cross-attention operation is performed on the heat distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level, including:
[0019] Calculate the cross-attention feature map using the following formula:
[0020]
[0021] in, It is a cross-attention feature map. It is a query matrix for visible light modes. It is the bond matrix of the thermal infrared modes. It is the value matrix of the thermal infrared modes. It is the dimension of the key vector. The square root, It is a feature-level index, with a total of Each level It is the kernel size parameter. It is a normalization function.
[0022] In one embodiment, the channel dimension of the stacked feature maps is adjusted to a preset dimension to obtain a fused feature map, including:
[0023] The channel dimension of the stacked feature map is linearly compressed, reducing the number of channels from 2C+64 to C, resulting in a fused feature map; C is the preset number of feature channels of the Swin-Transformer backbone network.
[0024] In one embodiment, the infrared thermal image and the high-resolution visible light image are preprocessed and registered to obtain the registered infrared thermal image and high-resolution visible light image, including:
[0025] Obtain the 3D model of the GIS device;
[0026] Based on the 3D model, the region of interest (ROI) of key components in the GIS equipment is extracted;
[0027] Using the region of interest as a spatial reference, the infrared thermal image and the high-resolution visible light image are spatially registered to obtain the registered infrared thermal image and the high-resolution visible light image.
[0028] In one embodiment, the Swin-Transformer backbone network is a Swin-Large architecture;
[0029] The fusion module in the CAFF-DINO model does not inject the fused features back into the Swin-Transformer backbone network.
[0030] In one embodiment, the method further includes:
[0031] Based on the preset defect detection evaluation indicators, the defect detection results are verified to obtain the verified defect confidence level; the preset defect detection evaluation indicators include at least one of feature matching degree, defect region overlap degree and consistency of historical detection results.
[0032] Filter out defect detection results that are below the preset defect confidence threshold, and determine the defect detection results that are below the preset defect confidence threshold as incremental samples;
[0033] Based on incremental samples, the CAFF-DINO model is incrementally learned and retrained to obtain updated CAFF-DINO model parameters.
[0034] Secondly, this application also provides a defect detection system for gas-insulated switchgear, comprising:
[0035] The acquisition and registration module is used to acquire infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected, and to preprocess and register the infrared thermal images and high-resolution visible light images to obtain the registered infrared thermal images and registered high-resolution visible light images.
[0036] The feature extraction module is used to input the registered infrared thermal image and the registered high-resolution visible light image into the CAFF-DINO model, and extract the thermal distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image through the Swin-Transformer backbone network.
[0037] The correlation feature generation module is used to generate cross-modal correlation feature maps based on thermal distribution feature maps and surface texture feature maps;
[0038] The stacked feature generation module is used to perform channel concatenation of the heat distribution feature map, surface texture feature map and cross-modal correlation feature map to obtain a stacked feature map.
[0039] The fusion feature generation module is used to adjust the channel dimension of the stacked feature map to a preset dimension to obtain the fusion feature map;
[0040] The defect detection module is used to input the fused feature map into the DINO detection head to obtain the defect detection results of the GIS equipment. The defect detection results include defect category and defect location information. The defect category includes at least one of overheating, cracking and discharge. The defect location information includes equipment unit number and coordinates.
[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.
[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.
[0043] The aforementioned defect detection method, system, equipment, and medium for gas-insulated switchgear involves acquiring infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected, and preprocessing and registering these images to obtain registered infrared thermal images and registered high-resolution visible light images. These images are then input into a CAFF-DINO model, where a Swin-Transformer backbone network extracts the corresponding thermal distribution feature map from the registered infrared thermal image and the corresponding surface texture feature map from the registered high-resolution visible light image. Based on the thermal distribution feature map and the surface texture feature map, a cross-modal correlation feature map is generated. Channels of the thermal distribution feature map, surface texture feature map, and cross-modal correlation feature map are then stitched together to obtain a stacked feature map. The channel dimensions of the stacked feature map are adjusted to a preset dimension to obtain a fused feature map. The fused feature map is then input into a DINO detection head to obtain the defect detection result of the GIS equipment. The defect detection result includes defect category and defect location information. The defect category includes at least one of overheating, cracking, and discharge. The defect location information includes the equipment unit number and coordinates. This method enables high-precision collaborative detection of internal and external defects in GIS equipment. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 A schematic flowchart illustrating a defect detection method for a gas-insulated switchgear, provided as an exemplary embodiment of this application;
[0046] Figure 2 This is a schematic diagram of the structure of a defect detection system for a gas-insulated switchgear provided as an exemplary embodiment of this application. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0048] First, a brief introduction to the terms used in the embodiments of this application will be given.
[0049] GIS stands for Gas Insulated Switchgear.
[0050] CAFF, short for Cross-Attention Feature Fusion, is a deep feature fusion mechanism designed specifically for multimodal data. Its core idea is to use cross-attention to model the semantic associations and complementary relationships between features of different modalities, rather than simply concatenating or weighting them.
[0051] DINO, short for Divide and Conquer Vision Transformer, is a self-supervised vision pre-training and object detection framework based on the Transformer architecture.
[0052] Swin-Transformer, short for Shifted Window Transformer, is a hierarchical window attention mechanism Transformer model designed specifically for computer vision tasks.
[0053] The defect detection method for gas-insulated switchgear provided in this application embodiment can be widely applied to multiple core aspects of the operation and maintenance of GIS equipment throughout its entire life cycle, such as daily inspections, periodic preventive tests, pre-commissioning acceptance, post-fault diagnosis, and remote monitoring of smart grids.
[0054] In one embodiment, such as Figure 1 As shown, a defect detection method for gas-insulated switchgear is provided. This embodiment illustrates the application of this method to a detection terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the detection terminal and the server. In this embodiment, the method includes the following steps:
[0055] Step S101: Acquire infrared thermal imaging and high-resolution visible light images of the GIS device to be inspected, and preprocess and register the infrared thermal imaging and high-resolution visible light images to obtain the registered infrared thermal imaging and the registered high-resolution visible light images.
[0056] Infrared thermal imaging can capture the infrared radiation energy of various components of the GIS equipment to be inspected using an infrared thermal imager and convert it into a visualized thermal distribution image.
[0057] High-resolution visible light images can be optical images of the surface of the GIS equipment to be inspected.
[0058] Specifically, the detection terminal can simultaneously acquire infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected by the acquisition device. Then, the detection terminal performs preprocessing and registration operations on the two types of images, first completing basic image optimization, and then aligning them based on a preset spatial reference benchmark, finally obtaining the registered infrared thermal images and the registered high-resolution visible light images.
[0059] Step S102: Input the registered infrared thermal image and the registered high-resolution visible light image into the CAFF-DINO model, and extract the thermal distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image through the Swin-Transformer backbone network.
[0060] The CAFF-DINO model is a multimodal defect detection model that integrates CAFF and DINO detection heads and uses the Swin-Transformer as the feature extraction backbone. The CAFF-DINO model only receives pre-processed and registered infrared thermal images and high-resolution visible light images, outputting results in two stages, considering both intermediate features and the final detection target: Intermediate output: thermal distribution feature map generated by the Swin-Transformer backbone network and cross-modal correlation feature map generated by the CAFF module; Final output: GIS equipment defect detection results output by the DINO detection head. The CAFF-DINO model can simultaneously capture complementary information from infrared temperature anomalies and visible light surface structures through parallel Swin-Transformer branches, overcoming the limitations of single-modal defect characterization.
[0061] The thermal distribution feature map can be an abstract feature matrix that represents the temperature distribution pattern of various components of GIS equipment after feature extraction from the registered infrared thermal image through the Swin-Transformer backbone network.
[0062] A surface texture feature map can be an abstract feature matrix that represents the physical morphology and texture details of the surface of a GIS device, output by the Swin-Transformer backbone network after extracting features from a registered high-resolution visible light image.
[0063] Specifically, the detection terminal can input registered infrared thermal images and high-resolution visible light images into the CAFF-DINO model. This model is a multimodal defect detection model that integrates CAFF and DINO detection heads and uses the Swin-Transformer backbone. Subsequently, the detection terminal extracts features from the two types of input images through the Swin-Transformer backbone network within the model. For the infrared thermal images, it generates a heat distribution feature map characterizing the temperature distribution pattern of the equipment components, and for the high-resolution visible light images, it generates a surface texture feature map characterizing the physical morphology and texture details of the equipment surface.
[0064] Step S103: Generate a cross-modal correlation feature map based on the thermal distribution feature map and the surface texture feature map.
[0065] Among them, the cross-modal correlation feature map can be an abstract feature matrix generated by a specific cross-modal feature correlation algorithm based on the thermal distribution feature map and the surface texture feature map, used to characterize the semantic correspondence between infrared thermal information and visible light texture information.
[0066] Specifically, the detection terminal can use a pre-defined cross-modal feature association algorithm to perform feature association processing based on the extracted thermal distribution feature map and surface texture feature map. This algorithm can mine the semantic correspondence between features, deeply bind infrared thermal information and visible light texture information, and finally generate a cross-modal association feature map to characterize the semantic association of bimodal features.
[0067] Step S104: Channel splicing is performed on the heat distribution feature map, surface texture feature map and cross-modal correlation feature map to obtain a stacked feature map.
[0068] Among them, the stacked feature map can be a multi-dimensional composite feature matrix formed by splicing and integrating the heat distribution feature map, surface texture feature map and cross-modal correlation feature map according to the channel dimension.
[0069] Specifically, the detection terminal can merge the channel information of the thermal distribution feature map, surface texture feature map and cross-modal correlation feature map according to the channel dimension, and retain the complete dimension and semantic information of each feature map. It retains the temperature-related features of infrared thermal distribution and the surface texture features of visible light, and integrates the cross-semantic correlation features of infrared thermal imaging and high-resolution visible light images, and finally generates a stacked feature map.
[0070] Step S105: Adjust the channel dimension of the stacked feature map to a preset dimension to obtain the fused feature map.
[0071] The fused feature map can be a dual-modal collaborative feature matrix with a unified dimension, generated by adjusting the channel dimension of the stacked feature map to match the preset number of feature channels of the Swin-Transformer backbone network.
[0072] Specifically, the detection terminal can adjust the channel dimension of the stacked feature map, unifying the channel dimension of the stacked feature map to the same specification as the preset number of feature channels of the Swin-Transformer backbone network. While retaining the core semantic information of heat distribution, surface texture and cross-modal association in the stacked feature map, the normalization and integration of feature dimensions are completed, the redundancy of multi-channel features is eliminated, and finally a fused feature map is generated.
[0073] Step S106: Input the fused feature map into the DINO detection head to obtain the defect detection results of the GIS equipment; the defect detection results include defect category and defect location information; the defect category includes at least one of overheating, cracking and discharge; the defect location information includes equipment unit number and coordinates.
[0074] Specifically, the inspection terminal can input the generated fused feature map into the DINO inspection head built into the CAFF-DINO model. This inspection head performs defect identification and location analysis on the input fused feature map. Subsequently, the inspection terminal obtains the defect detection results corresponding to the GIS equipment to be inspected through the feature modeling and target detection logic of the DINO inspection head. The results contain two core types of information: defect category and defect location. The defect category can cover at least one of overheating, cracks, and discharge, and the defect location information is specified down to the equipment unit number and corresponding coordinates.
[0075] In the aforementioned defect detection method for gas-insulated switchgear, the detection terminal acquires infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected, and preprocesses and registers the infrared thermal images and high-resolution visible light images to obtain registered infrared thermal images and registered high-resolution visible light images. The registered infrared thermal images and registered high-resolution visible light images are input into the CAFF-DINO model, and the Swin-Transformer backbone network extracts the thermal distribution feature map corresponding to the registered infrared thermal images and the surface texture feature map corresponding to the registered high-resolution visible light images. Based on the thermal distribution feature map and the surface texture feature map, a cross-modal correlation feature map is generated. The thermal distribution feature map, the surface texture feature map, and the cross-modal correlation feature map are channel-stitched to obtain a stacked feature map. The channel dimensions of the stacked feature map are adjusted to a preset dimension to obtain a fused feature map. The fused feature map is input into the DINO detection head to obtain the defect detection result of the GIS equipment. The defect detection result includes defect category and defect location information. The defect category includes at least one of overheating, cracking, and discharge. The defect location information includes the equipment unit number and coordinates. This method enables high-precision collaborative detection of internal and external defects in GIS equipment.
[0076] In one embodiment, generating a cross-modal correlation feature map based on the thermal distribution feature map and the surface texture feature map may include the following steps:
[0077] Step S201: Perform hierarchical cross-attention operation on the heat distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level.
[0078] Among them, the cross-attention feature map can be an abstract feature matrix generated that represents the semantic association between heat distribution feature maps and surface texture feature maps at different levels.
[0079] Specifically, the detection terminal can perform cross-attention operations on the thermal distribution feature map and surface texture feature map at different feature levels according to the feature levels of the Swin-Transformer backbone network. By establishing the semantic mapping relationship between infrared thermal information and visible light texture information at each level, the cross-attention feature map at each level is finally generated.
[0080] Step S202: Based on the cross-attention feature map, obtain the intermediate association features.
[0081] Among them, intermediate association features can be a set of transitional features with preliminary bimodal semantic associations generated after integrating and processing cross-attention feature maps at all levels.
[0082] Specifically, the detection terminal can perform multi-dimensional aggregation processing on cross-attention feature maps at different levels. While retaining the core information of bimodal semantic association at each level, it can achieve preliminary fusion and normalization of features and finally generate intermediate association features.
[0083] Step S203: The intermediate association features are enhanced using a cross-attention mechanism to obtain the enhanced association features.
[0084] Among them, the associated features can be a set of features that have stronger bimodal semantic relevance and feature recognition after intermediate associated features are enhanced by cross-attention mechanism.
[0085] Specifically, the detection terminal can further explore the deep correlation of bimodal information within intermediate correlation features through cross-attention mechanisms, strengthen the response intensity of defect-related features, weaken invalid background information, and ultimately obtain enhanced correlation features.
[0086] Step S204: Channel compression is performed on the enhanced correlation features to generate a cross-modal correlation feature map.
[0087] Among them, the cross-modal association feature map can be an abstract feature matrix with fixed dimensions that can characterize the deep semantic association between infrared and visible light dual modes after channel compression of the enhanced association features.
[0088] Specifically, the detection terminal can simplify and normalize the channel dimension of the features while retaining the core information of bimodal semantic association in the enhanced association features, compressing them to a preset fixed dimension, and finally generating an abstract feature matrix that can represent the deep bimodal association, namely the cross-modal association feature map.
[0089] In this embodiment, the detection terminal achieves accurate mining and efficient integration of multi-level semantic associations of bimodal features through progressive processing of hierarchical cross-attention operations, multi-dimensional aggregation, feature enhancement and channel compression. This not only preserves the bimodal association information at different levels, but also enhances the identification of defect features, and finally generates a standardized cross-modal association feature map, laying a high-quality association feature foundation for subsequent multi-feature fusion.
[0090] In one embodiment, a hierarchical cross-attention operation is performed on the heat distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level, including:
[0091] Calculate the cross-attention feature map using the following formula:
[0092]
[0093] in, It is a cross-attention feature map. It is a query matrix for visible light modes. It is the bond matrix of the thermal infrared modes. It is the value matrix of the thermal infrared modes. It is the dimension of the key vector. The square root, It is a feature-level index, with a total of Each level It is the kernel size parameter. It is a normalization function.
[0094] Specifically, It is a cross-attention feature map used to represent the first... The first feature level, the first Cross-attention outputs of kernel size; It is a query matrix for visible light modes, derived from the first... Layer feature map After the first One convolutional kernel is generated; It is the bond matrix of the thermal infrared modes, composed of the first... Layer feature map After the first Each convolutional kernel is generated and then transposed. It is the value matrix of the thermal infrared modes, derived from the first... Layer feature map After the first One convolutional kernel is generated; It is the dimension of the key vector. The square root of; It is a normalization function that can convert similarity scores into a probability distribution.
[0095] In this embodiment, the detection terminal constructs semantic associations between visible light and infrared modal features through a hierarchical cross-attention formula, realizing the matching and information interaction of dual-modal features at different levels and different convolutional kernel scales. This effectively mines the complementary association between thermal distribution and surface texture features, providing a high-precision hierarchical association feature foundation for the subsequent generation of cross-modal association feature maps.
[0096] In one embodiment, the channel dimension of the stacked feature maps is adjusted to a preset dimension to obtain a fused feature map, including:
[0097] The channel dimension of the stacked feature map is linearly compressed, reducing the number of channels from 2C+64 to C, resulting in a fused feature map; C is the preset number of feature channels of the Swin-Transformer backbone network.
[0098] In this embodiment, the detection terminal performs a channel dimension linear compression operation on the stacked feature map, reducing the number of channels from 2C+64 to the preset number of feature channels C of the Swin-Transformer backbone network. While preserving the core semantic information of heat distribution, surface texture and cross-modal correlation, it achieves feature dimension normalization, eliminates multi-channel feature redundancy, and generates a fusion feature map that adapts to the input requirements of the DINO detection head, providing a standardized and lightweight collaborative feature carrier for subsequent high-precision defect detection.
[0099] In one embodiment, preprocessing and registering the infrared thermal image and the high-resolution visible light image to obtain the registered infrared thermal image and high-resolution visible light image may include the following steps:
[0100] Step S301: Obtain the 3D model of the GIS equipment.
[0101] Among them, the three-dimensional model can be a three-dimensional spatial topology model formed by digitally modeling the overall structure, spatial location and shape of the key components of the GIS equipment to be inspected.
[0102] Specifically, the detection terminal can retrieve the standard 3D model corresponding to the GIS device to be detected from the pre-stored device model database, or obtain the 3D model in real time from the cloud model platform through a network interface.
[0103] Step S302: Based on the 3D model, extract the regions of interest of key components in the GIS equipment.
[0104] The region of interest can be a specific spatial area defined based on the 3D model of the GIS equipment, containing critical components that are prone to defects, such as circuit breakers, disconnect switches, and insulators.
[0105] Specifically, the detection terminal can identify key components with high defect incidence in the 3D model, including equipment units such as circuit breakers, disconnect switches, and insulators; then, based on the spatial coordinates and structural dimensions of each component in the model, it delineates the spatial range corresponding to each key component and generates a region of interest with precise coordinate boundaries; at the same time, it numbers and labels the regions with attributes to clarify the equipment unit type corresponding to each region.
[0106] Step S303: Using the region of interest as a spatial reference, spatial registration is performed between the infrared thermal image and the high-resolution visible light image to obtain the registered infrared thermal image and high-resolution visible light image.
[0107] Specifically, the detection terminal can extract feature points corresponding to infrared thermal imaging and high-resolution visible light images, match them with the spatial coordinates of the region of interest in the 3D model, and establish a mapping relationship between image pixels and 3D spatial coordinates. Then, the spatial transformation algorithm is used to calibrate the positions of the two types of images to eliminate spatial offsets caused by factors such as shooting angle and device posture. Finally, the registered infrared thermal imaging and high-resolution visible light images with the spatial positions of each key component are completely aligned are obtained.
[0108] In this embodiment, the detection terminal achieves high-precision spatial alignment between infrared thermal imaging and high-resolution visible light images through a progressive preprocessing process. This eliminates spatial offset caused by factors such as shooting angle and device posture, laying a spatially consistent image data foundation for subsequent feature extraction and correlation fusion, and effectively improving the reliability of early-stage data in the overall defect detection process.
[0109] In one embodiment, the Swin-Transformer backbone network is a Swin-Large architecture;
[0110] The fusion module in the CAFF-DINO model does not inject the fused features back into the Swin-Transformer backbone network.
[0111] In this embodiment, the detection terminal adopts the Swin-Large architecture as the backbone network for feature extraction, and sets the architecture design so that the CAFF-DINO model fusion module does not inject the fused features back into the backbone network. This ensures the depth and accuracy of feature extraction, and also achieves the isolation between the feature extraction and fusion modules. It can directly reuse pre-trained weights and improve modal interchangeability, providing an efficient and flexible model foundation for subsequent cross-modal feature association and defect detection.
[0112] In one embodiment, the method may further include the following steps:
[0113] Step S401: Based on the preset defect detection evaluation indicators, the defect detection results are verified to obtain the verified defect confidence level; the preset defect detection evaluation indicators include at least one of feature matching degree, defect area overlap degree and consistency of historical detection results.
[0114] Among them, the preset defect detection evaluation index can be a multi-dimensional quantitative evaluation standard for verifying the accuracy and reliability of defect detection results, covering at least one of feature matching degree, defect area overlap degree, and consistency of historical detection results.
[0115] The verified defect confidence level can be a quantitative value representing the credibility of the detection results after the initial defect detection results are substituted into preset evaluation indicators to complete the comprehensive verification. The higher the value, the higher the degree of fit between the detection results and the actual defect situation.
[0116] Specifically, the detection terminal can compare and calculate the defect detection results with the preset defect detection evaluation indicators one by one. It verifies the fit between defect features and standard features through feature matching degree, confirms the accuracy of defect location by defect area overlap degree, and verifies the data consistency by historical detection results consistency. Finally, it integrates the scores of various indicators to generate the verified defect confidence level.
[0117] Step S402: Filter out defect detection results that are below the preset defect confidence threshold, and determine the defect detection results that are below the preset defect confidence threshold as incremental samples.
[0118] Among them, incremental samples can be sample data corresponding to defect detection results whose confidence values are lower than a preset threshold after defect confidence verification.
[0119] Specifically, the detection terminal can retrieve a preset defect confidence threshold and compare the verification confidence of each defect detection result with the threshold one by one; then, it can filter out all defect detection results with confidence below the threshold and associate them with their corresponding registered infrared thermal images, high-resolution visible light images and full-process feature data; finally, it can standardize and encapsulate these low-confidence detection results and associated data and mark them as incremental samples.
[0120] Step S403: Based on the incremental samples, perform incremental learning and retraining on the CAFF-DINO model to obtain the updated CAFF-DINO model parameters.
[0121] Specifically, the detection terminal can retrieve the current parameters of the CAFF-DINO model as the initial weights for training. Then, it divides the incremental samples into training subsets and validation subsets according to a preset ratio. At the same time, it configures training strategies adapted to incremental learning, including low learning rate fine-tuning, freezing the parameters of the backbone network's basic layers, and updating only the parameters related to the fusion module and the detection head.
[0122] Furthermore, the detection terminal inputs incremental samples into the CAFF-DINO model for iterative training, using defect detection accuracy and confidence as the core verification indicators, and verifies the training effect of each round on the verification subset; finally, the model parameters corresponding to the optimal training round are selected to obtain the updated CAFF-DINO model parameters.
[0123] In this embodiment, the detection terminal uses a closed-loop optimization process of multi-index verification of defect detection results, screening and encapsulating low-confidence results into incremental samples, and targeted incremental learning and retraining of the model. This process not only achieves quantitative evaluation of the credibility of defect detection results, but also completes lightweight iterative optimization of the CAFF-DINO model through incremental samples. This effectively improves the model's ability to identify low-confidence defect scenarios and ensures the adaptability of the defect detection system.
[0124] In a practical application scenario for defect detection of gas-insulated switchgear, after acquiring infrared and visible light images of the GIS equipment collected by the acquisition device, the detection terminal extracts the region of interest (ROI) of key components (such as circuit breakers and disconnectors) based on the 3D model of the GIS equipment, performs spatial registration, and obtains the registered dual-modal image, which then enters the subsequent core detection process, as follows:
[0125] Step 1: Infrared and Visible Light Feature Extraction. Surface texture features (cracks, corrosion) and thermal distribution features (temperature gradient, hotspots) are extracted using the Swin-Transformer. At each stage of feature extraction, a component called the Cross-Attention Feature Fusion Module (CAFF) performs hierarchical cross-modal attention operations from infrared features to visible light features. By employing a multi-kernel cross-attention mechanism, the detection terminal focuses on multi-scale feature extraction. Using the following formula, the final associated information is fused with single-modal features. The cross-attention operation formula for associating thermal modal features with visible spectral features is as follows:
[0126]
[0127] In the formula, It is a cross-attention feature map. It is a query matrix for visible light modes. It is the bond matrix of the thermal infrared modes. It is the value matrix of the thermal infrared modes. It is the dimension of the key vector. The square root, It is a feature-level index, with a total of Each level It is the kernel size parameter. It is a normalization function.
[0128] Step 2: Concatenate the visible light feature map, thermal modality feature map, and cross-modal correlation feature map channels to obtain a stacked feature map. First, stack and compress the generated cross-attention matrix using a convolutional layer of depth C=256. To enhance the expressive power of feature correlation, multiple self-attention modules are superimposed on the fused cross-attention map. Then, a single-layer convolution is used to compress the correlation information to 64 channels, denoted as […]. Finally, the stacked feature map is obtained according to the following formula:
[0129]
[0130] In the formula, Indicates the first The stacked features after layer fusion consist of visible light features, thermal infrared features, and the correlation features between the two. This indicates the visible light mode in the 1st... The feature map extracted from the layer captures surface details (such as cracks and rust). Indicates the thermal infrared mode in the first... The feature map extracted from the layer reflects the temperature distribution (such as local overheating and thermal gradient). This represents the cross-modal correlation features calculated through the cross-attention mechanism, characterizing the semantic correlation between visible light and thermal infrared features.
[0131] Step 3: Apply a 1×1 convolution operation to the concatenated features, and use the following formula to adjust the dimension of the feature vector to meet the input requirements of the Transformer encoder head:
[0132]
[0133] In the formula, This represents the fused feature map. It represents the first... The output of the input features after the fusion operation has C channels; This represents a 1x1 convolution operation, which compresses the number of input channels from 2C+64 to C while maintaining the spatial resolution (height and width remain unchanged). Indicates the first The stacked features after layer fusion consist of visible light features, thermal infrared features, and the correlation features between the two.
[0134] Step 4: Associate thermal features and texture features at multiple scales to generate a fused feature map.
[0135] Step 5: Input the fused features into the DINO detection head. Based on the fused feature map, the DINO detection head outputs the defect category (such as overheating, crack, discharge) and location (equipment unit number, coordinates).
[0136] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0137] Based on the same inventive concept, this application also provides a defect detection system for implementing the aforementioned gas-insulated switchgear. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more gas-insulated switchgear defect detection system embodiments provided below can be found in the above-described limitations of a gas-insulated switchgear defect detection method, and will not be repeated here.
[0138] In one exemplary embodiment, such as Figure 2 As shown, a defect detection system 500 for gas-insulated switchgear is provided, comprising:
[0139] The acquisition and registration module 501 is used to acquire infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected, and to preprocess and register the infrared thermal images and high-resolution visible light images to obtain the registered infrared thermal images and registered high-resolution visible light images.
[0140] The feature extraction module 502 is used to input the registered infrared thermal image and the registered high-resolution visible light image into the CAFF-DINO model, and extract the heat distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image through the Swin-Transformer backbone network.
[0141] The correlation feature generation module 503 is used to generate a cross-modal correlation feature map based on the heat distribution feature map and the surface texture feature map;
[0142] The stacked feature generation module 504 is used to perform channel concatenation on the heat distribution feature map, surface texture feature map and cross-modal correlation feature map to obtain a stacked feature map.
[0143] The fusion feature generation module 505 is used to adjust the channel dimension of the stacked feature map to a preset dimension to obtain the fusion feature map;
[0144] The defect detection module 506 is used to input the fused feature map into the DINO detection head to obtain the defect detection results of the GIS equipment. The defect detection results include defect category and defect location information. The defect category includes at least one of overheating, cracking and discharge. The defect location information includes equipment unit number and coordinates.
[0145] In one embodiment, the associated feature generation module includes:
[0146] Attention operation unit, used to perform hierarchical cross-attention operation on heat distribution feature map and surface texture feature map to obtain cross-attention feature map at each level;
[0147] The intermediate association feature generation unit is used to obtain intermediate association features based on cross-attention feature maps;
[0148] The cross-attention enhancement unit for related features is used to enhance intermediate related features using a cross-attention mechanism to obtain enhanced related features;
[0149] The cross-modal association feature map generation unit is used to perform channel compression on the enhanced association features to generate a cross-modal association feature map.
[0150] In one embodiment, the attention operation unit performs hierarchical cross-attention operations on the heat distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level, including:
[0151] Calculate the cross-attention feature map using the following formula:
[0152]
[0153] in, It is a cross-attention feature map. It is a query matrix for visible light modes. It is the bond matrix of the thermal infrared modes. It is the value matrix of the thermal infrared modes. It is the dimension of the key vector. The square root, It is a feature-level index, with a total of Each level It is the kernel size parameter. It is a normalization function.
[0154] In one embodiment, the feature generation module includes:
[0155] The channel dimension of the stacked feature map is linearly compressed, reducing the number of channels from 2C+64 to C, resulting in a fused feature map; C is the preset number of feature channels of the Swin-Transformer backbone network.
[0156] In one embodiment, the acquisition and registration module includes:
[0157] The 3D model acquisition unit is used to acquire the 3D model of the gas-insulated switchgear.
[0158] The region extraction unit is used to extract the region of interest (ROI) of key components in GIS equipment based on a 3D model.
[0159] The spatial registration unit is used to spatially register the infrared thermal image and the high-resolution visible light image, using the region of interest as a spatial reference, to obtain the registered infrared thermal image and the high-resolution visible light image.
[0160] In one embodiment, the Swin-Transformer backbone network is a Swin-Large architecture;
[0161] The fusion module in the CAFF-DINO model does not inject the fused features back into the Swin-Transformer backbone network.
[0162] In one embodiment, the system further includes:
[0163] The detection result verification module is used to verify the defect detection results based on preset defect detection evaluation indicators to obtain the verified defect confidence level. The preset defect detection evaluation indicators include at least one of feature matching degree, defect area overlap degree and consistency of historical detection results.
[0164] The incremental sample determination module is used to filter out defect detection results that are below a preset defect confidence threshold and determine the defect detection results that are below the preset defect confidence threshold as incremental samples;
[0165] The model retraining module is used to perform incremental learning and retraining on the CAFF-DINO model based on incremental samples to obtain updated CAFF-DINO model parameters.
[0166] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a defect detection method for a gas-insulated switchgear as described above.
[0167] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0168] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0169] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.
Claims
1. A defect detection method for gas-insulated switchgear, characterized in that, The method includes: Infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected are acquired, and the infrared thermal images and high-resolution visible light images are preprocessed and registered to obtain registered infrared thermal images and registered high-resolution visible light images. The registered infrared thermal image and the registered high-resolution visible light image are input into the CAFF-DINO model, and the thermal distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image are extracted through the Swin-Transformer backbone network. Based on the thermal distribution feature map and the surface texture feature map, a cross-modal correlation feature map is generated; The heat distribution feature map, the surface texture feature map, and the cross-modal correlation feature map are concatenated by channels to obtain a stacked feature map; The channel dimension of the stacked feature map is adjusted to a preset dimension to obtain a fused feature map; The fused feature map is input into the DINO detection head to obtain the defect detection results of the GIS equipment; the defect detection results include defect category and defect location information; the defect category includes at least one of overheating, cracking and discharge; the defect location information includes equipment unit number and coordinates.
2. The method according to claim 1, characterized in that, The step of generating a cross-modal correlation feature map based on the thermal distribution feature map and the surface texture feature map includes: Perform hierarchical cross-attention operations on the heat distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level; Based on the cross-attention feature map, intermediate association features are obtained; The intermediate association features are enhanced using a cross-attention mechanism to obtain the enhanced association features; Channel compression is performed on the enhanced correlation features to generate the cross-modal correlation feature map.
3. The method according to claim 2, characterized in that, The hierarchical cross-attention operation performed on the thermal distribution feature map and the surface texture feature map to obtain cross-attention feature maps at each level includes: The cross-attention feature map is calculated using the following formula: in, It is a cross-attention feature map. It is a query matrix for visible light modes. It is the bond matrix of the thermal infrared modes. It is the value matrix of the thermal infrared modes. It is the dimension of the key vector. The square root, It is a feature-level index, with a total of Each level It is the kernel size parameter. It is a normalization function.
4. The method according to claim 1, characterized in that, The step of adjusting the channel dimension of the stacked feature map to a preset dimension to obtain the fused feature map includes: The channel dimension of the stacked feature map is linearly compressed, reducing the number of channels from 2C+64 to C, to obtain the fused feature map; where C is the preset number of feature channels of the Swin-Transformer backbone network.
5. The method according to any one of claims 1 to 4, characterized in that, The process of preprocessing and registering the infrared thermal image and the high-resolution visible light image to obtain the registered infrared thermal image and high-resolution visible light image includes: Obtain the 3D model of the GIS device; Based on the 3D model, the regions of interest for key components in the GIS equipment are extracted; Using the region of interest as a spatial reference, the infrared thermal image and the high-resolution visible light image are spatially registered to obtain the registered infrared thermal image and the high-resolution visible light image.
6. The method according to claim 1, characterized in that: The Swin-Transformer backbone network is a Swin-Large architecture; The fusion module in the CAFF-DINO model does not inject the fused features back into the Swin-Transformer backbone network.
7. The method according to claim 1, characterized in that, The method further includes: Based on preset defect detection evaluation indicators, the defect detection results are verified to obtain the verified defect confidence level; the preset defect detection evaluation indicators include at least one of feature matching degree, defect region overlap degree and consistency of historical detection results. Defect detection results that are below the preset defect confidence threshold are selected and identified as incremental samples. Based on the incremental samples, the CAFF-DINO model is incrementally learned and retrained to obtain the updated CAFF-DINO model parameters.
8. A defect detection system for gas-insulated switchgear, characterized in that, The system includes: The acquisition and registration module is used to acquire infrared thermal images and high-resolution visible light images of the GIS equipment to be inspected, and to preprocess and register the infrared thermal images and the high-resolution visible light images to obtain registered infrared thermal images and registered high-resolution visible light images. The feature extraction module is used to input the registered infrared thermal image and the registered high-resolution visible light image into the CAFF-DINO model, and extract the thermal distribution feature map corresponding to the registered infrared thermal image and the surface texture feature map corresponding to the registered high-resolution visible light image through the Swin-Transformer backbone network. The correlation feature generation module is used to generate a cross-modal correlation feature map based on the heat distribution feature map and the surface texture feature map; The stacked feature generation module is used to perform channel concatenation on the heat distribution feature map, the surface texture feature map, and the cross-modal correlation feature map to obtain a stacked feature map. The fusion feature generation module is used to adjust the channel dimension of the stacked feature map to a preset dimension to obtain the fusion feature map; The defect detection module is used to input the fused feature map into the DINO detection head to obtain the defect detection result of the GIS equipment; the defect detection result includes defect category and defect location information; the defect category includes at least one of overheating, cracking and discharge; the defect location information includes equipment unit number and coordinates.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.