A method and equipment for identifying defects in ultra-high voltage valve halls using multispectral methods

Abstract

This invention relates to a multispectral UHV valve hall defect identification method, comprising: acquiring multispectral image data of devices within the UHV valve hall and preprocessing it; constructing a multispectral feature extraction network; designing a cross-modal feature fusion network; inputting the fused features into an improved YOLOv11 network; forming a defect identification model; and training the defect identification model using a three-stage progressive framework. This invention constructs a high-quality multimodal training dataset through spatiotemporal synchronous registration and adaptive data augmentation strategies, significantly improving the model's generalization ability and achieving high-precision collaborative identification of multiple types of defects such as cracks, corrosion, and discharge. The multimodal identification accuracy remains greater than 90%, a 30% improvement over single-spectral methods. Based on the YOLO framework, it meets the real-time monitoring requirements of valve halls and has been successfully applied to UHV valve hall online monitoring systems, providing efficient and reliable technical support for intelligent operation and maintenance of power equipment, and possessing significant engineering application value and market prospects.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for power equipment, and in particular to a method and device for identifying defects in a multispectral ultra-high voltage valve hall. Background Technology

[0002] As a core hub of the power transmission system, the ultra-high voltage valve hall plays a crucial role in power conversion and transmission, and the stable operation of its internal equipment is directly related to the safety of the power grid. The equipment within the valve hall is subjected to extreme environments of high voltage, strong electromagnetic fields, and high temperature and humidity for extended periods, making it prone to mechanical, electrical, and thermal defects. Failure to detect these defects in a timely manner could lead to major accidents. Current detection technologies primarily rely on single-spectral methods: visible light imaging is susceptible to light interference, resulting in a false detection rate of up to 25%; infrared thermal imaging has low spatial resolution, making it difficult to locate defects smaller than 1 mm; and ultraviolet imaging lacks spatial information and cannot be accurately correlated with equipment structure. While multispectral fusion technology is gradually being applied, it is still limited by characteristic heterogeneity, slow processing speed, and sensitivity to environmental interference, making it difficult to meet real-time monitoring requirements.

[0003] With the development of deep learning technology, CNN and Transformer-based models have shown potential in multispectral detection, but still face significant challenges. Traditional CNNs excel at extracting local features from infrared images but struggle to model the global texture of visible light. While Vision Transformer performs well in classification tasks, its self-attention mechanism results in a quadratic increase in computational complexity for high-resolution data, requiring over 16GB of GPU memory, making it difficult to deploy on edge devices. Furthermore, existing methods often neglect the temporal characteristics of ultraviolet pulse signals, treating them as static images, leading to the loss of crucial information such as discharge frequency. Mainstream multispectral fusion strategies are prone to feature conflicts due to semantic gaps between modalities; directly splicing visible light and infrared features reduces classification accuracy by 12%, and the massive data volume of multi-branch networks further reduces inference speed to less than 10 FPS. Meanwhile, over 70% of defects in ultra-high-voltage valve halls are tiny targets smaller than 5mm; existing models suffer from detail loss due to downsampling in deep networks and rigid anchor frame design, resulting in an average accuracy of less than 65%, severely limiting detection performance.

[0004] To address the aforementioned issues, there is an urgent need for a novel multispectral defect identification method that deeply integrates complementary information from visible light, infrared, and ultraviolet data to cover various types of defects, including mechanical, electrical, and thermal defects. Summary of the Invention

[0005] To address the issues of low accuracy of visible light image sensor identification equipment and limitations in single-mode infrared and ultraviolet defect identification caused by strong electromagnetic interference in ultra-high voltage valve halls and tilted shooting angles of elevated equipment, the primary objective of this invention is to provide a multispectral ultra-high voltage valve hall defect identification method that can achieve high-precision collaborative identification of multiple types of defects such as cracks, corrosion, and discharge, and has high multi-mode identification accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a multispectral ultra-high voltage valve hall defect identification method, which includes the following sequential steps:

[0007] (1) Collect multispectral image data of the device in the UHV valve hall and perform preprocessing. The multispectral image data includes visible light images, infrared images and ultraviolet time series data, and perform data enhancement.

[0008] (2) Construct a multispectral feature extraction network, which includes a CNN module for extracting spatial features of infrared images, a Transformer module for extracting global features of visible light images, and a BiLSTM module for extracting temporal features of ultraviolet light. The spatial features of infrared images, the global features of visible light images, and the temporal features of ultraviolet light constitute multispectral features.

[0009] (3) Design a cross-modal feature fusion network to adaptively fuse multispectral features to obtain fused features; the cross-modal feature fusion network includes a channel attention module, a cross-modal interaction module, a temporal attention module and a splicing module;

[0010] (4) Input the fused features into the improved YOLOv11 network, and the improved YOLOv11 network outputs the defect classification and localization results;

[0011] (5) The defect identification model is composed of a multispectral feature extraction network, a cross-modal feature fusion network, and an improved YOLOv11 network;

[0012] (6) A three-stage progressive framework is adopted to train the defect recognition model: First stage: single-modality specific pre-training is carried out to independently optimize the feature extraction capabilities of the CNN module, Transformer module, and BiLSTM module; Second stage: the CNN module, Transformer module, and BiLSTM module are alternately unfrozen, and adversarial training is carried out between the modules to correct errors in detection; Third stage: the defect recognition model is iteratively trained in a loop, and the network parameters are jointly optimized through a multi-task loss function to complete the training.

[0013] Step (1) specifically includes the following steps in sequence:

[0014] (1a) Collect 5000 sets of multispectral data samples to form a dataset, label the defect types and location information, and obtain the labeled dataset. The defect types include cracks, corrosion and discharge.

[0015] (1b) Divide the labeled dataset into training set, validation set and test set in a ratio of 6:2:2;

[0016] (1c) The PWC-Net optical flow algorithm is used to align visible light images, infrared images and ultraviolet temporal data to ensure pixel-level spatiotemporal consistency and achieve spatiotemporal synchronous registration;

[0017] (1d) Gaussian noise was added to the infrared image to simulate thermal interference, Mosaic enhancement was applied to the visible light image, and sliding window truncation was applied to the ultraviolet time series data to achieve data enhancement.

[0018] Step (2) specifically refers to: the CNN module adopts a lightweight GhostNet structure, the input infrared image size is 640×640, the CNN module outputs a 256×256×64 feature map, focusing on local hot spots and temperature gradient features; the Transformer module is based on the Swin-Tiny architecture, the window size is 8×8, and the output multi-scale feature map is used to capture global texture information such as cracks and corrosion in visible light images; the input of the BiLSTM module is ultraviolet time-series data signal, the length of the ultraviolet time-series data signal is 100 frames, each frame contains pulse intensity and frequency, the hidden layer dimension of the BiLSTM module is 128, and the output of the BiLSTM module is a time-series feature vector.

[0019] Step (3) specifically includes the following steps in sequence:

[0020] (3a) Use the channel attention module to perform channel weighting on the infrared features, as shown in the following formula:

[0021] ;

[0022] In the formula, , All are weights of fully connected layers. For global average pooling; For adaptive learning weights, For the input infrared features, This is the output of the channel attention module;

[0023] (3b) Use the cross-modal interaction module to achieve semantic alignment of visible light and infrared features:

[0024] ;

[0025] in, , , ; For the output of the cross-modal interaction module; Scaling factor ; For visible light projection weights; It has visible light characteristics; For infrared bond projection weights, The projection weights are the infrared values.

[0026] (3c) Use the temporal attention module to filter keyframes for ultraviolet temporal features, using the following formula:

[0027] ;

[0028] in, For BiLSTM hidden state, This is the attention weight matrix; Normalized attention weights for time steps, ; The feature vector is weighted by temporal attention; t is the temporal index; T is the total temporal length;

[0029] (3d) The features are fused by the splicing module. The fused features are compressed by depthwise separable convolution to output a unified dimension feature map. The depthwise separable convolution has 3 kernels and a stride of 1.

[0030] Step (4) specifically includes the following steps in sequence:

[0031] (4a) The SPPF layer of the YOLOv11 network is replaced by the adaptive spatial pyramid pooling ASPP module, which enhances the multi-scale defect perception capability through dilated convolution. The adaptive spatial pyramid pooling ASPP module uses three sets of parallel dilated convolutional layers with dilation rates of 6, 12 and 18, respectively. The adaptive spatial pyramid pooling ASPP module outputs a multi-scale feature map with 256 channels.

[0032] (4b) The detection head adopts a dynamic anchor frame mechanism, which dynamically adjusts the anchor frame size according to the multispectral characteristics. The dynamic anchor frame mechanism refers to the following: the initial anchor frame of the infrared branch is 32×32, 64×64, 128×128, which is suitable for large-scale hot spots; the anchor frame of the visible light branch is 16×16, 32×32, 64×64, which is suitable for small-scale cracks.

[0033] In step (6), the multi-task loss function for:

[0034] ;

[0035] In the formula, To improve CIoU positioning loss, To classify the focus loss, Loss of consistency of multispectral features;

[0036] The The formula is:

[0037] ;

[0038] in, The distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. The length of the diagonal of the minimum bounding rectangle; Indicates the coordinates of the prediction box. Represents the coordinates of the true bounding box. It is a dynamic hyperparameter used to balance the impact of different parts of the loss on the total loss; This indicates the aspect ratio difference between the predicted bounding box and the ground truth bounding box; For intersection, union, and comparison;

[0039] The The formula is:

[0040] ;

[0041] in, ; This represents the combined value of the model's predicted positive sample probability and the true label.

[0042] The The formula is:

[0043] ;

[0044] In the formula, This represents the fused multispectral feature vector. This represents the eigenvector of a single spectrum.

[0045] Step (6) specifically includes the following steps in sequence:

[0046] (6a) First stage: Perform single-modality specific pre-training to independently optimize the feature extraction capabilities of each spectral module; the CNN module only inputs infrared images and corresponding defect labels, the Transformer module only inputs visible light images and corresponding defect labels, and the BiLSTM module only inputs ultraviolet time-series signals and corresponding defect labels; the training loss of the CNN module and the Transformer module is the focus loss, and the training loss of the BiLSTM module is the mean square error loss; the optimizer used in the training process of the CNN module, the Transformer module, and the BiLSTM module is AdamW, with a learning rate of 1×10. -4The batch size is 16, and the training epochs are 100. During training, the detection head parameters of the cross-modal feature fusion network and the improved YOLOv11 network are frozen.

[0047] (6b) Second stage: Adversarial training is conducted using an alternating unfreeze strategy, with adversarial error correction between modules to suppress cross-modal conflicts; the alternating unfreeze strategy includes three sub-stages:

[0048] Sub-phase 1: Unfreeze the CNN module during training, and freeze the Transformer and BiLSTM modules;

[0049] Sub-phase 2: Unfreeze the Transformer module during training, and freeze the CNN and BiLSTM modules;

[0050] Sub-phase 3: Unfreeze the BiLSTM module during training, and freeze the Transformer and CNN modules;

[0051] In each sub-stage, the unfreezing module is used as the generator, and the other two modules are used as discriminators. The discriminator outputs the false detection confidence and back-optimizes the generator's feature extraction.

[0052] The training loss in this stage consists of generator loss and discriminator loss; the optimizer used in this stage is SGD, with a learning rate of 5×10⁻⁶. -5 The batch size is 8, the training rounds for each sub-stage are 20, and the total training rounds are 120.

[0053] (6c) Third stage: Iterative training of the defect identification model, end-to-end optimization of defect identification performance; the training loss in this stage is a multi-task loss function. The optimizer used in this training phase is AdamW, with an initial learning rate of 1×10⁻⁶. -5 The cosine annealing method was used to attenuate the material to 1×10⁻⁶. -7 The batch size is 8, and the training rounds are 100.

[0054] Another object of the present invention is to provide an electronic device comprising:

[0055] Processor; and

[0056] A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the multispectral ultra-high voltage valve hall defect identification method as described above.

[0057] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the multispectral ultra-high voltage valve hall defect identification method as described above.

[0058] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention constructs a high-quality multimodal training dataset through spatiotemporal synchronous registration and adaptive data augmentation strategies, significantly improving the model's generalization ability and achieving high-precision collaborative identification of multiple defects such as cracks, corrosion, and discharge; Second, the present invention innovatively designs a cross-modal feature fusion network, combining channel attention, cross-modal interaction, and temporal attention mechanisms to optimize the semantic alignment and information complementarity of multispectral features, solving the problems of feature conflict and information redundancy in traditional methods; Third, the present invention maintains a multimodal recognition accuracy of more than 90% under extreme scenarios such as strong electromagnetic interference, high temperature, and transient discharge, which is 30% higher than that of single-spectral methods. Based on the YOLO framework, it meets the real-time monitoring requirements of valve halls and has been successfully applied to the online monitoring system of UHV valve halls, providing efficient and reliable technical support for the intelligent operation and maintenance of power equipment, and has significant engineering application value and market prospects. Attached Figure Description

[0059] Figure 1 This is a flowchart of the method of the present invention;

[0060] Figure 2 This is a network structure diagram of the defect identification model in this invention;

[0061] Figure 3 This is a diagram of the cross-modal feature fusion network structure in this invention. Detailed Implementation

[0062] like Figure 1 As shown, a multispectral ultra-high voltage valve hall defect identification method includes the following sequential steps:

[0063] (1) Collect multispectral image data of the device in the UHV valve hall and perform preprocessing. The multispectral image data includes visible light images, infrared images and ultraviolet time series data, and perform data enhancement.

[0064] (2) Construct a multispectral feature extraction network, which includes a CNN module for extracting spatial features of infrared images, a Transformer module for extracting global features of visible light images, and a BiLSTM module for extracting temporal features of ultraviolet light. The spatial features of infrared images, the global features of visible light images, and the temporal features of ultraviolet light constitute multispectral features.

[0065] (3) Design a cross-modal feature fusion network to adaptively fuse multispectral features to obtain fused features; the cross-modal feature fusion network includes a channel attention module, a cross-modal interaction module, a temporal attention module and a splicing module;

[0066] (4) Input the fused features into the improved YOLOv11 network, and the improved YOLOv11 network outputs the defect classification and localization results;

[0067] (5) The defect identification model is composed of a multispectral feature extraction network, a cross-modal feature fusion network, and an improved YOLOv11 network, such as... Figure 2 As shown;

[0068] (6) A three-stage progressive framework is adopted to train the defect recognition model: First stage: single-modality specific pre-training is carried out to independently optimize the feature extraction capabilities of the CNN module, Transformer module, and BiLSTM module; Second stage: the CNN module, Transformer module, and BiLSTM module are alternately unfrozen, and adversarial training is carried out between the modules to correct errors in detection; Third stage: the defect recognition model is iteratively trained in a loop, and the network parameters are jointly optimized through a multi-task loss function to complete the training.

[0069] Step (1) specifically includes the following steps in sequence:

[0070] (1a) Collect 5000 sets of multispectral data samples to form a dataset, label the defect types and location information, and obtain the labeled dataset. The defect types include cracks, corrosion and discharge.

[0071] (1b) Divide the labeled dataset into training set, validation set and test set in a ratio of 6:2:2; the training set contains 3000 sets of multispectral data, the validation set contains 1000 sets of multispectral data and the test set contains 1000 sets of multispectral data.

[0072] (1c) The PWC-Net optical flow algorithm is used to align visible light images, infrared images and ultraviolet temporal data to ensure pixel-level spatiotemporal consistency and achieve spatiotemporal synchronous registration;

[0073] (1d) Gaussian noise was added to the infrared image to simulate thermal interference, Mosaic enhancement was applied to the visible light image, and sliding window truncation was applied to the ultraviolet time series data to achieve data enhancement.

[0074] Step (2) specifically refers to: the CNN module adopts a lightweight GhostNet structure, the input infrared image size is 640×640, the CNN module outputs a 256×256×64 feature map, focusing on local hot spots and temperature gradient features; the Transformer module is based on the Swin-Tiny architecture, the window size is 8×8, and the output multi-scale feature map is used to capture global texture information such as cracks and corrosion in visible light images; the input of the BiLSTM module is ultraviolet time-series data signal, the length of the ultraviolet time-series data signal is 100 frames, each frame contains pulse intensity and frequency, the hidden layer dimension of the BiLSTM module is 128, and the output of the BiLSTM module is a time-series feature vector.

[0075] like Figure 3 As shown, step (3) specifically includes the following steps in sequence:

[0076] (3a) Use the channel attention module to perform channel weighting on the infrared features, as shown in the following formula:

[0077] ;

[0078] In the formula, , All are weights of fully connected layers. For global average pooling; For adaptive learning weights, For the input infrared features, This is the output of the channel attention module;

[0079] (3b) Use the cross-modal interaction module to achieve semantic alignment of visible light and infrared features:

[0080] ;

[0081] in, , , ; For the output of the cross-modal interaction module; Scaling factor ; For visible light projection weights; It has visible light characteristics; For infrared bond projection weights, The projection weights are the infrared values.

[0082] (3c) Use the temporal attention module to filter keyframes for ultraviolet temporal features, using the following formula:

[0083] ;

[0084] in, For BiLSTM hidden state, This is the attention weight matrix; Normalized attention weights for time steps, ; The feature vector is weighted by temporal attention; t is the temporal index; T is the total temporal length;

[0085] (3d) The features are fused by the splicing module. The fused features are compressed by depthwise separable convolution to output a unified dimension feature map. The depthwise separable convolution has 3 kernels and a stride of 1.

[0086] Step (4) specifically includes the following steps in sequence:

[0087] (4a) The SPPF layer of the YOLOv11 network is replaced by the adaptive spatial pyramid pooling ASPP module, which enhances the multi-scale defect perception capability through dilated convolution. The adaptive spatial pyramid pooling ASPP module uses three sets of parallel dilated convolutional layers with dilation rates of 6, 12 and 18, respectively. The adaptive spatial pyramid pooling ASPP module outputs a multi-scale feature map with 256 channels.

[0088] (4b) The detection head adopts a dynamic anchor frame mechanism, which dynamically adjusts the anchor frame size according to the multispectral characteristics. The dynamic anchor frame mechanism refers to the following: the initial anchor frame of the infrared branch is 32×32, 64×64, 128×128, which is suitable for large-scale hot spots; the anchor frame of the visible light branch is 16×16, 32×32, 64×64, which is suitable for small-scale cracks.

[0089] In step (6), the multi-task loss function for:

[0090] ;

[0091] In the formula, To improve CIoU positioning loss, To classify the focus loss, Loss of consistency of multispectral features;

[0092] The The formula is:

[0093] ;

[0094] in, The distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. The length of the diagonal of the minimum bounding rectangle; Indicates the coordinates of the prediction box. Represents the coordinates of the true bounding box. It is a dynamic hyperparameter used to balance the impact of different parts of the loss on the total loss; This indicates the aspect ratio difference between the predicted bounding box and the ground truth bounding box; For intersection, union, and comparison;

[0095] The The formula is:

[0096] ;

[0097] in, ; This represents the combined value of the model's predicted positive sample probability and the true label.

[0098] The The formula is:

[0099] ;

[0100] In the formula, This represents the fused multispectral feature vector. This represents the eigenvector of a single spectrum.

[0101] Step (6) specifically includes the following steps in sequence:

[0102] (6a) First stage: Perform single-modality specific pre-training to independently optimize the feature extraction capabilities of each spectral module; use the training set as the training set for this stage, with the CNN module only inputting infrared images and corresponding defect labels, the Transformer module only inputting visible light images and corresponding defect labels, and the BiLSTM module only inputting ultraviolet time-series signals and corresponding defect labels; the training loss for the CNN and Transformer modules is the focus loss, and the training loss for the BiLSTM module is the mean square error loss; the optimizer for the CNN, Transformer, and BiLSTM modules during training is AdamW, with a learning rate of 1×10. -4 The batch size is 16, and the training epochs are 100. During training, the detection head parameters of the cross-modal feature fusion network and the improved YOLOv11 network are frozen.

[0103] (6b) Second stage: Adversarial training is conducted using an alternating unfreeze strategy, with adversarial error correction between modules to suppress cross-modal conflicts; the training set is used as the training set for this stage, and the alternating unfreeze strategy includes three sub-stages:

[0104] Sub-phase 1: Unfreeze the CNN module during training, and freeze the Transformer and BiLSTM modules;

[0105] Sub-phase 2: Unfreeze the Transformer module during training, and freeze the CNN and BiLSTM modules;

[0106] Sub-phase 3: Unfreeze the BiLSTM module during training, and freeze the Transformer and CNN modules;

[0107] In each sub-stage, the unfreezing module is used as the generator, and the other two modules are used as discriminators. The discriminator outputs the false detection confidence and back-optimizes the generator's feature extraction.

[0108] The training loss in this stage consists of generator loss and discriminator loss; the optimizer used in this stage is SGD, with a learning rate of 5×10⁻⁶. -5 The batch size is 8, the training rounds for each sub-stage are 20, and the total training rounds are 120.

[0109] (6c) Third stage: Iterative training of the defect identification model, end-to-end optimization of defect identification performance; the training set and validation set are used as the dataset for this stage, and the training loss for this stage is the multi-task loss function. The optimizer used in this training phase is AdamW, with an initial learning rate of 1×10⁻⁶. -5 The cosine annealing method was used to attenuate the material to 1×10⁻⁶. -7 The batch size is 8, and the training rounds are 100.

[0110] In summary, to improve the accuracy of multispectral defect identification in UHV valve halls, this invention constructs a high-quality training dataset through spatiotemporal registration and adaptive enhancement strategies. In the feature extraction stage, CNN, Transformer, and BiLSTM modules are designed for different spectral characteristics, and a cross-modal attention mechanism is embedded. Adaptive feature fusion is achieved through channel weighting, semantic alignment, and temporal filtering. In the detection stage, the YOLOv11 network is optimized, and an adaptive spatial pyramid pooling (ASPP) module is introduced to enhance multi-scale perception capabilities. The anchor frame size is dynamically adjusted based on spectral characteristics.

[0111] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for identifying defects in a multispectral ultra-high voltage valve hall, characterized in that: The method includes the following steps in sequence: (1) Collect multispectral image data of the device in the UHV valve hall and perform preprocessing. The multispectral image data includes visible light images, infrared images and ultraviolet time series data; use PWC-Net optical flow algorithm for spatiotemporal synchronization registration. (2) Construct a multispectral feature extraction network, which includes a CNN module for extracting spatial features of infrared images, a Transformer module for extracting global features of visible light images, and a BiLSTM module for extracting temporal features of ultraviolet light. The spatial features of infrared images, the global features of visible light images, and the temporal features of ultraviolet light constitute multispectral features. (3) Design a cross-modal feature fusion network to adaptively fuse multispectral features to obtain fused features; The cross-modal feature fusion network includes a channel attention module, a cross-modal interaction module, a temporal attention module, and a splicing module; (4) Input the fused features into the improved YOLOv11 network, and the improved YOLOv11 network outputs the defect classification and localization results; (5) The defect identification model is composed of a multispectral feature extraction network, a cross-modal feature fusion network, and an improved YOLOv11 network; (6) A three-stage progressive framework is adopted to train the defect recognition model: First stage: single-modality specific pre-training is carried out to independently optimize the feature extraction capabilities of the CNN module, Transformer module, and BiLSTM module; Second stage: the CNN module, Transformer module, and BiLSTM module are alternately unfrozen, and adversarial training is carried out between the modules to correct errors in detection; Third stage: the defect recognition model is iteratively trained in a loop, and the network parameters are jointly optimized through a multi-task loss function to complete the training.

2. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: Step (1) specifically includes the following steps in sequence: (1a) Collect 5000 sets of multispectral data samples to form a dataset, label the defect types and location information, and obtain the labeled dataset. The defect types include cracks, corrosion and discharge. (1b) Divide the labeled dataset into training set, validation set and test set in a ratio of 6:2:2; (1c) The PWC-Net optical flow algorithm is used to align visible light images, infrared images and ultraviolet temporal data to ensure pixel-level spatiotemporal consistency and achieve spatiotemporal synchronous registration; (1d) Gaussian noise was added to the infrared image to simulate thermal interference, Mosaic enhancement was applied to the visible light image, and sliding window truncation was applied to the ultraviolet time series data to achieve data enhancement.

3. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: Step (2) specifically refers to: the CNN module adopts a lightweight GhostNet structure, the input infrared image size is 640×640, the CNN module outputs a 256×256×64 feature map, focusing on local hot spots and temperature gradient features; the Transformer module is based on the Swin-Tiny architecture, the window size is 8×8, and the output multi-scale feature map is used to capture global texture information such as cracks and corrosion in visible light images; the input of the BiLSTM module is ultraviolet time-series data signal, the length of the ultraviolet time-series data signal is 100 frames, each frame contains pulse intensity and frequency, the hidden layer dimension of the BiLSTM module is 128, and the output of the BiLSTM module is a time-series feature vector.

4. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: Step (3) specifically includes the following steps in sequence: (3a) Use the channel attention module to perform channel weighting on the infrared features, as shown in the following formula: ； In the formula, , All are weights of fully connected layers. For global average pooling; For adaptive learning weights, For the input infrared features, This is the output of the channel attention module; (3b) Use the cross-modal interaction module to achieve semantic alignment of visible light and infrared features: ； in, , , ; For the output of the cross-modal interaction module; Scaling factor ; For visible light projection weights; It has visible light characteristics; For infrared bond projection weights, The projection weights are the infrared values. (3c) Use the temporal attention module to filter keyframes for ultraviolet temporal features, using the following formula: ； in, For BiLSTM hidden state, This is the attention weight matrix; Normalized attention weights for time steps, , ; The feature vector is weighted by temporal attention; t is the temporal index; T1 is the total temporal length; (3d) The features are fused by the splicing module. The fused features are compressed by depthwise separable convolution to output a unified dimension feature map. The depthwise separable convolution has 3 kernels and a stride of 1.

5. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: Step (4) specifically includes the following steps in sequence: (4a) The SPPF layer of the YOLOv11 network is replaced by an adaptive spatial pyramid pooling ASPP module, which enhances the multi-scale defect perception capability through dilated convolution. The adaptive spatial pyramid pooling ASPP module uses three sets of parallel dilated convolutional layers with dilation rates of 6, 12 and 18, respectively. The adaptive spatial pyramid pooling ASPP module outputs a multi-scale feature map with 256 channels. (4b) The detection head adopts a dynamic anchor frame mechanism, which dynamically adjusts the anchor frame size according to the multispectral characteristics. The dynamic anchor frame mechanism refers to the following: the initial anchor frame of the infrared branch is 32×32, 64×64, 128×128, which is suitable for large-scale hot spots; the anchor frame of the visible light branch is 16×16, 32×32, 64×64, which is suitable for small-scale cracks.

6. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: In step (6), the multi-task loss function for: ； In the formula, To improve CIoU positioning loss, To classify the focus loss, Loss of consistency of multispectral features; The The formula is: ； in, The distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. The length of the diagonal of the minimum bounding rectangle; Indicates the coordinates of the prediction box. Represents the coordinates of the true bounding box. It is a dynamic hyperparameter used to balance the impact of different parts of the loss on the total loss; This indicates the aspect ratio difference between the predicted bounding box and the ground truth bounding box; For intersection, union, and comparison; The The formula is: ； in, ; This represents the combined value of the model's predicted positive sample probability and the true label. The The formula is: ； In the formula, This represents the fused multispectral feature vector. This represents the eigenvector of a single spectrum.

7. The multispectral ultra-high voltage valve hall defect identification method according to claim 1, characterized in that: Step (6) specifically includes the following steps in sequence: (6a) First stage: Perform single-modality specific pre-training to independently optimize the feature extraction capabilities of each spectral module; the CNN module only inputs infrared images and corresponding defect labels, the Transformer module only inputs visible light images and corresponding defect labels, and the BiLSTM module only inputs ultraviolet time-series signals and corresponding defect labels; the training loss of the CNN and Transformer modules is the focus loss, and the training loss of the BiLSTM module is the mean square error loss; the optimizer used in the training process for the CNN, Transformer, and BiLSTM modules is AdamW, with a learning rate of 1×10. -4 The batch size is 16, and the training epochs are 100. During training, the detection head parameters of the cross-modal feature fusion network and the improved YOLOv11 network are frozen. (6b) Second stage: Adversarial training is conducted using an alternating unfreeze strategy, with adversarial error correction between modules to suppress cross-modal conflicts; the alternating unfreeze strategy includes three sub-stages: Sub-phase 1: Unfreeze the CNN module during training, and freeze the Transformer and BiLSTM modules; Sub-phase 2: Unfreeze the Transformer module during training, and freeze the CNN and BiLSTM modules; Sub-phase 3: Unfreeze the BiLSTM module during training, and freeze the Transformer and CNN modules; In each sub-stage, the unfreezing module is used as the generator, and the other two modules are used as discriminators. The discriminator outputs the false detection confidence and back-optimizes the generator's feature extraction. The training loss in this stage consists of generator loss and discriminator loss; the optimizer used in this stage is SGD, with a learning rate of 5×10⁻⁶. -5 The batch size is 8, the training rounds for each sub-stage are 20, and the total training rounds are 120. (6c) Third stage: Iterative training of the defect identification model, end-to-end optimization of defect identification performance; the training loss in this stage is a multi-task loss function. The optimizer used in this training phase is AdamW, with an initial learning rate of 1×10⁻⁶. -5 The cosine annealing method was used to attenuate the material to 1×10⁻⁶. -7 The batch size is 8, and the training rounds are 100.

8. An electronic device, comprising: processor; as well as A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the multispectral ultra-high voltage valve hall defect identification method as described in any one of claims 1-7.

9. A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the multispectral ultra-high voltage valve hall defect identification method as described in any one of claims 1-7.

Images