An ultra-high voltage valve hall early warning diagnosis method and device based on multi-spectral image feature fusion technology

Abstract

The present application relates to a kind of based on multispectral image feature fusion technology's ultrahigh voltage valve hall early warning diagnosis method, comprising: constitute multispectral fusion dataset;Calculate multidimensional fault feature vector, comprehensive fault probability and the three-dimensional coordinates of fault point;Realize the early warning of equipment potential fault, when fault occurs, automatically trigger early warning;In digital twin platform, the three-dimensional visualization interface of fault point is dynamically rendered, and closed-loop operation and maintenance work order is generated and disposal scheme is executed.The present application captures partial discharge radiation by ultraviolet light, identifies temperature rise anomaly by infrared imaging, detects surface texture defects by visible light image, three-spectrum complementary fusion, effectively covers insulation deterioration, poor contact, mechanical crack, pollution accumulation and other various early fault forms, significantly improves detection sensitivity and adaptability;Intelligent diagnosis efficiency is realized leap, improved ResNet-50 network is used in combination with bidirectional attention mechanism, feature fusion efficiency is significantly optimized, and feature extraction and decision time are shortened.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology for power equipment, and in particular to an ultra-high voltage valve hall early warning diagnosis method and equipment based on multispectral image feature fusion technology. Background Technology

[0002] With the rapid development of ultra-high voltage direct current (UHVDC) transmission technology, UHV converter valves, as core equipment of the power system, directly affect the safety of the power grid due to their operational stability. However, under long-term high voltage, strong electromagnetic fields, and complex thermal stress, key components of valve hall equipment are prone to insulation degradation, partial discharge, and other faults. In recent years, several accidents in converter station valve halls caused by equipment insulation degradation have occurred, resulting in significant economic losses.

[0003] Currently, ultra-high voltage valve halls mainly rely on visible light imaging and infrared thermal imaging technology for equipment monitoring, which still has the following shortcomings:

[0004] (1) Data isolation: It relies on single-spectral imaging and cannot simultaneously capture discharge, temperature rise and surface defects;

[0005] (2) Diagnostic lag: The manual inspection cycle is long, and the fault is discovered later than the actual time of occurrence, which leads to the risk of unplanned downtime;

[0006] (3) Coverage limitations: There is a lack of real-time monitoring capabilities for key weak links such as the internal sleeve connection of the valve hall;

[0007] (4) Fragmented analysis: Multi-source data are not effectively integrated, fault location relies on experience judgment, and lacks support for three-dimensional spatial modeling.

[0008] While existing technologies can compensate for the shortcomings of partial discharge monitoring, ultraviolet detection has not yet been deeply integrated with infrared, visible light, and multispectral data, and lacks dynamic viewing angle adjustment and digital twin collaborative analysis capabilities. Therefore, there is an urgent need for an early warning and diagnostic method that integrates multispectral sensing, intelligent algorithms, and closed-loop operation and maintenance. Summary of the Invention

[0009] To address the issues of isolated multispectral data, delayed diagnosis, and lack of closed-loop operation and maintenance, the primary objective of this invention is to provide an early warning and diagnostic method for ultra-high voltage valve halls based on multispectral image feature fusion technology. This method enables full-spectrum collaborative detection, overcomes the limitations of single spectra in information coverage and interference suppression, achieves complementary fusion of three spectra, significantly improves detection sensitivity and adaptability, and ensures that maintenance personnel can take timely early warning measures in the early stages of a fault.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: a method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology, the method comprising the following sequential steps:

[0011] (1) Simultaneously acquire ultraviolet light images, infrared light images, and visible light images of the valve hall equipment, and form a multispectral fusion dataset;

[0012] (2) Preprocess, spatiotemporally align and feature-fuse the collected multispectral fusion data, and calculate multidimensional fault feature vectors. Overall failure probability and the three-dimensional coordinates P of the fault point;

[0013] (3) Based on multi-dimensional fault feature vectors Overall failure probability The three-dimensional coordinates P of the fault point enable early warning of potential equipment faults, and the warning is automatically triggered when a fault occurs.

[0014] (4) Dynamically render the three-dimensional visualization interface of the fault point in the digital twin platform, generate a closed-loop operation and maintenance work order and execute the disposal plan.

[0015] Step (1) specifically refers to: acquiring ultraviolet light images through an ultraviolet sensor unit to detect the discharge intensity or abnormality on the surface of the equipment, with a working wavelength of 240-280nm and a photon counting sensitivity of ≥1000 photons / second; acquiring infrared light images through an infrared thermal imager unit to detect temperature changes in the equipment, with a temperature measurement range of -40℃ to 550℃; acquiring visible light images through a 4K visible light camera; and the ultraviolet light images, infrared light images, and visible light images constitute a multispectral fusion dataset.

[0016] Step (2) specifically includes the following steps in sequence:

[0017] (2a) Preprocessing of ultraviolet, infrared and visible light images: removing environmental noise from ultraviolet images and extracting effective discharge photon signals by photon counting method; performing non-uniformity correction on infrared images to improve temperature measurement accuracy and eliminate equipment non-uniformity errors; performing image enhancement processing on visible light images;

[0018] (2b) Using the preprocessed visible light image as a reference, the preprocessed ultraviolet and infrared images are precisely registered to obtain the registered image, so as to unify the multispectral image spatial coordinate system;

[0019] (2c) Design an improved ResNet-50 network to extract multispectral specific fault features from the registered images and generate a 128-dimensional fault feature vector; the improved ResNet-50 network includes three parallel branches, a cross-spectral feature interaction layer, and a fully connected dimensionality reduction layer; the three parallel branches are the ultraviolet image branch, the infrared image branch, and the visible light image branch, respectively; the three parallel branches extract discharge intensity feature vectors for the aligned ultraviolet image, infrared image, and visible light image, respectively. Temperature gradient eigenvector and texture defect feature vectors The cross-spectral feature interaction layer is positioned between the feature extraction layer and the fully connected dimensionality reduction layer in three parallel branches, employing a bidirectional attention mechanism.

[0020] (2d) Dynamically allocate discharge intensity feature vectors through a bidirectional attention mechanism across spectral feature interaction layers. Temperature gradient eigenvector and texture defect feature vectors Fusion weights:

[0021] by For Query, with As Key, with For Value, projected through a learnable linear mapping matrix, it is:

[0022] ;

[0023] ;

[0024] ;

[0025] in, Both are weight matrices. The number of input feature channels, For the projection feature dimension, for The query vector, for The key vector, for The value vector;

[0026] Calculate attention output:

[0027] ;

[0028] in, For normalized probability distribution, Indicates To enhance cross-spectral correlation features;

[0029] Similarly, with For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features :

[0030] ;

[0031] In the formula, for The query vector, for The key vector, for The value vector;

[0032] by For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features :

[0033] ;

[0034] In the formula, for The query vector, for The key vector, for The value vector;

[0035] Will , , The spectral data are cascaded into a 768-dimensional vector, and after layer normalization, the fused features are obtained. :

[0036] ;

[0037] In the formula, For splicing, For layer normalization

[0038] (2e) Fusion features after normalization Dimensionality reduction is performed using fully connected layers with nonlinear activation functions. By performing projection and transformation, a multi-dimensional fault feature vector is obtained. :

[0039] ;

[0040] in, This is the weight matrix of the fully connected layer; For bias terms; It has 128 dimensions;

[0041] (2f) Based on the discharge intensity eigenvector Temperature gradient eigenvector and texture defect feature vectors Calculate the overall failure probability :

[0042] ;

[0043] in, , , All of these are trainable fusion weight coefficients. This represents the maximum discharge intensity output from the ultraviolet image branch. The L2 norm of the infrared temperature rise gradient. The information entropy of visible light texture features;

[0044] (2g) Fault location and spatial source tracing under multi-point multispectral fusion: For synchronous multispectral image input from three or more monitoring points, based on the gimbal pose parameters and camera intrinsic parameter matrix corresponding to each monitoring point. The gimbal pose parameters include horizontal and pitch angles, and the three-dimensional coordinates of the fault point are calculated. :

[0045]

[0046] in, For at least three different observation perspectives, For the first The pixel coordinates of the image were collected from each monitoring point. The intrinsic parameter matrix corresponding to the camera, For multi-view triangulation algorithms, These are the spatial coordinates of the fault point.

[0047] Step (3) specifically refers to: based on the multi-dimensional fault feature vector The data is divided into four feature segments: discharge intensity, temperature rise gradient, texture defect, and correlation. The dimensions of these segments are 0–31, 32–63, 64–95, and 96–127, respectively. The discharge intensity segment characterizes the discharge photon density in the ultraviolet image; when the maximum value exceeds 500 photons / second in three consecutive frames, it is considered an insulation degradation fault. The temperature rise gradient segment measures the local temperature rise in the infrared image. The gradient feature is used to determine a fault, i.e., an abnormal temperature, when the L2 norm exceeds 10℃ / cm. The texture defect feature segment is based on the statistical characteristics of LBP values; when the mean variance exceeds 0.25, a fault is identified. If the spatial variance distribution is concentrated, it is identified as surface contamination; if the distribution is discrete, it is identified as mechanical cracks. The correlation feature segment is used to measure the coupling strength between different modes; when the Pearson correlation coefficient is greater than 0.8, a fault, i.e., arc discharge, is identified; when the Pearson correlation coefficient is less than 0.5, a fault, i.e., local overheating, is identified.

[0048] Overall failure probability This is used to quantify the severity of fault conditions and to construct a tiered response mechanism based on the fault status of each feature segment: when any one of the four feature segments fails, the location of the fault point is dynamically rendered in the digital twin platform and a yellow warning is issued, while an alarm message is pushed; when any two of the four feature segments fail or When the value is ≥0.8, an audible and visual alarm is triggered, a maintenance work order is automatically generated, and historical cases are linked; when When the value is ≥0.95, the system will shut down and isolate the fault, link the fire control system, and push the location of the fault point and the emergency plan to the emergency terminal simultaneously.

[0049] Step (4) specifically refers to:

[0050] A digital twin platform is built based on the Three.js engine, which integrates multispectral monitoring data in real time and dynamically renders temperature field distribution, discharge intensity and defect location;

[0051] Based on the fault type, retrieve similar historical cases with a matching degree of over 90% from the knowledge graph, automatically output a structured handling plan, and notify the operation and maintenance personnel in real time.

[0052] After maintenance is completed, the system is re-inspected. If the discharge intensity drops to the safe threshold and the temperature returns to normal, the equipment health score is updated and the alarm threshold is automatically optimized, completing the closed-loop management of the entire process from perception to analysis, and then to decision-making and execution.

[0053] Step (2b) specifically includes the following steps in sequence:

[0054] (2b1) The SIFT algorithm is used to extract feature points and descriptors from the preprocessed visible light image, ultraviolet image, and infrared image, respectively;

[0055] (2b2) ​​Feature matching is performed using Euclidean distance;

[0056] (2b3) Use the RANSAC algorithm to estimate the transformation matrix and remove mismatched points, and calculate the optimal homography matrix;

[0057] (2b4) Resample the ultraviolet and infrared images according to the optimal homography matrix to make the ultraviolet and infrared images spatially aligned with the visible light image, and unify all images into the spatial coordinate system of the visible light image.

[0058] Step (2c) specifically includes the following steps in sequence:

[0059] (2c1) The ultraviolet image branch uses convolutional layers and LSTM layers to extract the discharge intensity feature vector from the aligned ultraviolet image. :

[0060] To extract the dynamic features of discharge intensity from ultraviolet images, the input ultraviolet image is first processed by a 3×3 convolutional kernel. Feature extraction is performed, and the feature map is obtained by applying the ReLU activation function. :

[0061] ;

[0062] in, The convolution kernel weight matrix is... These are the bias parameters for the convolutional layer. This represents a spatial feature map of 64 channels. and These are the height and width of the image, i.e., the spatial dimensions;

[0063] right Perform spatial pooling to obtain the pooled feature sequence. ;

[0064] Next, the pooled feature sequences The input is fed into the LSTM layer, combined with the hidden state from the previous time step. and memory unit Perform time series modeling to obtain the state at the current moment:

[0065] ;

[0066] in, Let this be the hidden state vector at the current time step. The state of the memory cell at the current time step;

[0067] The temporal state information of the features in consecutive frames of ultraviolet images is modeled using LSTM layers, outputting a 64-dimensional discharge intensity feature vector. ;

[0068] (2c2) The infrared image branch uses residual blocks and the HOG algorithm to extract the temperature rise gradient feature vector from the aligned infrared image. :

[0069] By extracting features from the input infrared image using residual blocks, the thermal distribution structure is enhanced to obtain an intermediate feature map. :

[0070] ;

[0071] in, To input an infrared image, It has a 3×3 convolution kernel; For activation functions;

[0072] Gradient calculations are performed on infrared images to obtain the spatial derivative of temperature changes. and direction angle :

[0073] ;

[0074] In the formula, This is the temperature value;

[0075] Intermediate feature map extracted from residual block and By fusing the HOG algorithm, the local temperature rise direction and amplitude information are encoded, and a 64-dimensional temperature rise gradient feature vector is output. ;

[0076] (2c3) The visible light image branch uses multi-scale LBP and residual blocks to extract texture defect feature vectors from the aligned visible light image. :

[0077] By encoding the gray-level relationship between each pixel and its neighborhood through multi-scale LBP, fine-grained texture features are extracted to obtain the LBP value at the pixel location:

[0078] ;

[0079] in, This is the grayscale value of the current pixel. For the first The grayscale value of each neighbor, This indicates which neighboring node is currently in the list. It is a step function. The grayscale difference between the current neighboring pixel and the center pixel. The position of the current pixel;

[0080] The LBP values ​​of all pixels in the entire image are calculated to form an LBP feature map; then, the LBP feature map is fused with the deep texture features extracted from the residual blocks to output a 64-dimensional texture defect feature vector. .

[0081] Another object of the present invention is to provide an electronic device comprising:

[0082] Processor; and

[0083] The memory stores computer program instructions that, when executed by the processor, cause the processor to perform the ultra-high pressure valve hall early warning and diagnosis method based on multispectral image feature fusion technology as described above.

[0084] The present invention also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the ultra-high pressure valve hall early warning and diagnosis method based on multispectral image feature fusion technology as described above.

[0085] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention achieves full-spectrum collaborative detection, breaking through the limitations of single spectrum in terms of information coverage and interference suppression. By capturing partial discharge radiation with ultraviolet light, identifying abnormal temperature rise with infrared imaging, and detecting surface texture defects with visible light images, the three spectra are complementary and fused, effectively covering various early fault modes such as insulation degradation, poor contact, mechanical cracks, and dirt accumulation, significantly improving detection sensitivity and adaptability. Second, it achieves a leap in intelligent diagnostic efficiency. By adopting an improved ResNet-50 network combined with a bidirectional attention mechanism, it significantly optimizes feature fusion efficiency, shortens feature extraction and decision-making time, achieves fault location accuracy of ±3cm, and improves classification accuracy to over 95%. At the same time, it supports multi-target parallel detection and real-time location, ensuring that maintenance personnel can take timely early warning measures in the early stages of faults. Third, it achieves a breakthrough in key industry technologies. By adaptively adjusting spectral weights, it can cope with complex environments such as different lighting, weather, and background interference, realizing the transformation from "periodic maintenance" to "condition prediction" maintenance mode. Attached Figure Description

[0086] Figure 1 This is a flowchart of the method of the present invention;

[0087] Figure 2 This is a flowchart of the method for calculating the comprehensive failure probability in this invention;

[0088] Figure 3 This is a schematic diagram of the structure of the improved ResNet-50 network in this invention. Detailed Implementation

[0089] like Figure 1 As shown, a method for early warning diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology is presented. The method includes the following sequential steps:

[0090] (1) Simultaneously acquire ultraviolet light images, infrared light images, and visible light images of the valve hall equipment, and form a multispectral fusion dataset;

[0091] (2) Preprocess, spatiotemporally align and feature-fuse the collected multispectral fusion data, and calculate multidimensional fault feature vectors. Overall failure probability and the three-dimensional coordinates P of the fault point;

[0092] (3) Based on multi-dimensional fault feature vectors Overall failure probability The three-dimensional coordinates P of the fault point enable early warning of potential equipment faults, and the warning is automatically triggered when a fault occurs.

[0093] (4) Dynamically render the three-dimensional visualization interface of the fault point in the digital twin platform, generate a closed-loop operation and maintenance work order and execute the disposal plan.

[0094] Step (1) specifically refers to: acquiring ultraviolet light images through an ultraviolet sensor unit to detect the discharge intensity or abnormality on the surface of the equipment, with a working wavelength of 240-280nm and a photon counting sensitivity of ≥1000 photons / second; acquiring infrared light images through an infrared thermal imager unit to detect temperature changes in the equipment, with a temperature measurement range of -40℃ to 550℃; acquiring visible light images through a 4K visible light camera; and the ultraviolet light images, infrared light images, and visible light images constitute a multispectral fusion dataset.

[0095] like Figure 2 As shown, step (2) specifically includes the following steps in sequence:

[0096] (2a) Preprocessing of ultraviolet, infrared and visible light images: removing environmental noise from ultraviolet images and extracting effective discharge photon signals by photon counting method; performing non-uniformity correction on infrared images to improve temperature measurement accuracy and eliminate equipment non-uniformity errors; performing image enhancement processing on visible light images;

[0097] (2b) Using the preprocessed visible light image as a reference, the preprocessed ultraviolet and infrared images are precisely registered to obtain the registered image, so as to unify the multispectral image spatial coordinate system;

[0098] (2c) Design an improved ResNet-50 network, such as Figure 3 As shown, multispectral specific fault features are extracted from the registered images to generate a 128-dimensional fault feature vector. The improved ResNet-50 network includes three parallel branches, a cross-spectral feature interaction layer, and a fully connected dimensionality reduction layer. The three parallel branches are the ultraviolet image branch, the infrared image branch, and the visible light image branch, respectively. The three parallel branches extract discharge intensity feature vectors for the aligned ultraviolet, infrared, and visible light images, respectively. Temperature gradient eigenvector and texture defect feature vectors The cross-spectral feature interaction layer is positioned between the feature extraction layer and the fully connected dimensionality reduction layer in three parallel branches, employing a bidirectional attention mechanism.

[0099] (2d) Dynamically allocate discharge intensity feature vectors through a bidirectional attention mechanism across spectral feature interaction layers. Temperature gradient eigenvector and texture defect feature vectors Fusion weights:

[0100] by For Query, with As Key, with For Value, projected through a learnable linear mapping matrix, it is:

[0101] ;

[0102] ;

[0103] ;

[0104] in, Both are weight matrices. The number of input feature channels, For the projection feature dimension, for The query vector, for The key vector, for The value vector;

[0105] Calculate attention output:

[0106] ;

[0107] in, For normalized probability distribution, Indicates To enhance cross-spectral correlation features;

[0108] Similarly, with For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features :

[0109] ;

[0110] In the formula, for The query vector, for The key vector, for The value vector;

[0111] by For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features :

[0112] ;

[0113] In the formula, for The query vector, for The key vector, for The value vector;

[0114] Will , , The spectral data are cascaded into a 768-dimensional vector according to spectral order, and then subjected to layer normalization to obtain the fused features. :

[0115] ;

[0116] In the formula, For splicing, For layer normalization

[0117] (2e) Fusion features after normalization Dimensionality reduction is performed using fully connected layers with non-linear activation functions. By performing projection and transformation, a multi-dimensional fault feature vector is obtained. :

[0118] ;

[0119] in, This is the weight matrix of the fully connected layer; For bias terms; It has 128 dimensions;

[0120] (2f) Based on the discharge intensity eigenvector Temperature gradient eigenvector and texture defect feature vectors Calculate the overall failure probability :

[0121] ;

[0122] in, , , All of these are trainable fusion weight coefficients. This represents the maximum discharge intensity output from the ultraviolet image branch. The L2 norm of the infrared temperature rise gradient. The information entropy of visible light texture features;

[0123] (2g) Fault location and spatial source tracing under multi-point multispectral fusion: For synchronous multispectral image input from three or more monitoring points, based on the gimbal pose parameters and camera intrinsic parameter matrix corresponding to each monitoring point. The gimbal pose parameters include horizontal and pitch angles, and the three-dimensional coordinates of the fault point are calculated. :

[0124]

[0125] in, For at least three different observation perspectives, For the first The pixel coordinates of the image were collected from each monitoring point. The intrinsic parameter matrix corresponding to the camera, For multi-view triangulation algorithms, These are the spatial coordinates of the fault point.

[0126] Step (3) specifically refers to: based on the multi-dimensional fault feature vector The data is divided into four feature segments: discharge intensity, temperature rise gradient, texture defect, and correlation. The dimensions of these segments are 0–31, 32–63, 64–95, and 96–127, respectively. The discharge intensity segment characterizes the discharge photon density in the ultraviolet image; when the maximum value exceeds 500 photons / second in three consecutive frames, it is considered an insulation degradation fault. The temperature rise gradient segment measures the local temperature rise in the infrared image. The gradient feature is used to determine a fault, i.e., an abnormal temperature, when the L2 norm exceeds 10℃ / cm. The texture defect feature segment is based on the statistical characteristics of LBP values; when the mean variance exceeds 0.25, a fault is identified. If the spatial variance distribution is concentrated, it is identified as surface contamination; if the distribution is discrete, it is identified as mechanical cracks. The correlation feature segment is used to measure the coupling strength between different modes; when the Pearson correlation coefficient is greater than 0.8, a fault, i.e., arc discharge, is identified; when the Pearson correlation coefficient is less than 0.5, a fault, i.e., local overheating, is identified.

[0127] Overall failure probability This is used to quantify the severity of fault conditions and to construct a tiered response mechanism based on the fault status of each feature segment: when any one of the four feature segments fails, the location of the fault point is dynamically rendered in the digital twin platform and a yellow warning is issued, while an alarm message is pushed; when any two of the four feature segments fail or When the value is ≥0.8, an audible and visual alarm is triggered, a maintenance work order is automatically generated, and historical cases are linked; when When the value is ≥0.95, the system will shut down and isolate the fault, link the fire control system, and push the location of the fault point and the emergency plan to the emergency terminal simultaneously.

[0128] Step (4) specifically refers to:

[0129] A digital twin platform is built based on the Three.js engine, which integrates multispectral monitoring data in real time and dynamically renders temperature field distribution, discharge intensity and defect location;

[0130] Based on the fault type, retrieve similar historical cases with a matching degree of over 90% from the knowledge graph, automatically output a structured handling plan, and notify the operation and maintenance personnel in real time.

[0131] After maintenance is completed, the system is re-inspected. If the discharge intensity drops to the safe threshold and the temperature returns to normal, the equipment health score is updated and the alarm threshold is automatically optimized, completing the closed-loop management of the entire process from perception to analysis, and then to decision-making and execution.

[0132] Step (2b) specifically includes the following steps in sequence:

[0133] (2b1) The SIFT algorithm is used to extract feature points and descriptors from the preprocessed visible light image, ultraviolet image, and infrared image, respectively;

[0134] (2b2) ​​Feature matching is performed using Euclidean distance;

[0135] (2b3) Use the RANSAC algorithm to estimate the transformation matrix and remove mismatched points, and calculate the optimal homography matrix;

[0136] (2b4) Resample the ultraviolet and infrared images according to the optimal homography matrix to make the ultraviolet and infrared images spatially aligned with the visible light image, and unify all images into the spatial coordinate system of the visible light image.

[0137] Step (2c) specifically includes the following steps in sequence:

[0138] (2c1) The ultraviolet image branch uses convolutional layers and LSTM layers to extract the discharge intensity feature vector from the aligned ultraviolet image. :

[0139] To extract the dynamic features of discharge intensity from ultraviolet images, the input ultraviolet image is first processed by a 3×3 convolutional kernel. Feature extraction is performed, and the feature map is obtained by applying the ReLU activation function. :

[0140] ;

[0141] in, The convolution kernel weight matrix is... These are the bias parameters for the convolutional layer. This represents a spatial feature map of 64 channels. and These are the height and width of the image, i.e., the spatial dimensions;

[0142] right Perform spatial pooling to obtain the pooled feature sequence. ;

[0143] Next, the pooled feature sequences The input is fed into the LSTM layer, combined with the hidden state from the previous time step. and memory unit Perform time series modeling to obtain the state at the current moment:

[0144] ;

[0145] in, Let this be the hidden state vector at the current time step. The state of the memory cell at the current time step;

[0146] The temporal state information of the features in consecutive frames of ultraviolet images is modeled using LSTM layers, outputting a 64-dimensional discharge intensity feature vector. ;

[0147] (2c2) The infrared image branch uses residual blocks and the HOG algorithm to extract the temperature rise gradient feature vector from the aligned infrared image. :

[0148] By extracting features from the input infrared image using residual blocks, the thermal distribution structure is enhanced to obtain an intermediate feature map. :

[0149] ;

[0150] in, To input an infrared image, It has a 3×3 convolution kernel; For activation functions;

[0151] Gradient calculations are performed on infrared images to obtain the spatial derivative of temperature changes. and direction angle :

[0152] ;

[0153] In the formula, This is the temperature value;

[0154] Intermediate feature map extracted from residual block and By fusing the HOG algorithm, the local temperature rise direction and amplitude information are encoded, and a 64-dimensional temperature rise gradient feature vector is output. ;

[0155] (2c3) The visible light image branch uses multi-scale LBP and residual blocks to extract texture defect feature vectors from the aligned visible light image. :

[0156] By encoding the gray-level relationship between each pixel and its neighborhood through multi-scale LBP, fine-grained texture features are extracted to obtain the LBP value at the pixel location:

[0157] ;

[0158] in, This is the grayscale value of the current pixel. For the first The grayscale value of each neighbor, This indicates which neighboring node is currently in the list. It is a step function. The grayscale difference between the current neighboring pixel and the center pixel. The position of the current pixel;

[0159] The LBP values ​​of all pixels in the entire image are calculated to form an LBP feature map; then, the LBP feature map is fused with the deep texture features extracted from the residual blocks to output a 64-dimensional texture defect feature vector. .

[0160] In summary, this invention achieves full-spectrum collaborative detection, overcoming the limitations of single-spectrum detection in terms of information coverage and interference suppression. It captures partial discharge radiation with ultraviolet light, identifies abnormal temperature rise through infrared imaging, and detects surface texture defects using visible light images. The complementary fusion of these three spectra effectively covers various early fault modes such as insulation degradation, poor contact, mechanical cracks, and dirt accumulation, significantly improving detection sensitivity and adaptability. It also achieves a leap in intelligent diagnostic efficiency, employing an improved ResNet-50 network combined with a bidirectional attention mechanism to significantly optimize feature fusion efficiency, shorten feature extraction and decision-making time, achieve fault location accuracy of ±3cm, and improve classification accuracy to over 95%. Simultaneously, it supports multi-target parallel detection and real-time location, ensuring that maintenance personnel can take timely early warning measures in the early stages of faults. Furthermore, it represents a breakthrough in key industry technologies, adaptively adjusting spectral weights to cope with complex environments such as different lighting conditions, weather, and background interference, enabling a transformation from "periodic maintenance" to "condition prediction" in the maintenance model.

[0161] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology, characterized in that: The method includes the following steps in sequence: (1) Simultaneously acquire ultraviolet light images, infrared light images, and visible light images of the valve hall equipment, and form a multispectral fusion dataset; (2) Preprocess, spatiotemporally align and feature-fuse the collected multispectral fusion data, and calculate multidimensional fault feature vectors. Overall failure probability and the three-dimensional coordinates P of the fault point; (3) Based on multi-dimensional fault feature vectors Overall failure probability The three-dimensional coordinates P of the fault point enable early warning of potential equipment faults, and the warning is automatically triggered when a fault occurs. (4) Dynamically render the three-dimensional visualization interface of the fault point in the digital twin platform, generate a closed-loop operation and maintenance work order and execute the disposal plan; Step (2) specifically includes the following steps in sequence: (2a) Preprocessing of ultraviolet, infrared and visible light images: removing environmental noise from ultraviolet images and extracting effective discharge photon signals by photon counting method; performing non-uniformity correction on infrared images to improve temperature measurement accuracy and eliminate equipment non-uniformity errors; performing image enhancement processing on visible light images; (2b) Using the preprocessed visible light image as a reference, the preprocessed ultraviolet and infrared images are precisely registered to obtain the registered image, so as to unify the multispectral image spatial coordinate system; (2c) Design an improved ResNet-50 network to extract multispectral specific fault features from the registered image and generate a 128-dimensional fault feature vector; the improved ResNet-50 network includes three parallel branches, a cross-spectral feature interaction layer and a fully connected dimensionality reduction layer; the three parallel branches are the ultraviolet image branch, the infrared image branch and the visible light image branch, respectively. Three parallel branches extract discharge intensity feature vectors from the aligned ultraviolet, infrared, and visible light images, respectively. Temperature gradient eigenvector and texture defect feature vectors The cross-spectral feature interaction layer is positioned between the feature extraction layer and the fully connected dimensionality reduction layer in three parallel branches, employing a bidirectional attention mechanism. (2d) Dynamically allocate discharge intensity feature vectors through a bidirectional attention mechanism across spectral feature interaction layers. Temperature gradient eigenvector and texture defect feature vectors Fusion weights: by For Query, with As Key, with For Value, projected through a learnable linear mapping matrix, it is: ； ； ； in, Both are weight matrices. The number of input feature channels, For the projection feature dimension, for The query vector, for The key vector, for The value vector; Calculate attention output: ； in, For normalized probability distribution, Indicates To enhance cross-spectral correlation features; Similarly, with For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features : ； In the formula, for The query vector, for The key vector, for The value vector; by For Query, with As Key, with For Value, get To guide the enhancement of cross-spectral correlation features : ； In the formula, for The query vector, for The key vector, for The value vector; Will , , The spectral data are cascaded into a 768-dimensional vector according to spectral order, and then subjected to layer normalization to obtain the fused features. : ； In the formula, For splicing, For layer normalization; (2e) Fusion features after normalization Dimensionality reduction is performed using fully connected layers with non-linear activation functions. By performing projection and transformation, a multi-dimensional fault feature vector is obtained. : ； in, This is the weight matrix of the fully connected layer; For bias terms; It has 128 dimensions; (2f) Based on the discharge intensity eigenvector Temperature gradient eigenvector and texture defect feature vectors Calculate the overall failure probability : ； in, , , All of these are trainable fusion weight coefficients. This represents the maximum discharge intensity output from the ultraviolet image branch. The L2 norm of the infrared temperature rise gradient. The information entropy of visible light texture features; (2g) Fault location and spatial source tracing under multi-point multispectral fusion: For synchronous multispectral image input from three or more monitoring points, based on the gimbal pose parameters and camera intrinsic parameter matrix corresponding to each monitoring point. The gimbal pose parameters include horizontal and pitch angles, and the three-dimensional coordinates of the fault point are calculated. : ； in, For at least three different observation perspectives, For the first The pixel coordinates of the image were collected from each monitoring point. The intrinsic parameter matrix corresponding to the camera, For multi-view triangulation algorithms, These are the spatial coordinates of the fault point.

2. The method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology according to claim 1, characterized in that: Step (1) specifically refers to: acquiring ultraviolet light images through an ultraviolet sensor unit to detect the discharge intensity or abnormality on the surface of the equipment, with a working wavelength of 240-280nm and a photon counting sensitivity of ≥1000 photons / second; acquiring infrared light images through an infrared thermal imager unit to detect temperature changes in the equipment, with a temperature measurement range of -40℃ to 550℃; acquiring visible light images through a 4K visible light camera; and the ultraviolet light images, infrared light images, and visible light images constitute a multispectral fusion dataset.

3. The method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology according to claim 1, characterized in that: Step (3) specifically refers to: based on the multi-dimensional fault feature vector The data is divided into four feature segments: discharge intensity, temperature rise gradient, texture defect, and correlation. The dimensions of these segments are 0–31, 32–63, 64–95, and 96–127, respectively. The discharge intensity segment characterizes the discharge photon density in the ultraviolet image; when the maximum value exceeds 500 photons / second in three consecutive frames, it is considered an insulation degradation fault. The temperature rise gradient segment measures the local temperature rise in the infrared image. The gradient feature is used to determine a fault, i.e., an abnormal temperature, when the L2 norm exceeds 10℃ / cm. The texture defect feature segment is based on the statistical characteristics of LBP values; when the mean variance exceeds 0.25, a fault is identified. If the spatial variance distribution is concentrated, it is identified as surface contamination; if the distribution is discrete, it is identified as mechanical cracks. The correlation feature segment is used to measure the coupling strength between different modes; when the Pearson correlation coefficient is greater than 0.8, a fault, i.e., arc discharge, is identified; when the Pearson correlation coefficient is less than 0.5, a fault, i.e., local overheating, is identified. Overall failure probability This is used to quantify the severity of fault conditions and to construct a tiered response mechanism based on the fault status of each feature segment: when any one of the four feature segments fails, the location of the fault point is dynamically rendered in the digital twin platform and a yellow warning is issued, while an alarm message is pushed; when any two of the four feature segments fail or When the value is ≥0.8, an audible and visual alarm is triggered, a maintenance work order is automatically generated, and historical cases are linked; when When the value is ≥0.95, the system will shut down and isolate the fault, link the fire control system, and push the location of the fault point and the emergency plan to the emergency terminal simultaneously.

4. The method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology according to claim 1, characterized in that: Step (4) specifically refers to: A digital twin platform is built based on the Three.js engine, which integrates multispectral monitoring data in real time and dynamically renders temperature field distribution, discharge intensity and defect location; Based on the fault type, retrieve similar historical cases with a matching degree of over 90% from the knowledge graph, automatically output a structured handling plan, and notify the operation and maintenance personnel in real time. After maintenance is completed, the system is re-inspected. If the discharge intensity drops to the safe threshold and the temperature returns to normal, the equipment health score is updated and the alarm threshold is automatically optimized, completing the closed-loop management of the entire process from perception to analysis, and then to decision-making and execution.

5. The method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology according to claim 1, characterized in that: Step (2b) specifically includes the following steps in sequence: (2b1) The SIFT algorithm is used to extract feature points and descriptors from the preprocessed visible light image, ultraviolet image, and infrared image, respectively; (2b2) ​​Feature matching is performed using Euclidean distance; (2b3) Use the RANSAC algorithm to estimate the transformation matrix and remove mismatched points, and calculate the optimal homography matrix; (2b4) Resample the ultraviolet and infrared images according to the optimal homography matrix to make the ultraviolet and infrared images spatially aligned with the visible light image, and unify all images into the spatial coordinate system of the visible light image.

6. The method for early warning and diagnosis of ultra-high voltage valve halls based on multispectral image feature fusion technology according to claim 1, characterized in that: Step (2c) specifically includes the following steps in sequence: (2c1) The ultraviolet image branch uses convolutional layers and LSTM layers to extract the discharge intensity feature vector from the aligned ultraviolet image. : To extract the dynamic features of discharge intensity from ultraviolet images, the input ultraviolet image is first processed by a 3×3 convolutional kernel. Feature extraction is performed, and the feature map is obtained by applying the ReLU activation function. : ； in, The convolution kernel weight matrix is... These are the bias parameters for the convolutional layer. This represents a spatial feature map of 64 channels. and These are the height and width of the image, i.e., the spatial dimensions; right Perform spatial pooling to obtain the pooled feature sequence. ; Next, the pooled feature sequences The input is fed into the LSTM layer, combined with the hidden state from the previous time step. and memory unit Perform time series modeling to obtain the state at the current moment: ； in, Let this be the hidden state vector at the current time step. The state of the memory cell at the current time step; The temporal state information of the features in consecutive frames of ultraviolet images is modeled using LSTM layers, outputting a 64-dimensional discharge intensity feature vector. ; (2c2) The infrared image branch uses residual blocks and the HOG algorithm to extract the temperature rise gradient feature vector from the aligned infrared image. : By extracting features from the input infrared image using residual blocks, the thermal distribution structure is enhanced to obtain an intermediate feature map. : ； in, To input an infrared image, It has a 3×3 convolution kernel; For activation functions; Gradient calculations are performed on infrared images to obtain the spatial derivative of temperature changes. and direction angle : ； In the formula, This is the temperature value; Intermediate feature map extracted from residual block and By fusing the HOG algorithm, the local temperature rise direction and amplitude information are encoded, and a 64-dimensional temperature rise gradient feature vector is output. ; (2c3) The visible light image branch uses multi-scale LBP and residual blocks to extract texture defect feature vectors from the aligned visible light image. : By encoding the gray-level relationship between each pixel and its neighborhood through multi-scale LBP, fine-grained texture features are extracted to obtain the LBP value at the pixel location: ； in, This is the grayscale value of the current pixel. For the first The grayscale value of each neighbor, This indicates which neighboring node is currently in the network. It is a step function. The grayscale difference between the current neighboring pixel and the center pixel. The position of the current pixel; The LBP values ​​of all pixels in the entire image are calculated to form an LBP feature map; then, the LBP feature map is fused with the deep texture features extracted from the residual blocks to output a 64-dimensional texture defect feature vector. .

7. An electronic device, comprising: processor; as well as A memory storing computer program instructions, which, when executed by the processor, cause the processor to perform the ultra-high pressure valve hall early warning and diagnosis method based on multispectral image feature fusion technology as described in any one of claims 1-6.

8. A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the ultra-high pressure valve hall early warning and diagnosis method based on multispectral image feature fusion technology as described in any one of claims 1-6.

Images