A dual-spectrum fault diagnosis method, device and equipment for power equipment

Abstract

A method, apparatus, and device for dual-spectrum fault diagnosis of power equipment are disclosed, relating to the field of power equipment fault diagnosis technology. The method includes: constructing a static spatial mapping relationship between a visible light image region and an infrared temperature measurement point using prior information from a fixed monitoring location; acquiring infrared temperature data and a real-time visible light image of the equipment under test; mapping the infrared temperature data to a visible light coordinate system based on the static spatial mapping relationship to generate structured temperature data; generating an infrared feature vector based on the structured temperature data; extracting morphological semantic feature vectors from the real-time visible light image; training the visible light feature extraction model using virtual samples generated based on a multi-physics coupling mechanism; inputting the infrared feature vectors and morphological semantic feature vectors into a classification model for fault identification; verifying the fault identification results using physical rules; and outputting the diagnostic results. This invention improves the reliability and interpretability of the diagnostic results.

Description

Technical Field

[0001] This invention relates to the field of power equipment fault diagnosis technology, and specifically to a dual-spectrum fault diagnosis method, apparatus, and equipment for power equipment. Background Technology

[0002] With the advancement of smart grid construction, fault diagnosis technology for substation power equipment has transitioned from traditional manual inspection to automated monitoring. Currently, the mainstream technologies in this field mainly involve infrared thermal imaging detection and visible light imaging, but both suffer from significant technical bottlenecks. 1. Diagnostic technology based on infrared thermal imaging Infrared diagnostics primarily rely on power industry standards (such as DL / T 664), triggering alarms by setting absolute temperature thresholds or relative temperature differences. However, this method has the following problems: (1) Poor anti-interference ability: This type of method is extremely sensitive to the environment. Sunlight reflection, radiation interference from surrounding high-heat equipment, and the insignificant temperature rise during low-load operation can all lead to extremely high false alarm or false alarm rates; (2) Obvious blind spots: It can only detect faults accompanied by heat. For mechanical damage in a cold state (such as insulator cracks, early stage of hardware corrosion, dirt accumulation, etc.), infrared technology is completely ineffective and cannot achieve full-dimensional state perception; (3) Redundancy in data storage and transmission bandwidth: The continuous streaming of high-resolution infrared video matrix in industrial sites consumes a lot of memory and network bandwidth of edge computing terminals, while the data with actual diagnostic value is only the temperature measurement points on the surface of key equipment, resulting in a waste of computing power and resources.

[0003] 2. Visible Light Image Detection Technology Based on Deep Learning With the development of deep learning, visible light image detection based on supervised learning models such as YOLO, Faster R-CNN, and ResNet has been widely applied. These methods typically achieve device localization and defect identification by training end-to-end models on large-scale datasets. However, these methods suffer from the following problems: (1) Heavy reliance on massive labeled samples: Power systems are high-reliability scenarios, and equipment failures exhibit typical "long-tail distribution" characteristics (massive normal samples, extremely scarce failure samples). Existing fully supervised learning models are difficult to converge and have extremely poor generalization ability when there are insufficient failure samples (especially rare failures).

[0004] (2) Inefficiency of “training from scratch”: Most existing solutions train models from scratch for specific power scenarios, ignoring the common characteristics of common defects in the industry (such as steel corrosion and concrete cracks), and failing to use transfer learning to solve the small sample problem.

[0005] (3) Unable to detect internal hidden dangers: Visible light technology can only identify abnormal surface morphology and is powerless to detect hidden faults such as poor internal contact of crimped pipes and broken strands inside wires that are "heated but not broken". Summary of the Invention

[0006] This invention provides a dual-spectrum fault diagnosis method, apparatus, and equipment for power equipment, which can solve the technical problems of extremely scarce fault samples, large blind zone and easy conflict in single-mode diagnosis in the prior art.

[0007] In a first aspect, the present invention provides a dual-spectrum fault diagnosis method for power equipment, the method comprising: By utilizing prior information from fixed monitoring locations, a static spatial mapping relationship between visible light image regions and infrared temperature measurement points is constructed. The infrared temperature data and real-time visible light image of the device under test are acquired. Based on the above static spatial mapping relationship, the infrared temperature data is mapped to the visible light coordinate system to generate structured temperature data, and an infrared feature vector is generated based on the above structured temperature data. The morphological semantic feature vectors are extracted from the real-time visible light images using a pre-trained visible light feature extraction model; wherein the visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism. The infrared feature vector and the morphological semantic feature vector are input into the classification model for fault identification, and the fault identification results are verified by combining physical rules to output the diagnostic results.

[0008] In conjunction with the first aspect, in one implementation, the above-mentioned construction of a static spatial mapping relationship between a visible light image region and an infrared temperature measurement point using prior information from a fixed monitoring location specifically includes: Offline acquisition of visible light and infrared reference images at fixed monitoring locations; Based on the equipment ledger structure, the component areas of the key monitoring objects in the aforementioned visible light reference images are given unique names; By establishing the mapping relationship between the aforementioned component regions and the infrared temperature measurement points in the aforementioned infrared reference image through calibration, and combining the naming of the aforementioned component regions, the aforementioned static spatial mapping relationship is constructed.

[0009] In conjunction with the first aspect, in one embodiment, after mapping the aforementioned infrared temperature data to a visible light coordinate system to generate structured temperature data, the method further includes: Release the storage space occupied by the original infrared video stream corresponding to the above infrared temperature data.

[0010] In conjunction with the first aspect, in one implementation, generating an infrared feature vector based on the aforementioned structured temperature data specifically includes: The structured temperature data above is analyzed to obtain the temperature value and spatial location information corresponding to each visible light image region; based on the thermodynamic conduction law of power equipment, the interphase temperature difference and ambient temperature rise of the region are calculated. The above temperature values, phase-to-phase temperature difference, and ambient temperature rise are fused with the above spatial location information to generate the above infrared feature vector.

[0011] In conjunction with the first aspect, in one implementation, the training process of the aforementioned visible light feature extraction model includes: The backbone network of a deep convolutional neural network was pre-trained using a general industrial surface defect dataset as the source domain. After freezing the weights of the backbone network, the classification layer of the deep convolutional neural network is fine-tuned using the virtual samples to obtain the visible light feature extraction model.

[0012] In conjunction with the first aspect, in one implementation, the above method further includes generating virtual samples generated by a multiphysics coupling mechanism, specifically including: The virtual fault type is determined based on the multiphysics coupling mechanism; Based on the aforementioned visible light reference image and the aforementioned virtual fault types, a generation strategy is determined; According to the above generation strategy, the visible light defect morphology is rendered using a generative model, and the corresponding virtual temperature data is generated synchronously based on the thermodynamic equation to output virtual samples.

[0013] In conjunction with the first aspect, in one implementation, the classification model is a gradient boosting tree model. The infrared feature vector and the morphological semantic feature vector are input into the classification model for fault identification, specifically including: The infrared feature vectors and the morphological semantic feature vectors are concatenated to form a joint feature matrix; The nonlinear decision boundary of the joint feature matrix is ​​learned through the gradient boosting tree model described above, and the fault identification result is output. The SHAP algorithm is used to calculate the marginal contribution of each feature in the joint feature matrix to the prediction results of the above fault categories, and the dominant feature factor is extracted.

[0014] In conjunction with the first aspect, in one implementation method, the above-mentioned verification of the fault identification results based on physical rules specifically includes: Obtain the physical attributes of the equipment in the equipment ledger of the device under test, and combine the above fault identification results and dominant feature factors to match the physical mechanism template through the predicate logic engine; When the above fault identification results conflict with the logical deduction results of the above physical mechanism template, the above fault identification results shall be corrected or marked as pending manual review; the above diagnostic results include physical mechanism deduction information.

[0015] In a second aspect, the present invention provides a dual-spectrum fault diagnosis device for power equipment, the device comprising: The static coordinate mapping module is used to construct a static spatial mapping relationship between the visible light image area and the infrared temperature measurement point using prior information from a fixed monitoring location; it is also used to acquire the infrared temperature data and real-time visible light image of the device under test, and to map the infrared temperature data to the visible light coordinate system based on the above static spatial mapping relationship to generate structured temperature data. The infrared feature extraction module is used to generate infrared feature vectors based on the structured temperature data mentioned above. The visible light feature extraction module is used to extract morphological and semantic feature vectors from the real-time visible light image using a pre-trained visible light feature extraction model; wherein the visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism. The decision fusion and diagnosis module is used to input the above-mentioned infrared feature vectors and the above-mentioned morphological semantic feature vectors into the classification model for fault identification, and to verify the fault identification results in combination with physical rules, and output the diagnosis results.

[0016] Thirdly, the present invention provides a dual-spectrum fault diagnosis device for power equipment, the device comprising a processor, a memory, and a dual-spectrum fault diagnosis program for power equipment stored in the memory and executable by the processor, wherein when the dual-spectrum fault diagnosis program for power equipment is executed by the processor, the steps of the dual-spectrum fault diagnosis method for power equipment described above are implemented.

[0017] The beneficial effects of the technical solutions provided by the embodiments of the present invention include: By training a visible light feature extraction model using virtual samples generated based on multi-physics coupling mechanisms, the problem of poor model generalization ability caused by the extreme scarcity of power system fault samples is effectively solved. Through dual-spectral fusion diagnosis, the blind spots of single-mode detection are compensated, and full-dimensional state perception is realized. By combining physical rules to verify the fault identification results, the hidden danger of easy conflict in multi-source information fusion is eliminated, and the reliability and interpretability of the diagnostic results are improved. This solves the technical problems of extreme scarcity of fault samples and large blind spots and easy conflicts in single-mode diagnosis in related technologies. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an embodiment of the dual-spectrum fault diagnosis method for power equipment of the present invention. Figure 2 This is a detailed flowchart of step A1 in an embodiment of the present invention; Figure 3This is a detailed flowchart of step A2 in an embodiment of the present invention; Figure 4 This is a detailed flowchart of step A3 in an embodiment of the present invention; Figure 5 This is a detailed flowchart of step A4 in an embodiment of the present invention; Figure 6 This is a detailed flowchart of step A5 in an embodiment of the present invention; Figure 7 This is a functional module diagram of an embodiment of the dual-spectrum fault diagnosis device for power equipment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of the dual-spectrum fault diagnosis device for power equipment involved in the embodiments of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In a first aspect, embodiments of the present invention provide a dual-spectrum fault diagnosis method for power equipment.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the dual-spectrum fault diagnosis method for power equipment according to the present invention. The aforementioned dual-spectrum fault diagnosis method for power equipment is applied to intelligent operation and maintenance scenarios in power systems. Combining industrial image processing and artificial intelligence deep learning technologies, it aims to solve technical problems in existing substation power equipment condition monitoring, such as an extreme scarcity of fault samples, large and easily conflicting blind spots in single-modal diagnosis, redundant computing power for dual-modal fusion, and a lack of physical interpretability of the model.

[0022] like Figure 1 As shown, the above method specifically includes: S1. Using prior information from fixed monitoring locations, construct a static spatial mapping relationship between visible light image regions and infrared temperature measurement points; In this embodiment, in a fixed-position online monitoring scenario, a static spatial mapping relationship between the visible light region of interest and the infrared temperature measurement point is established, and the static spatial mapping relationship is solidified into a spatial mapping dictionary.

[0023] S2. Acquire the infrared temperature data and real-time visible light image of the device under test. Based on the above static spatial mapping relationship, map the infrared temperature data to the visible light coordinate system to generate structured temperature data, and generate an infrared feature vector based on the above structured temperature data. The aforementioned visible light coordinate system is a reference image coordinate system established based on a visible light reference image at a fixed monitoring position during the offline phase. During the online phase, the real-time visible light image acquired uses the aforementioned reference image coordinate system to index the component area under the condition that the fixed monitoring position, field of view, and imaging parameters remain stable, so that the static spatial mapping relationship can attach the infrared temperature data to the corresponding visible light component area.

[0024] S3. Extract morphological semantic feature vectors from the above real-time visible light images using a pre-trained visible light feature extraction model; wherein, the above visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism; S4. Input the above infrared feature vector and the above morphological semantic feature vector into the classification model for fault identification, and verify the fault identification results in combination with physical rules, and output the diagnostic results.

[0025] In this embodiment, the visible light feature extraction model is trained using virtual samples generated based on the multi-physics coupling mechanism, which effectively solves the problem of poor model generalization ability caused by the extreme scarcity of power system fault samples; the blind spots of single-mode detection are made up for by dual-spectrum fusion diagnosis, realizing full-dimensional state perception; and the fault identification results are verified by combining physical rules, eliminating the hidden danger of easy conflict in multi-source information fusion and improving the reliability and interpretability of the diagnostic results.

[0026] Based on the above embodiments, in this embodiment, step S1 above utilizes prior information from a fixed monitoring location to construct a static spatial mapping relationship between the visible light image region and the infrared temperature measurement point, specifically including: First, visible light and infrared reference images were acquired offline at fixed monitoring locations; Then, based on the equipment ledger structure, the component areas of the key monitoring objects in the above visible light reference images are uniquely named; Finally, the mapping relationship between the above-mentioned component regions and the infrared temperature measurement points in the above-mentioned infrared reference image is established by calibration, and the above-mentioned static spatial mapping relationship is constructed by combining the naming of the above-mentioned component regions.

[0027] In this embodiment, by utilizing prior information from fixed monitoring locations to construct static spatial mapping relationships offline, the high computational overhead and parallax shift risk caused by environmental jitter in traditional online dynamic image registration are avoided, significantly improving the stability of the mapping and the efficiency of data processing. By uniquely naming component regions according to the equipment ledger structure and combining the naming with the mapping relationship, semantic association between image regions and equipment physical attributes (such as material and type) is realized, providing a reliable data foundation for subsequent fault diagnosis and verification based on physical rules.

[0028] Furthermore, in one embodiment, after the above-mentioned infrared temperature data is mapped to the visible light coordinate system to generate structured temperature data, the method further includes: Release the storage space occupied by the original infrared video stream corresponding to the above infrared temperature data.

[0029] In this embodiment, by directly extracting infrared temperature data and attaching it to the visible light coordinate system, and then destroying the original infrared video stream, the computing power and transmission bandwidth of the edge computing terminal are greatly released.

[0030] Furthermore, in one embodiment, generating an infrared feature vector based on the aforementioned structured temperature data specifically includes: First, the structured temperature data above is analyzed to obtain the temperature value and spatial location information corresponding to each visible light image region; based on the thermodynamic conduction law of power equipment, the interphase temperature difference and ambient temperature rise of the region are calculated. Then, the temperature values, phase-to-phase temperature difference, and ambient temperature rise mentioned above are fused with the spatial location information to generate the infrared feature vector.

[0031] In this embodiment, by directly parsing the structured temperature data, i.e., the visible light coordinate-temperature dictionary, instead of processing the original image matrix, the data dimensionality and computing power consumption are significantly reduced. The phase-to-phase temperature difference and ambient temperature rise in the region are calculated according to the power guidelines, ensuring that the infrared features meet industry standards and have clear physical interpretability, and eliminating environmental interference from absolute temperature measurement. By fusing spatial location information (such as center coordinates and scale), an infrared feature vector containing spatial topological anchor points and thermodynamic states is output, achieving precise spatial alignment between infrared features and the visible light region, providing a reliable spatial and physical basis for subsequent multimodal fusion diagnosis.

[0032] Furthermore, in one embodiment, the training process of the above-mentioned visible light feature extraction model includes: First, the backbone network of the deep convolutional neural network is pre-trained using a general industrial surface defect dataset as the source domain. Then, after freezing the weights of the backbone network, the classification layer of the deep convolutional neural network is fine-tuned using the virtual samples to obtain the visible light feature extraction model.

[0033] Preferably, in this embodiment, the above method further includes generating virtual samples generated by the multiphysics coupling mechanism, specifically including: First, the virtual fault type is determined based on the multiphysics coupling mechanism; Among them, the visible light reference image and spatial mapping dictionary are received as reference data. Based on the multi-physics coupling mechanism of the device's underlying layer, the virtual fault types are strictly divided into three categories: dual-modal abnormal heating type (Category A), cold mechanical damage type (Category B), and concealed internal overheating type (Category C).

[0034] Among them, Class A faults are dual-mode abnormal heating faults, which refer to faults in which the device under test simultaneously exhibits visible light morphological defects and abnormal infrared temperature rises. For example, corrosion, oxidation, or contact deterioration at metal connection points leads to increased contact resistance and temperature rise. Class B faults are cold-state mechanical damage faults, which refer to faults in which the device under test has visible light morphological defects, but does not show obvious infrared temperature rises under the current monitoring state. For example, insulator cracks, damage, or surface defects. Class C faults are hidden internal overheating faults, which refer to faults in which the device under test does not show obvious visible light surface defects, but internal heat sources cause abnormal infrared temperature rises on the surface of the device through heat conduction. For example, poor contact of internal contacts or internal contact deterioration of current-conducting connection points.

[0035] Then, based on the aforementioned visible light reference image and the aforementioned virtual fault type, a generation strategy is determined; Specifically, if the virtual fault type is A, the generation strategy is set to synchronously render the visible light defect morphology in the target component area and generate the corresponding abnormal temperature rise data based on the thermodynamic model; if the virtual fault type is B, the generation strategy is set to render only the visible light defect morphology, while the infrared temperature data of the corresponding area remains in a baseline normal state; if the virtual fault type is C, the generation strategy is set to maintain the baseline normal morphology of the visible light image and inject the abnormal temperature rise data formed by internal heat source conduction into the structured temperature data of the corresponding area based on the thermodynamic model.

[0036] Finally, following the above generation strategy, the visible light defect morphology of the target component region is rendered using a generative model, and corresponding virtual temperature data is generated synchronously based on the thermodynamic equation to output virtual samples.

[0037] In this embodiment, a virtual sample is formed by combining a generative large model with thermodynamic laws and simultaneously synthesizing a visible light image with specific morphological defects and a corresponding virtual temperature dictionary.

[0038] In this embodiment, cross-domain transfer learning pre-training using a general industrial surface defect dataset effectively reduces the reliance on massive labeled samples in the power scenario. After locking the network model weights, forward propagation of real or virtual generated visible light component area images enables rapid decoupling and extraction of multi-label morphological probability vectors with generalized physical meanings such as "rust" and "cracks," avoiding the risk of overfitting caused by full parameter fine-tuning. At the same time, combining virtual samples generated based on multi-physics coupling mechanisms ensures the physical and logical consistency of the training data, enhancing the model's cross-scenario generalization ability and feature interpretability for different power equipment.

[0039] In some embodiments, fault diagnosis is performed by combining image-level infrared and visible light fusion techniques, i.e., pixel-level fusion and simple feature-level stitching. However, this approach has the following problems: (1) Registration is difficult and computational cost is extremely high: Pixel-level fusion requires strict alignment of infrared and visible light images. However, in outdoor natural environments, due to wind and thermal expansion and contraction, dual-light cameras are prone to parallax shift, resulting in "ghosting". At the same time, maintaining the geometric transformation registration of the two high-definition image matrices in real time at the edge computing end has a huge computational load.

[0040] (2) “Pseudo-fusion” lacking physical mechanism support: Existing feature splicing often simply concatenates the extracted feature vectors and directly inputs them into the classifier. This approach does not consider the physical and logical conflicts between different modal features (e.g., false heat caused by strong light vs. real overheating), and lacks a verification mechanism similar to a “logic gate”, resulting in low reliability at the decision level.

[0041] Data augmentation techniques based on traditional Generative Adversarial Networks (GANs) have been employed in some systems to synthesize fault samples using GAN models. However, this approach suffers from poor generation quality, semantic drift, and a lack of multiphysics coupling. This is because unconstrained generative models are prone to artifacts or pattern collapse when synthesizing high-resolution industrial textures. More critically, purely visual GAN ​​models neglect the underlying physical coupling between equipment damage and thermodynamic temperature rise, often resulting in multimodal data that violates objective physical laws. This leads to the model fitting a specific noise distribution, resulting in a decrease in recognition rate in real-world scenarios.

[0042] Furthermore, in this embodiment, the classification model is a gradient boosting tree model. The infrared feature vector and the morphological semantic feature vector are input into the classification model for fault identification, specifically including: The infrared feature vectors and the morphological semantic feature vectors are concatenated to form a joint feature matrix; The nonlinear decision boundary of the joint feature matrix is ​​learned through the gradient boosting tree model described above, and the fault identification result is output. The SHAP algorithm is used to calculate the marginal contribution of each feature in the joint feature matrix to the prediction results of the above fault categories, and the dominant feature factor is extracted.

[0043] In this embodiment, by using the gradient boosting tree model to perform nonlinear classification decisions on the dual-stream heterogeneous feature matrix, the infrared thermal features and visible light morphological semantic features are effectively integrated, improving the accuracy and generalization ability of fault identification. Combined with the SHAP algorithm to remove the decision-making dominant factors, the accurate extraction of feature attribution and dominant factors is achieved, breaking the black-box nature of machine learning.

[0044] Preferably, in this embodiment, the above-mentioned verification of the fault identification result based on physical rules specifically includes: Obtain the physical attributes of the equipment in the equipment ledger of the device under test, and combine the above fault identification results and dominant feature factors to match the physical mechanism template through the predicate logic engine; When the above fault identification results conflict with the logical deduction results of the above physical mechanism template, the above fault identification results shall be corrected or marked as pending manual review; the above diagnostic results include physical mechanism deduction information.

[0045] In this embodiment, by automatically calling the device's physical attribute tags (such as metal / insulator) and triggering the underlying physical equations (such as Joule's law), and using the predicate logic engine for consistency verification, the physical self-consistency and reliability of the diagnostic results are ensured, eliminating the logical conflicts that may exist in a pure data-driven model; finally, a white-box structured diagnostic report with rigorous mechanism support is output, which significantly enhances the interpretability and operational trust of the diagnostic results.

[0046] This embodiment takes the fault diagnosis of a high-voltage circuit breaker in phase A of a single circuit as an example, specifically involving a dual-spectrum fault diagnosis method for power equipment based on cross-modal dictionary dimensionality reduction, multi-physics field generation, and transfer learning, applicable to fixed-location online monitoring scenarios.

[0047] The specific steps of the above method are as follows: Step A1, static coordinate mapping and infrared data dimensionality reduction extraction, aims to leverage the fixed camera positions of substation monitoring cameras to establish a low-level data index benchmark through offline calibration, and to completely reduce the high-dimensional infrared image stream into a structured temperature dictionary, thereby maximizing the computing power and storage of edge computing terminals. Figure 2 As shown, the specific implementation is as follows: Step A101: Acquisition of the reference image: During system initialization, the dual-spectrum camera is positioned at a preset monitoring location (denoted as fixed camera position X). Under normal operating conditions and good lighting, one visible light reference image is captured. and an infrared reference image These two images will serve as the base map for subsequent coordinate calibration, i.e., the reference image pair.

[0048] Step A102: Component area definition and naming: Based on the equipment ledger structure of the substation, key monitoring objects for the baseline image pairs are uniquely named and defined. In this embodiment, for circuit breaker equipment, the following key component regions (ROIs) are defined: 1. "One-phase A circuit breaker equalizing ring" (for metal fittings); 2. "One-phase A circuit breaker insulating bushing" (for the insulating medium part); 3. "One-phase A circuit breaker outgoing terminal" (for the current-conducting connection part).

[0049] Step A103: Establish a spatial mapping dictionary: Using manual calibration software, a one-to-one mapping relationship between infrared thermometry point sets and visible light region bounding boxes is established on reference image pairs, and this mapping is then fixed into a configuration dictionary. Each entry in the dictionary contains the following core fields: 1. ID: A unique identifier for the component (e.g., "equalizing ring_01"); 2. : Coordinates of the rectangular region selected in the visible light image These represent the pixel coordinates of the top left and bottom right corners, respectively, and are used for subsequent texture feature extraction. 3. The set of coordinates of several key temperature measurement points in the infrared image. This is used for subsequent temperature data extraction; 4. Type (optional): Component physical type marker (such as "Metal" or "Insulation"), used for subsequent reasoning logic differentiation.

[0050] The specific dictionary structure example is as follows: { "One-time A-phase circuit breaker equalizing ring": { "ID": "Equalizing Ring_01", Type: "Metal", "Vis_ROI": [378, 582, 1346, 1500], "IR_Points": [(22, 183), (45, 190), (60, 185),…] }, "One-time A-phase circuit breaker insulating bushing": { "ID": "Casing_01", "Type": "Insulation", "Vis_ROI": [913, 272, 1327, 727], "IR_Points": [(165, 136), (170, 140),…] } } Step A104: Infrared temperature extraction and cross-modal dimensionality reduction: First, the system extracts the actual physical temperature data of each calibration point (IR_Points) in the real-time infrared image and constructs an intermediate infrared coordinate-temperature dictionary in memory. Then, cross-modal mounting is performed; that is, based on the spatial mapping dictionary in step A103, the temperature data arrays of the infrared temperature measurement points are directly mapped and assigned to the corresponding Vis_ROI key name in the visible light image, thus obtaining the final merged visible light coordinate-temperature dictionary. Finally, the memory space occupied by the real-time infrared image and the infrared coordinate-temperature dictionary is released, retaining only the visible light coordinate-temperature dictionary for subsequent processing.

[0051] Step A105: Output structured temperature data The module's final output format is a reduced-dimensional dictionary as follows: { "One-time A-phase circuit breaker insulating bushing": { "ID": "Casing_01", "Type": "Insulation", "Vis_ROI": [913, 272, 1327, 727], "IR_Points_temp": [68.5, 70.2, 69.1,...] } } Step A2: Virtual Fault Generation Based on Multiphysics Mechanism Classification This step introduces a joint enhancement mechanism based on the Latent Diffusion Model and thermodynamic principles, generating multimodal paired data according to three physical characteristics (A, B, and C) for training the visible light feature extraction model. For example... Figure 3 As shown, it specifically includes: Step A201: Construct a multiphysics joint generation framework based on electrical mechanisms: (1) Fault type A (bimodal anomaly, such as severe corrosion): The diffusion model generates heavily corroded textures in the corresponding ROI region of the visible light reference image. Simultaneously, according to Joule's law... Contact resistance Due to a significant increase in the oxide layer. The dictionary entry for this region... Perform additional temperature simulation: (1) in, The thermal resistivity of the material. For the set corrosion intensity, This is the rated load current. This is the reference infrared temperature value for the infrared thermometer corresponding to this ROI region. The virtual fault state infrared temperature value is superimposed after the temperature rise caused by corrosion.

[0052] (2) Fault type B (cold defect, such as insulation crack): The diffusion model generates mechanical cracks in the visible light reference image. Since there is no leakage current heating path, the infrared physical field remains unchanged, i.e. .

[0053] (3) Fault type C (hidden overheating, such as internal strand breakage): Keep the original visible light image unchanged (virtual image) Based on the steady-state heat conduction equation Internal heat source Conducted to the surface through the metal casing, the system sets corresponding parameters in the dictionary. Temperature rose significantly: .

[0054] Step A202: Generative Model Construction Generation of visual defect features (model principle): A conditional diffusion model combining ControlNet structural constraints is adopted. Assume a normal visible light image. Edge features are The text prompt is: During model training, random noise is added to the latent features and a denoising network is trained to predict this noise. This enables the model to learn the ability to generate fault textures under structural and textual constraints. The objective function is as follows: (2) in, This is the training loss function for the potential diffusion model; Represents the mathematical expectation; The input is a normal visible light image sample. For the corresponding fault type or text prompt message; The noise is sampled from a standard normal distribution; t is the time step in the diffusion process. In the first Each time step for latent features The noise latent features obtained after adding noise, among which , This represents the encoder that maps the input image to the latent space; For parameters Denoising networks in the latent features of noise Time step Text condition vector and structural constraint features Noise predicted under common constraints; To provide text prompts The conditional semantic vector obtained through encoding; These are edge or structural constraint features extracted from normal visible light images; This represents the squared L2 norm. The objective function constrains the predicted noise to the actual noise. The differences between them enable the model to learn and generate virtual visible light images with specified fault textures while preserving the original device structure outline.

[0055] Step A3: Extract infrared feature vectors Obtain the reduced-dimensional dictionary, i.e., the visible light coordinate-temperature dictionary. Extract the temperature measurement point data based on this dictionary and construct a 6-dimensional thermal feature vector, which serves as the infrared feature vector. Assume the current monitoring component is... (e.g., phase A equalizing ring), other phase components in the same group are (e.g., phase B and phase C equalizing rings). For example... Figure 4 As shown, it specifically includes: Step A301: Extracting original physical quantities and spatial positions from the visible light coordinate-temperature dictionary. Read the dictionary of the current component Array, get the highest temperature in the region and average temperature At the same time, read Extract the spatial location information (such as center point coordinates) from the rectangular coordinates of the field. ), which serves as the spatial topological anchor point for this feature vector.

[0056] Step A302: Calculate the temperature difference index and temperature rise index. Original physical quantity: highest temperature in the region Average temperature .

[0057] ① Interphase temperature difference ( ): The temperature difference between this component and other phases in the same group, used to eliminate environmental and load interference. The calculation formula is: (3) in, The temperature of other phase equipment in the same group.

[0058] ② Ambient temperature rise ( ): The temperature difference between this component and the ambient temperature. The calculation formula is: (4) in, This is the ambient reference temperature.

[0059] Step A303: Calculate the severity level characteristics: According to the power industry standard (DL / T 664), the above continuous temperature difference values ​​are mapped to discrete grade scores (0-4 points), denoted as... : For example, temperature difference Mapped to 3 (critical). Mapped to 2 (serious).

[0060] Step A304: Infrared Feature Vector Output: (5) Step A4, visible light feature extraction, aims to address the cold start problem of deep networks failing to converge due to the extreme scarcity (zero or few samples) of real-world fault samples in power equipment, thereby resolving the underlying algorithmic logic issue. This embodiment abandons the traditional, inefficient approach of training from scratch to build an end-to-end object detection network for specific power equipment, instead constructing a low-level visual cognition foundation based on cross-domain transfer learning, such as... Figure 5 As shown, the specific algorithm flow is as follows: Step A401: Large-scale pre-training of the source domain and construction of residual maps During the offline preparation phase, a public dataset of general industrial surface defects was selected. (Databases such as NEU-DET steel defects and concrete cracks contain a large number of real samples of common physical degradation such as "rust", "cracks", "dents" and "normal") as the source domain.

[0061] A deep residual network was selected as the backbone feature extractor. The vanishing gradient problem in deep networks is solved by introducing residual blocks. The core mapping mechanism is as follows: (6) in, The input features for the residual block are... The output characteristics of the residual block, Let be the residual mapping function composed of convolutional layers, normalization layers, and nonlinear activation layers in the residual block. These are the trainable network weight parameters in the residual mapping function. Indicates input features The residual mapping result is added to the identity mapping branch. This formula indicates that the residual block is not directly learned from... arrive Instead of learning the complete mapping, it learns the residual changes between input and output features, thereby alleviating the gradient vanishing and feature degradation problems in deep network training.

[0062] In the source domain, the network minimizes the multi-class cross-entropy loss function. Massive iterative training forces the convolution kernel matrix to learn spatial and frequency domain feature representations of general material degradation: (7) in, This refers to the multi-class cross-entropy loss function used in the source domain pre-training phase. The number of training samples in the source domain. The total number of common defect categories defined for the source domain. For the source domain training sample index, and For category indexing, For the first The source domain sample at the th th The true label on the class, when the sample belongs to the first... Class Time ,otherwise ; For the network to the first The sample belongs to the first The unnormalized predicted value of the class output, i.e., the Logits value; For the network to the first The sample belongs to the first The Logits value output by the class; It is an exponential function. This indicates that after Softmax normalization, the th The sample belongs to the first The predicted probability of a class. This loss function, by constraining the difference between the predicted probability and the true class label, enables the backbone network to learn the texture, edge, and morphological semantic features of common industrial defects.

[0063] Step A402: Target Domain Network Layer Weight Freezing and Classification Head Fine-tuning When the backbone network After convergence in the source domain, its shallow convolutional layers have learned common Gabor filter features such as texture and edge, while its deep convolutional layers have the ability to generalize higher-order semantic concepts such as "rust" and "crack". To prevent semantic drift or catastrophic forgetting when processing specific power equipment images later, this embodiment strictly freezes the backbone feature extractor when migrating the network to an online environment. All deep and shallow network weight parameters It is used only as a constant "semantic yardstick".

[0064] Simultaneously, a multi-label semantic classification head layer is added at the end of the network. The generated virtual visible light samples are input into the frozen backbone network and the initialized multi-label semantic classification head. The training loss is calculated, and only the weights of the multi-label semantic classification head are updated until the model converges. Specifically, the multi-label semantic classification head is fine-tuned using virtual samples to adapt to the semantic discrimination boundaries of surface defect morphologies in power equipment. After training, the weights of the multi-label semantic classification head are frozen, resulting in the final visible light feature extraction model, which outputs semantic feature vectors for morphologies such as rust, cracks, dents, and normal surfaces.

[0065] Step A403: Multi-label semantic dimensionality reduction and orthogonal feature vector extraction During the offline training phase, the system receives a virtual visible light component region image output by the virtual fault generation module; during the online inference phase, it receives a component region image cropped from the real-time visible light image by the static coordinate mapping module. Both types of images are represented as follows: The input is then fed into a pre-trained network with frozen weights for forward propagation. In the offline training phase, feature vectors are extracted to train the downstream gradient boosting tree model; in the online inference phase, feature vectors are extracted for actual fault diagnosis.

[0066] Understandably, since in the real physical world, the surface of a device may have both "rust" and "cracks" at the same time, and the two are not mutually exclusive in probability, the original Softmax mutual exclusion classification layer at the end of the network is discarded. Extracting continuous feature vectors after the Global Average Pooling layer. To decouple it into probability values ​​with independent physical meaning, this embodiment introduces multiple parallel Sigmoid activation functions to map it to... Interval: (8) in, For the first Predicted probability or response intensity of morphological semantic features; Use the Sigmoid activation function; For multi-label semantic classification head targeting the first The unnormalized predicted value of the class output, i.e., the Logits value; It is an exponential function; Represents category index It can take four types of morphological semantic labels: rust, cracks, dents, and normal. This formula uses the Sigmoid function to... Mapped to The interval allows each type of morphological semantic feature to have an independent probability output, thus supporting the simultaneous existence of multiple surface defect features in the same device area.

[0067] Finally, the module outputs a highly condensed morphological semantic feature vector with clear generalized physical meaning: (9) in, This is a visible light morphological semantic feature vector; The probability or response intensity of the presence of corrosion features in the target component area; The probability or response intensity of the presence of crack features in the target component region; The probability or response intensity of the presence of a depression feature in the target component region; This vector represents the probability or response intensity of the target component region exhibiting a normal morphology. It compresses surface morphology information from visible light images into structured semantic features, which are then used for feature-level fusion with infrared feature vectors and input into a gradient boosting tree model for fault identification.

[0068] Each dimension in this vector This directly quantifies the independent probability strength of the existence of a certain general industrial degradation phenomenon on the surface of the current target power equipment component, providing an extremely clean morphological input boundary for downstream gradient boosting tree multimodal fusion.

[0069] Step A5, Decision Fusion and Diagnosis, aims to solve the nonlinear crossover problem of multimodal heterogeneous features and establish a rigorous reasoning chain from "probabilistic black-box prediction" to "deterministic physical white-box". For example... Figure 6 As shown, it specifically includes: Step A501: Heterogeneous feature asymmetric concatenation and GBDT (Gradient Boosting Decision Tree) decision domain construction Because infrared light outputs continuous / discrete physical quantities with definite dimensions (such as temperature, temperature difference, and guide score), while visible light outputs quantities that are in a state of flux. The probabilistic semantic values ​​of the interval (such as corrosion degree). For such highly heterogeneous structured feature vectors, existing fully connected deep neural networks are prone to getting trapped in local optima and suffer from severe overfitting risks. Therefore, this embodiment selects extreme gradient boosting trees (XGBoost / GBDT) as the fusion model.

[0070] infrared feature vectors With visible light morphological semantic feature vector Perform feature-level late fusion (LateFusion) and concatenate them into a joint feature matrix. The model, during offline training, includes... The ensemble prediction output of the regression trees is Its goal is to minimize the amount of regularization terms included. objective function : (10) in, For the first The objective function during round iteration; n This represents the number of multimodal training samples used to train the gradient boosting tree model. For sample index; For the first The true fault category labels corresponding to each multimodal training sample; For the front After round of iterations, the model is on the first The predicted output for each sample; Let t be the joint feature matrix of the regression tree pair. The output; For the first The joint feature matrix corresponding to each sample; For training the loss function; For the first The regularization term of the regression tree is used to constrain the complexity of the tree model and reduce the risk of overfitting.

[0071] By minimizing the above objective function, the model learns the nonlinear correlation between infrared features and visible light morphological semantic features, and outputs the predicted fault category of the current component during the online operation phase. .

[0072] Step A502: Introduce the SHAP algorithm to calculate the marginal contribution of features and remove dominant factors. To overcome the black-box nature of machine learning and enable its predictions to align with physical rules, the Shapley Value (SHAP) algorithm is introduced to calculate the fusion vector. Global and local marginal contributions of each feature dimension to the current output prediction value : (11) in, For the first The Shapley contribution of each feature to the prediction result of the current fault category; The set of all features that participate in the model prediction; Not including the first Any subset of features of features, and ; Indicates based only on feature subsets The predicted output of the time model, Indicating in the feature subset China added the first The model prediction output after each feature; Indicates the first The marginal contribution of each feature under this combination of features; These are the weighting coefficients for the corresponding feature combinations. This formula obtains the weighted sum of the marginal contributions under all possible feature combinations. The combined contribution of each feature to the model's prediction results.

[0073] After determining the contribution of each feature dimension, instead of directly outputting the original SHAP value, a dynamic threshold is set. Contribution The feature dimensions are extracted to construct the "dominant factor activation subset". (For example: This subset successfully transformed continuous mathematical probabilities into discrete logical triggering conditions.

[0074] Step A503: Perform predicate logic self-test and output white-box diagnostic report. Receive the above prediction categories With activation subset Furthermore, by combining the underlying ontology physical attributes (Type, such as Metal or Insulation) preloaded in the dimensionality reduction dictionary, a rigorous self-checking engine based on predicate logic is established to automatically trigger and output the corresponding underlying physical formulas and structured conclusions: (1) Logic scenario A (determined to be external deterioration of the metal surface causing heat): Triggering conditions: Established.

[0075] Mechanism Output: Determines that the multimodal logic is completely self-consistent, and calls the electrothermal template. Output Diagnostic Report: "Diagnosed as external corrosion defect; Main cause: Visible light corrosion probability surges and abnormal temperature rise occurs in infrared light; Physical mechanism: Metal surface oxidation leads to a reduction in effective current-conducting cross-sectional area and contact resistance." The increase conforms to Joule's law. "External heating characteristics".

[0076] (2) Logic scenario B (determined as cold mechanical damage to the insulating medium): Triggering conditions: Established.

[0077] Mechanism output: Invoke the insulation mechanics template. Output diagnostic report: "Diagnosed as a cracked insulating porcelain bushing; main cause of judgment: visible light crack probability is the dominant factor, infrared guide score is 0 (no heat generation); physical mechanism: the equipment is in a cold state with no leakage current and heat generation, but the insulating medium has suffered mechanical structural damage, the insulation strength has decreased, and there is a risk of flashover breakdown."

[0078] (3) Logical scenario C (determined to be internal concealed heat generation): Triggering conditions: Established.

[0079] Mechanism Output: Invokes the internal heat source conduction template. Output diagnostic report: "Diagnosed as poor internal contact or broken strand; Main cause: Visible light appearance is highly likely to be normal, but infrared temperature rise is consistent with severe defects; Physical mechanism: Appearance is intact with no morphological defects, internal heat source..." Based on the steady-state heat conduction equation The high temperature is conducted to the surface, indicating a hidden internal fault.

[0080] The beneficial effects of this embodiment are as follows: (1) By using fixed camera position priors, the infrared absolute temperature points are accurately mounted to the visible light ROI coordinate system to construct a static dictionary. The original infrared video stream matrix is ​​destroyed directly during online operation, resulting in an exponential decrease in data volume. At the same time, the static index lookup based on the dictionary avoids all dynamic image registration algorithms. It overcomes the problem of cross-modal image registration in complex scenes from an engineering perspective, breaks through the limitations of traditional fully supervised learning and pixel-level registration fusion, and constructs a cross-modal dictionary to reduce the dimensionality of infrared images and release computing power.

[0081] (2) Three types of generation constraints, A / B / C, based on physical mechanisms are proposed. When synthesizing visible light morphological defects (such as surface corrosion) using a generative model, underlying physical equations (such as Joule's law and the heat conduction equation) are invoked simultaneously, and corresponding simulated temperature rise data are added to the temperature dictionary. The generation mechanism that combines visual morphological features with thermodynamic constraints ensures the logical self-consistency of the synthesized samples in the multimodal physical space, thereby improving the domain adaptability of the multimodal classification model in real-world scenarios.

[0082] (3) A cross-domain transfer learning architecture is adopted. A deep neural network is pre-trained on a general industrial surface defect dataset and the network weights are frozen to serve as a general morphological and semantic feature extractor. Combined with Sigmoid multi-label decoupling, an independent basic physical degradation probability is output. This mechanism achieves decoupling between the feature extraction layer and the specific power equipment ontology. When changing the target substation or monitoring new types of equipment, it is not necessary to re-annotate massive amounts of power images and repeatedly train the visual backbone network.

[0083] (4) Provides white-box diagnostics with underlying logical verification, effectively eliminating single-modal blind spots and false hotspot interference. The decision layer employs Gradient Boosting Tree (GBDT) to process heterogeneous multimodal features, and introduces the SHAP algorithm to calculate the marginal contribution of features. The extracted high-contribution feature subset is cross-referenced with predefined ontological material attributes (such as metal / insulator) in the device space dictionary using predicate logic. By establishing this logical verification rule, it is possible to effectively identify and eliminate infrared false hotspots or single-modal detection blind spots, and output structured diagnostic results containing physical mechanism deductions, meeting the interpretability requirements of industrial field maintenance.

[0084] Secondly, embodiments of the present invention also provide a dual-spectrum fault diagnosis device for power equipment.

[0085] In one embodiment, reference is made to Figure 7 , Figure 7 This is a functional module diagram of an embodiment of the dual-spectrum fault diagnosis device for power equipment of the present invention. The aforementioned dual-spectrum fault diagnosis device for power equipment includes a static coordinate mapping module, an infrared feature extraction module, a visible light feature extraction module, and a decision fusion and diagnosis module.

[0086] The aforementioned static coordinate mapping module is used to construct a static spatial mapping relationship between the visible light image region and the infrared temperature measurement point using prior information from a fixed monitoring location; it is also used to acquire the infrared temperature data and real-time visible light image of the device under test, and based on the aforementioned static spatial mapping relationship, to map the aforementioned infrared temperature data to the visible light coordinate system to generate structured temperature data.

[0087] The aforementioned infrared feature extraction module is used to generate infrared feature vectors based on the structured temperature data described above. The aforementioned visible light feature extraction module is used to extract morphological semantic feature vectors from the aforementioned real-time visible light image using a pre-trained visible light feature extraction model; wherein, the aforementioned visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism.

[0088] The aforementioned decision fusion and diagnosis module is used to input the aforementioned infrared feature vector and the aforementioned morphological semantic feature vector into the classification model for fault identification, and to verify the fault identification results in conjunction with physical rules, and output the diagnosis results.

[0089] Furthermore, the aforementioned dual-spectrum fault diagnosis device for power equipment also includes a virtual fault generation module, which is used to generate virtual samples based on the multi-physics coupling mechanism.

[0090] The device in this embodiment follows strict business logic of static dimensionality reduction, mechanism generation, dual-stream extraction, and fusion reasoning in terms of data flow and module connection relationships.

[0091] The static coordinate mapping module is connected to the virtual fault generation module, the infrared feature extraction module, and the visible light feature extraction module, respectively. As the data dimensionality reduction hub of the device, the static coordinate mapping module establishes a strong spatial mapping dictionary between the visible light region of interest (ROI) and the infrared temperature measurement point under the premise of fixed position, and provides the dimensionality-reduced visible light coordinate-temperature dictionary and visible light reference base map.

[0092] The aforementioned virtual fault generation module serves as an engine for addressing the lack of cold start data. It is connected to the visible light feature extraction module to provide it with the virtual multimodal data (i.e., the tampered temperature dictionary and synthetic image) required for offline pre-training.

[0093] The infrared feature extraction module and the visible light feature extraction module mentioned above operate in parallel and independently, and both are connected to the end-point decision fusion and diagnosis module, responsible for transmitting the decoupled multidimensional feature vector array to it.

[0094] Furthermore, in one embodiment, the static coordinate mapping module described above is used for: Offline acquisition of visible light and infrared reference images at fixed monitoring locations; Based on the equipment ledger structure, the component areas of the key monitoring objects in the aforementioned visible light reference images are given unique names; By establishing the mapping relationship between the aforementioned component regions and the infrared temperature measurement points in the aforementioned infrared reference image through calibration, and combining the naming of the aforementioned component regions, the aforementioned static spatial mapping relationship is constructed.

[0095] Furthermore, in one embodiment, the static coordinate mapping module is also used for: Release the storage space occupied by the original infrared video stream corresponding to the above infrared temperature data.

[0096] Furthermore, in one embodiment, the infrared feature extraction module described above is used for: The structured temperature data above is analyzed to obtain the temperature value and spatial location information corresponding to each visible light image region; based on the thermodynamic conduction law of power equipment, the interphase temperature difference and ambient temperature rise of the region are calculated. The above temperature values, phase-to-phase temperature difference, and ambient temperature rise are fused with the above spatial location information to generate the above infrared feature vector.

[0097] Furthermore, in one embodiment, the training process of the above-mentioned visible light feature extraction model includes: The backbone network of a deep convolutional neural network was pre-trained using a general industrial surface defect dataset as the source domain. After freezing the weights of the backbone network, the classification layer of the deep convolutional neural network is fine-tuned using the virtual samples to obtain the visible light feature extraction model.

[0098] Furthermore, in one embodiment, the virtual fault generation module described above is used for: The virtual fault type is determined based on the multiphysics coupling mechanism; Based on the aforementioned visible light reference image and the aforementioned virtual fault types, a generation strategy is determined; According to the above generation strategy, the visible light defect morphology is rendered using a generative model, and the corresponding virtual temperature data is generated synchronously based on the thermodynamic equation to output virtual samples.

[0099] Furthermore, in one embodiment, the classification model described above is a gradient boosting tree model, and the decision fusion and diagnosis module, as the final decision-making center of the entire architecture, is used for: The infrared feature vectors and the morphological semantic feature vectors are concatenated to form a joint feature matrix; The nonlinear decision boundary of the joint feature matrix is ​​learned through the gradient boosting tree model described above, and the fault identification result is output. The SHAP algorithm is used to calculate the marginal contribution of each feature in the joint feature matrix to the prediction results of the above fault categories, and the dominant feature factor is extracted.

[0100] Furthermore, in one embodiment, the decision fusion and diagnosis module is also used for: Obtain the physical attributes of the equipment in the equipment ledger of the device under test, and combine the above fault identification results and dominant feature factors to match the physical mechanism template through the predicate logic engine; When the above fault identification results conflict with the logical deduction results of the above physical mechanism template, the above fault identification results shall be corrected or marked as pending manual review; the above diagnostic results include physical mechanism deduction information.

[0101] The functions of each module in the above-mentioned dual-spectrum fault diagnosis device for power equipment correspond to the steps in the above-mentioned dual-spectrum fault diagnosis method for power equipment, and their functions and implementation processes will not be described in detail here.

[0102] Thirdly, embodiments of the present invention provide a dual-spectrum fault diagnosis device for power equipment. The dual-spectrum fault diagnosis device for power equipment can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0103] Reference Figure 8 , Figure 8 This is a schematic diagram of the hardware structure of a dual-spectrum fault diagnosis device for power equipment involved in an embodiment of the present invention. In this embodiment, the dual-spectrum fault diagnosis device for power equipment may include a processor, a memory, a communication interface, and a communication bus.

[0104] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0105] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the dual-spectrum fault diagnosis equipment for power equipment, as well as interfaces used for interconnecting the dual-spectrum fault diagnosis equipment for power equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0106] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0107] The processor can be a general-purpose processor, which can call the power equipment dual-spectrum fault diagnosis program stored in the memory and execute the power equipment dual-spectrum fault diagnosis method provided in the embodiments of the present invention. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the power equipment dual-spectrum fault diagnosis program is called can be referred to in various embodiments of the power equipment dual-spectrum fault diagnosis method of the present invention, and will not be described again here.

[0108] Those skilled in the art will understand that Figure 8 The hardware structure shown does not constitute a limitation of the invention and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0109] It should be noted that the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0110] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0111] In the description of the embodiments of the present invention, terms such as "exemplary," "for example," or "for instance" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary," "for example," or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0112] In the description of the embodiments of the present invention, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" means two or more.

[0113] In some processes described in the embodiments of the present invention, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of the present invention, or may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of the present invention.

[0115] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A dual-spectrum fault diagnosis method for power equipment, characterized in that, The method includes: By utilizing prior information from fixed monitoring locations, a static spatial mapping relationship between visible light image regions and infrared temperature measurement points is constructed. The infrared temperature data and real-time visible light image of the device under test are acquired. Based on the static spatial mapping relationship, the infrared temperature data is mapped to the visible light coordinate system to generate structured temperature data, and an infrared feature vector is generated based on the structured temperature data. A pre-trained visible light feature extraction model is used to extract morphological semantic feature vectors from the real-time visible light image; wherein, the visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism; The infrared feature vector and the morphological semantic feature vector are input into the classification model for fault identification, and the fault identification results are verified by combining physical rules to output the diagnostic results.

2. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, The method of constructing a static spatial mapping relationship between visible light image regions and infrared temperature measurement points using prior information from fixed monitoring locations specifically includes: Offline acquisition of visible light and infrared reference images at fixed monitoring locations; Based on the equipment ledger structure, the component regions of key monitoring objects in the visible light reference image are uniquely named; The mapping relationship between the component region and the infrared temperature measurement point in the infrared reference image is established by calibration, and the static spatial mapping relationship is constructed by combining the naming of the component region.

3. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, After mapping the infrared temperature data to the visible light coordinate system to generate structured temperature data, the process also includes: Release the storage space occupied by the original infrared video stream corresponding to the infrared temperature data.

4. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, Generating infrared feature vectors based on the structured temperature data specifically includes: The structured temperature data is analyzed to obtain the temperature value and spatial location information corresponding to each visible light image region; based on the thermodynamic conduction law of power equipment, the interphase temperature difference and ambient temperature rise are calculated. The temperature value, phase-to-phase temperature difference, and ambient temperature rise are fused with the spatial location information to generate the infrared feature vector.

5. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, The training process of the visible light feature extraction model includes: The backbone network of a deep convolutional neural network was pre-trained using a general industrial surface defect dataset as the source domain. After freezing the weights of the backbone network, the classification layer of the deep convolutional neural network is fine-tuned using the virtual samples to obtain the visible light feature extraction model.

6. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, The method also includes generating virtual samples based on multiphysics coupling mechanisms, specifically including: The virtual fault type is determined based on the multiphysics coupling mechanism; The generation strategy is determined based on the visible light reference image and the virtual fault type; According to the generation strategy, the visible light defect morphology is rendered using a generative model, and the corresponding virtual temperature data is generated synchronously based on the thermodynamic equation to output virtual samples.

7. The dual-spectrum fault diagnosis method for power equipment as described in claim 1, characterized in that, The classification model is a gradient boosting tree model. The infrared feature vector and the morphological semantic feature vector are input into the classification model for fault identification, specifically including: The infrared feature vector and the morphological semantic feature vector are concatenated to form a joint feature matrix; The nonlinear decision boundary of the joint feature matrix is ​​learned through the gradient boosting tree model, and the fault identification result is output. The marginal contribution of each feature in the joint feature matrix to the fault category prediction result is calculated using the SHAP algorithm, and the dominant feature factor is extracted.

8. The dual-spectrum fault diagnosis method for power equipment as described in claim 7, characterized in that, The verification of fault identification results by combining physical rules specifically includes: Obtain the physical attributes of the device in the device ledger of the device under test, and combine the fault identification results and dominant feature factors to match the physical mechanism template through the predicate logic engine; When the fault identification result conflicts with the logical deduction result of the physical mechanism template, the fault identification result is corrected or marked as pending manual review; the diagnostic result includes physical mechanism deduction information.

9. A dual-spectrum fault diagnosis device for power equipment, characterized in that, The device includes: The static coordinate mapping module is used to construct a static spatial mapping relationship between the visible light image area and the infrared temperature measurement point using prior information from a fixed monitoring location; it is also used to acquire the infrared temperature data and real-time visible light image of the device under test, and to map the infrared temperature data to the visible light coordinate system based on the static spatial mapping relationship to generate structured temperature data. An infrared feature extraction module is used to generate infrared feature vectors based on the structured temperature data; The visible light feature extraction module is used to extract morphological semantic feature vectors from the real-time visible light image using a pre-trained visible light feature extraction model; wherein the visible light feature extraction model is trained based on virtual samples generated by a multi-physics coupling mechanism. The decision fusion and diagnosis module is used to input the infrared feature vector and the morphological semantic feature vector into the classification model for fault identification, and to verify the fault identification results in combination with physical rules, and output the diagnosis results.

10. A dual-spectrum fault diagnosis device for power equipment, characterized in that, The device includes a processor, a memory, and a power equipment dual-spectrum fault diagnosis program stored in the memory and executable by the processor, wherein when the power equipment dual-spectrum fault diagnosis program is executed by the processor, it implements the steps of the power equipment dual-spectrum fault diagnosis method as described in any one of claims 1 to 8.

Images