Fault diagnosis method and device of transformer, terminal equipment and storage medium

Abstract

The invention discloses a transformer fault diagnosis method and device, terminal equipment and a storage medium, and belongs to the technical field of transformer fault diagnos.The method comprises the steps that image feature extraction is conducted on image data of a transformer to generate element image features; the method comprises the following steps: performing data conversion mapping on analysis data of gas dissolved in oil to generate two-dimensional image features, performing semantic alignment and structured processing on historical operation and maintenance data to generate text features, and further performing feature fusion on element image features, the two-dimensional image features and the text features to generate a multi-modal fusion feature vector; according to the invention, fault diagnosis is carried out on the transformer through the fault diagnosis model according to the multi-modal fusion feature vector, and then the large language model is adopted to output the fault diagnosis report of the transformer in the natural language format, so that the fault diagnosis efficiency and accuracy of the transformer are improved.

Description

Technical Field

[0001] This invention relates to the field of transformer fault diagnosis technology, and in particular to a method, apparatus, terminal equipment and storage medium for transformer fault diagnosis. Background Technology

[0002] In the field of power equipment operation and maintenance, the reliable operation of transformers is crucial to the stability and safety of the entire power system. However, the existing technologies currently available have revealed many limitations in transformer fault diagnosis.

[0003] Traditional fault diagnosis methods suffer from serious shortcomings in data utilization. Most rely solely on single-modal data analysis, such as using DGA (Dissolved Gas Analysis) data as the core, inferring fault types by analyzing the composition and content of dissolved gases in transformer oil. However, this approach fails to comprehensively reflect the complex operating conditions of transformers. Focusing only on oil gases misses fault information such as winding deformation and multi-point grounding of the core, which cannot be directly reflected by gas changes. This over-reliance on single data points prevents a comprehensive and accurate understanding of the transformer's true condition.

[0004] Secondly, existing fault diagnosis systems suffer from poor human-machine collaboration. Maintenance personnel need specialized system operation and maintenance knowledge to input commands and query data through complex interfaces, then analyze and extract transformer fault information from complex and massive datasets. This not only increases the workload and learning curve for maintenance personnel but also makes them susceptible to human error, leading to inaccurate diagnoses and severely limiting maintenance efficiency and accuracy. Therefore, improving the efficiency and accuracy of transformer fault diagnosis has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a method, apparatus, terminal equipment, and storage medium for diagnosing transformer faults. The method can improve the efficiency and accuracy of transformer fault diagnosis.

[0006] An embodiment of the present invention provides a method for diagnosing transformer faults, comprising: Acquire image data of the transformer, dissolved gas analysis data in the oil, and historical operation and maintenance data; Image features are extracted from the transformer components in the image data to generate component image features. Data mapping and transformation are performed on each dissolved gas component data in the dissolved gas analysis data in the oil to generate corresponding pixel values. Two-dimensional image features are generated based on the pixel values. Semantic alignment and structuring processing are performed on the historical operation and maintenance data to generate text features. The component image features, the two-dimensional image features, and the text features are fused to generate a multimodal fusion feature vector; The multimodal fusion feature vector is input into a preset fault diagnosis model, so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, and performs fault area localization and fault type analysis based on preset technical knowledge, outputting transformer fault location and transformer fault type. Using a pre-defined language model, a fault diagnosis report for the transformer is output in natural language format based on the location and type of the transformer fault.

[0007] Furthermore, the image data includes: infrared images and visible light images; Image features are extracted from the transformer components in the image data to generate component image features, including: Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding bounding boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. The infrared image and the visible light image to be identified are divided into grids, and several target grids containing several bounding boxes are identified. A first grid position code is generated based on several target grids of the infrared image to be identified, and a second grid position code is generated based on several target grids of the visible light image to be identified; Visual features of several components of the transformer in the infrared image and the visible light image are extracted to generate visual features of each component; Based on the first grid position code, the second grid position code, the infrared image to be identified, the visible light image to be identified, and the visual features, the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified are determined. Based on the first image coordinates and the second image coordinates, a coordinate sequence of each element is generated, and the coordinate sequence of several elements is used as the element image feature.

[0008] Furthermore, the step of performing data mapping and transformation on each dissolved gas component data in the dissolved gas analysis data of the oil to generate corresponding pixel values, and generating two-dimensional image features based on the pixel values, includes: The dissolved gas component data in the dissolved gas analysis data of the oil are normalized to generate normalized gas component data, and the polar angle and polar radius of each normalized gas component data are determined. Using Gram angle summation field and Gram angle difference field, based on the polar angle and polar radius of each normalized gas component data, the Gram angle summation matrix and Gram angle difference matrix of the dissolved gas analysis data in the oil are generated; The pixel values ​​of each normalized gas component data are formed based on the Gram angle summation matrix and the Gram angle difference matrix, and the two-dimensional image features are generated based on the pixel values.

[0009] Furthermore, the step of semantically aligning and structuring the historical operation and maintenance data to generate text features includes: The historical operation and maintenance data is cleaned to generate operation and maintenance data to be processed; Based on preset keywords, target operation and maintenance data is extracted from the operation and maintenance data to be processed; The target operation and maintenance data is semantically aligned and structured to generate the text features.

[0010] Furthermore, the step of fusing the element image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector includes: The component image features and the two-dimensional image features are input into the image encoder of the preset feature fusion model, so that the image encoder adopts a multi-layer attention mechanism and nonlinear transformation to generate a first image embedding vector in a preset dimension based on the component image features, and a second image embedding vector in a preset dimension based on the two-dimensional image features. The text features are input into the text encoder of the feature fusion model so that the text encoder generates text embedding vectors of a preset dimension based on a self-attention mechanism. The first image embedding vector, the second image embedding vector, and the text embedding vector are concatenated to generate a concatenated feature vector. The concatenated feature vector is input into a linear connection layer, so that the linear connection layer maps the concatenated feature vector into the multimodal fusion feature vector through matrix multiplication and bias adjustment.

[0011] Furthermore, the method employs a preset language model to output a transformer fault diagnosis report in natural language format based on the transformer fault location and the transformer fault type, including: Using the aforementioned language model, the transformer fault type and the transformer fault location are converted into a standardized text format to generate an initial fault prompt text. Using the initial fault message text as a clue, several target maintenance solutions are matched among several maintenance solutions pre-stored in a preset industrial database. Using the aforementioned language model, the initial fault prompt text and several target repair solutions are converted into a fault diagnosis report and fault repair strategy in natural language format.

[0012] An embodiment of the present invention also provides a transformer fault diagnosis device, comprising: The fault data acquisition module is used to acquire transformer image data, dissolved gas analysis data in oil, and historical operation and maintenance data; The fault feature extraction module is used to extract image features from the transformer components in the image data, generate component image features, perform data mapping and transformation based on each dissolved gas component data in the dissolved gas analysis data in the oil, generate corresponding pixel values, generate two-dimensional image features based on the pixel values, and perform semantic alignment and structured processing on the historical operation and maintenance data to generate text features. The fault feature fusion module is used to fuse the component image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector. The transformer fault prediction module is used to input the multimodal fusion feature vector into a preset fault diagnosis model, so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, performs fault area localization and fault type analysis based on preset technical knowledge, and outputs the transformer fault location and transformer fault type. The transformer fault diagnosis module is used to output a fault diagnosis report of the transformer in natural language format based on the location and type of the transformer fault, using a preset language model.

[0013] Furthermore, the image data includes: infrared images and visible light images; The fault feature extraction module extracts image features from the transformer components in the image data to generate component image features, including: Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding bounding boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. The infrared image and the visible light image to be identified are divided into grids, and several target grids containing several bounding boxes are identified. A first grid position code is generated based on several target grids of the infrared image to be identified, and a second grid position code is generated based on several target grids of the visible light image to be identified; Visual features of several components of the transformer in the infrared image and the visible light image are extracted to generate visual features of each component; Based on the first grid position code, the second grid position code, the infrared image to be identified, the visible light image to be identified, and the visual features, the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified are determined. Based on the first image coordinates and the second image coordinates, a coordinate sequence of each element is generated, and the coordinate sequence of several elements is used as the element image feature.

[0014] This application also provides a terminal device, including: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a transformer fault diagnosis method as described in the above embodiments of the invention.

[0015] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a transformer fault diagnosis method as described in the above embodiments.

[0016] The following benefits can be obtained by implementing the present invention: This invention provides a method, apparatus, terminal device, and storage medium for transformer fault diagnosis. The method involves extracting image features from transformer image data to generate component image features, performing data conversion and mapping on dissolved gas analysis data in oil to generate two-dimensional image features, and performing semantic alignment and structured processing on historical operation and maintenance data to generate text features. Then, the component image features, the two-dimensional image features, and the text features are fused to generate a multimodal fusion feature vector. Therefore, this invention effectively overcomes the inaccuracies caused by traditional fault diagnosis methods that use single-modal data for fault diagnosis by extracting and fusing features from multimodal transformer data. Furthermore, the fault diagnosis model diagnoses transformer faults based on the multimodal fusion feature vector, and then uses a large language model to convert the transformer fault location and fault type output by the fault diagnosis model into a fault diagnosis report in natural language format. This avoids human errors caused by complex system operations and data analysis, improving the efficiency and accuracy of transformer fault diagnosis. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a transformer fault diagnosis method provided in a certain embodiment of this application; Figure 2 This is a schematic diagram of the structure of a transformer fault diagnosis device provided in a certain embodiment of this application; Figure 3 This is a schematic diagram of the structure of a terminal device provided in a certain embodiment of this application; Figure 4 This is a schematic diagram of an embodiment of the present application for extracting image features of an element; Figure 5 This is a schematic diagram illustrating the data mapping and transformation of dissolved gas analysis data in oil according to a certain embodiment of this application; Figure 6 This is a schematic diagram of the structure of an asset management shell provided in a certain embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0021] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0024] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0025] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0026] See Figure 1 To improve the efficiency and accuracy of transformer fault diagnosis, an embodiment of the present invention provides a transformer fault diagnosis method, comprising: S1. Acquire image data of the transformer, dissolved gas analysis data in the oil, and historical operation and maintenance data; In a preferred embodiment of the present invention, infrared thermal imagers, DGA online monitoring devices, and visible light cameras installed in the transformer's working environment are used to acquire infrared images, DGA (Dissolved Gas Analysis) data, and visible light images of the transformer, while historical operation and maintenance data are collected from a database used to store transformer operation and maintenance records.

[0027] S2. Extract image features from the transformer components in the image data to generate component image features. Perform data mapping and transformation on each dissolved gas component data in the dissolved gas analysis data in the oil to generate corresponding pixel values. Generate two-dimensional image features based on the pixel values. Perform semantic alignment and structured processing on the historical operation and maintenance data to generate text features. In a preferred embodiment of the present invention, a multimodal gating adapter is provided to extract features from multimodal image data, dissolved gas analysis data in oil, and historical operation and maintenance data, so that they can be spliced ​​and fused in the subsequent process.

[0028] Preferably, the image data includes: infrared images and visible light images; Image features are extracted from the transformer components in the image data to generate component image features, including: S211. Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding annotation boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. S212. Divide the infrared image to be identified and the visible light image to be identified into grids respectively, and identify several target grids in which several bounding boxes are located; S213. Generate a first grid position code based on a plurality of target grids in the infrared image to be identified, and generate a second grid position code based on a plurality of target grids in the visible light image to be identified; S214. Visual features are extracted from several components of the transformer in the infrared image and the visible light image to generate visual features for each component. S215. Based on the first grid position code, the second grid position code, the infrared image to be identified, the visible light image to be identified, and the visual features, determine the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified. S216. Based on the first image coordinates and the second image coordinates, generate a coordinate sequence for each of the elements, and use the coordinate sequences of several elements as the image features of the elements.

[0029] In a preferred embodiment of the present invention, such as Figure 4 As shown, both the acquired infrared and visible light images need to be manually or automatically labeled. Label boxes are added to key areas of the equipment (such as sleeves, oil tanks, radiators, etc.) to form infrared and visible light images to be identified, providing a basis for subsequent location information processing.

[0030] Furthermore, the infrared image and the visible light image to be identified are processed by an adaptive grid position information processor; In the encoder part of the adaptive grid position information processor, the infrared image and the visible light image to be identified are divided into adaptive grids. Each grid is encoded and marked. The intersection of the bounding box and the grid is identified to filter out the target grid where the bounding box is located. These target grids are arranged in order from left to right and from top to bottom to form a grid position code that represents the location of the bounding box. The decoder section inputs the grid position codes output by the encoder, the infrared image to be identified, and the visible light image to be identified into the position prediction model. The position prediction model is an LLAVA (Large Language and Vision Assistant) model. After model inference, it outputs the predicted first and second image coordinates of each element. The LLAVA model can be fine-tuned through pre-training and labeled data, allowing the model to learn domain-specific knowledge and transfer its general inference capabilities to coordinate prediction. Specifically, based on pre-defined encoding rules, it identifies the corresponding grid positions in the grid position codes. By calculating the coordinates of the center point of each identified grid, and using statistical methods to analyze and process a large number of grid center point coordinates, the coordinates of the labeled area in the image are finally approximated. In practical applications, the grid division granularity can be flexibly adjusted according to specific needs and computing resources, ensuring coordinate data accuracy while reasonably controlling the computational load.

[0031] Preferably, the step of performing data mapping and transformation on each dissolved gas component data in the dissolved gas analysis data of the oil to generate corresponding pixel values, and generating two-dimensional image features based on the pixel values, includes: S221. Normalize the data of each dissolved gas component in the dissolved gas analysis data of the oil to generate normalized gas component data, and determine the polar angle and polar radius of each normalized gas component data. S222. Using the Gram angle summation field and the Gram angle difference field, the Gram angle summation matrix and the Gram angle difference matrix of the dissolved gas analysis data in the oil are generated according to the polar angle and polar radius of each normalized gas component data. S223. Based on the Gram angle summation matrix and the Gram angle difference matrix, pixel values ​​of each normalized gas component data are formed, and the two-dimensional image features are generated based on the pixel values.

[0032] In a preferred embodiment of the present invention, such as Figure 5 As shown in (a) and (b), the original one-dimensional DGA data (dissolved gas analysis data in oil) is first normalized to map the data to a specific interval for subsequent calculations. Specifically, the data for each dissolved gas component is normalized using a formula to obtain normalized data. Then, based on the principle of polar coordinates, the polar angle and polar radius of each normalized data point are calculated. Figure 5 As shown in (c) and (d), the Gram angle summation field (GASF) and as... Figure 5 As shown in (f) and (g), the calculation method of Gram angle difference field (GADF) involves operations on the polar angle and polar radius to generate the corresponding GASF matrix and GADF matrix. Finally, as... Figure 5As shown in (e) and (h), these matrices are processed and mapped to the pixel value range of [0, 255] to form pixel values ​​of three channels, thereby constructing a two-dimensional feature image, and using some pixel values ​​as two-dimensional image features. GAF, a data dimension transformation method based on polar coordinate Gram matrices, first normalizes the original DGA data, and then converts the normalized gas component data into pixel values ​​using Gramian Angular Summation Field (GASF) and Gramian Angular Difference Field (GADF) calculation methods. Specifically: ; ; ; ; ; ; ; in, It includes A one-dimensional array of variables representing dissolved gas analysis data in oil; This represents the data of the i-th dissolved gas component in the dissolved gas analysis data of oil; This represents the normalized gas component data for the i-th component. This represents an array composed of normalized gas component data; This represents the polar angle of the i-th normalized gas component data. Represents the polar radius of the i-th normalized gas component data; Represents the Gram angle summation matrix. Represents the Gram angle difference matrix; This represents the polar angle of the j-th normalized gas component data; This represents the pixel values ​​of the three channels that map the normalized gas component data to the pixel value range of [0, 255]. Represents the Gram angle summation matrix and Gram angle difference matrix , As row and column indices of the corresponding matrix, they are used to identify the matrix being constructed (e.g., ...). , The specific position of each element in this type of matrix; Indicates the participants in the calculation Correlation matrix (e.g.) , The maximum value of all elements in this type of matrix; Indicates the participants in the calculation Correlation matrix (e.g.) , The minimum value of all elements in this type of matrix.

[0033] Preferably, the step of semantically aligning and structuring the historical operation and maintenance data to generate text features includes: S231. Perform data cleaning on the historical operation and maintenance data to generate operation and maintenance data to be processed; S232. Extract target maintenance data from the maintenance data to be processed according to preset keywords; S233. Perform semantic alignment and structuring processing on the target operation and maintenance data to generate the text features.

[0034] In a preferred embodiment of the present invention, historical operation and maintenance data is cleaned to remove duplicate and erroneous data. Unstructured text is structured using natural language processing technology. Based on preset keywords, key information, such as equipment operating parameters, fault records, and maintenance time, is extracted to generate target operation and maintenance data. In terms of semantic alignment, the target operation and maintenance data is associated and mapped with the semantic concepts of professional terms in the transformer domain knowledge graph. By calculating semantic similarity, accurate alignment at the semantic level is achieved, and text features are generated.

[0035] S3. Perform feature fusion on the component image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector; Preferably, the step of fusing the element image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector includes: S31. Input the component image features and the two-dimensional image features into the image encoder of the preset feature fusion model, so that the image encoder adopts a multi-layer attention mechanism and nonlinear transformation to generate a first image embedding vector in a preset dimension based on the component image features, and a second image embedding vector in a preset dimension based on the two-dimensional image features. S32. Input the text features into the text encoder of the feature fusion model so that the text encoder generates a text embedding vector of a preset dimension based on the self-attention mechanism. S33. Concatenate the first image embedding vector, the second image embedding vector, and the text embedding vector to generate a concatenated feature vector; S34. Input the concatenated feature vector into the linear connection layer, so that the linear connection layer can map the concatenated feature vector into the multimodal fusion feature vector through matrix multiplication and bias adjustment.

[0036] In a preferred embodiment of the present invention, a CLIP (Contrastive Language-Image Pretraining) model is used as the feature fusion model to achieve cross-modal alignment of the component image features, two-dimensional image features, and text features. The core innovation of the CLIP model lies in constructing a visual-text semantic association through a "contrastive learning" mechanism: during the pretraining stage, the model learns the ability to judge "matching / not matching" through massive image-text pairs—for input image-text pairs, if the two are semantically related (e.g., "transformer winding overheating image" and "winding temperature abnormality" text), their distance in the feature space is shortened; if they are unrelated, their distance is widened. This pretraining method enables the model to acquire general cross-modal semantic understanding capabilities without redesigning feature mapping rules for specific tasks. Specifically, the feature fusion model consists of two core encoders that achieve cross-modal alignment through contrastive learning. One is the image encoder: which encodes the input image features into fixed-dimensional embedding vectors. It supports multiple architectures, such as ResNet (convolutional neural network) or ViT (visual Transformer). For the component image features (such as the appearance image features of transformer components) and two-dimensional image features (such as pseudo-image features converted from DGA data) in this invention, the image encoder extracts hierarchical features through multi-layer convolution / self-attention operations—the bottom layer captures basic visual patterns such as edges and textures, and the top layer fuses to form abstract semantic features such as "overheated area" and "component deformation," and finally maps them to a fixed-dimensional embedding vector. Secondly, the text encoder encodes the input text features into a vector with the same dimension as the image embedding. Based on the Transformer architecture, the text is first segmented and positionally encoded, and then semantic features are extracted through a self-attention mechanism to output the text embedding vector. For technical terms in the text features (such as "abnormal noise from tap changer" and "excessive dielectric loss"), the text encoder preserves the integrity of technical terms through subword tokenization and uses self-attention weights to focus on key fault descriptions, enhancing the expression of semantic features related to equipment status.

[0037] Furthermore, the three embedding vectors output by CLIP are concatenated dimensionally to form a concatenated feature vector, the length of which is the sum of the dimensions of the three embedding vectors. In this process, to avoid weight imbalance of different modal features, dynamic weight allocation is first performed on each embedding vector—the importance coefficients of each modality are learned through a small fully connected network (e.g., when DGA data is abnormally significant, the weight of 2D image feature embedding is increased), and then weighted concatenation is performed.

[0038] The concatenated feature vector is input into a linear connected layer. This layer undergoes L2 normalization to standardize the mapped vector to a unit sphere, ensuring consistency in numerical scale across different modal features and providing a unified feature foundation for subsequent cross-modal attention computation in the fault diagnosis model. This step preserves key information from each modality while eliminating redundancy, ultimately outputting a multimodal fusion feature vector.

[0039] S4. Input the multimodal fusion feature vector into a preset fault diagnosis model so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, performs fault area localization and fault type analysis based on preset technical knowledge, and outputs transformer fault location and transformer fault type. In a preferred embodiment of the present invention, the fault diagnosis model is an LLAVA model. LLAVA (Large Language and Vision Assistant) is a multimodal large model trained end-to-end. Its core architecture includes a visual encoder, a language model (such as Vicuna), and a projection layer connecting the two: the visual encoder (such as ViT in CLIP) is used to perform deep encoding on the image-type features (component image features, two-dimensional image features) in the input multimodal fusion feature vector, transforming visual information into a high-dimensional feature vector; the projection layer maps the features output by the visual encoder to the semantic space of the language model through linear transformation, realizing the dimensional alignment of visual and text features; the language model processes text features and aligned visual features based on an autoregressive mechanism, and establishes cross-modal associations through a cross-attention module.

[0040] After receiving the multimodal fusion feature vector, the fault diagnosis model outputs a predicted encoding sequence to characterize the fault location and transformer fault type, and simultaneously feeds the encoding sequence back to the location prediction model. This step serves two purposes: first, to perform closed-loop calibration feature processing, using the model's inferred location encoding results to reverse-verify the encoding logic in the location prediction model (e.g., comparing the deviation between the model's predicted location and the original image's labeled area), dynamically adjusting the grid division granularity and encoding rules, and optimizing the accuracy of coordinate processing in the image's labeled area; second, to enhance modal collaboration, the model's inference feedback on "location features" is passed to the GAF converter and text feature processing module to assist in adjusting the DGA data conversion strategy. Based on the model's inference of the equipment fault location, the focus is on associating the fault gas features of the corresponding area in the DGA data, and the text feature extraction direction prioritizes capturing historical maintenance records related to the model's focus location, thereby strengthening the collaborative association of multimodal data.

[0041] Specifically, in this embodiment, the fault diagnosis model uses self-attention or cross-attention mechanisms to uncover dependencies between features of different modalities. For example, when dealing with transformer faults, the model can dynamically focus on the correlation between "visual features of overheating in the winding region" and "DGA features of increased methane concentration" through attention weights. Specifically, after receiving the multimodal fusion feature vector, the LLAVA model first parses it into visual feature parts (from component image features and 2D image features) and text feature parts (from historical operation and maintenance data). The visual encoder processes the visual feature parts and maps them to the input space of the language model through a projection layer, where they, along with the text feature parts, serve as input to the language model. The language model captures the semantic relationships within text features through self-attention mechanisms in a multi-layered Transformer structure and captures the mapping relationship between visual features and text features through cross-attention mechanisms, thereby achieving cross-modal correlation reasoning of "visual features - text description - fault knowledge".

[0042] Furthermore, the fault diagnosis model achieves knowledge transfer through general pre-training and domain-specific fine-tuning. Specifically, it first pre-trains on large-scale internet image-text pairs (such as COCO and LAION) to learn general visual-language associations (e.g., the semantic association between "flame" and "high temperature"), forming a basic cross-modal understanding capability. For example, the model can recognize the visual pattern of "overheated area" and associate it with the text description of "temperature rise." Then, in the transformer fault diagnosis scenario, through image-text pairs labeled with data such as fault location and type, fine-tuning is performed to enable the model to learn specific knowledge in the transformer technology field. This allows the model to transfer general reasoning capabilities to the transformer domain, such as inferring "winding short-circuit fault" from the combined features of "winding overheating + acetylene concentration increase." Combining visual and text features, it simulates the "chain of evidence" reasoning process of human experts. For example, when "abnormal temperature in the bushing area (visual), sudden increase in hydrogen concentration (DGA), and text record 'bushing insulation breakdown will produce hydrogen'" occur simultaneously, the model can comprehensively judge it as "bushing insulation breakdown fault."

[0043] S5. Using a preset language model, output a fault diagnosis report of the transformer in natural language format based on the location and type of the transformer fault.

[0044] Preferably, the step of using a preset language model to output a transformer fault diagnosis report in natural language format based on the transformer fault location and the transformer fault type includes: S51. Using the language model, the transformer fault type and the transformer fault location are converted into a standardized text format to generate an initial fault prompt text. S52. Using the initial fault prompt text as a reasoning clue, match several target maintenance solutions among several maintenance solutions pre-stored in the preset industrial database. S53. Using the aforementioned language model, the initial fault prompt text and several target maintenance solutions are converted into a fault diagnosis report and fault maintenance strategy in natural language format.

[0045] In a preferred embodiment of the present invention, such as Figure 6As shown, this embodiment employs an Asset Management Shell (AAS), which constructs a digital "shell" for various resources involved in multimodal reasoning and human-computer dialogue. Its functions, interfaces, and data resources are encapsulated into independent service units, with each service unit's attributes, functions, content, and access methods described in detail. Simultaneously, interfaces of different enterprise information systems, different database entities, and specific functional program instances are also encapsulated and managed. The Industrial Internet Identifier Resolution System assigns a unique identity code to each service unit and resource, serving as the path for resource access. When a user requests access to a resource, the identity resolution system verifies the visitor's identity and security, retrieves relevant information from the corresponding asset shell based on the request content, and performs precise data and service location and allocation, achieving unified management and efficient sharing of multimodal resources.

[0046] The central connector serves as the "scheduling hub" for multimodal reasoning. Upon receiving the transformer fault type (such as coordinates of abnormal temperature areas and fault types corresponding to DGA data) and the transformer fault location (such as component wear locations and historical maintenance difficulties of similar faults) output by the fault diagnosis model, it calls a large language model (such as ChatGPT-4) to convert them into standardized Prompt text, generating an initial fault prompt text that incorporates transformer domain knowledge and fault diagnosis language templates. Then, it retrieves historical operation and maintenance plans for similar fault cases and maintenance schemes from the industrial database, performs "maintenance decision path" deduction on the abnormal characteristics of the historical correspondence between DGA gas component anomalies and insulation faults, simulates the impact of different maintenance schemes on equipment operation, and outputs the optimal power outage maintenance window and spare parts replacement suggestions.

[0047] The central connector, through the Industrial Internet Identifier Resolution System, calls the "Transformer Operation and Maintenance Database" service unit, which includes equipment ledgers, fault databases, and maintenance records. It compares the initial fault prompt text with the "Fault Feature Database" in the database (such as querying the same abnormal temperature pattern in history, fault types of DGA components) to help ChatGPT quickly identify the anomaly category, such as determining whether it is winding overheating or tap changer failure. The maintenance decision-making process is responsible for retrieving "Equipment Same Part Maintenance Records" (corresponding to the fault location coordinates output by the decoder) and "Spare Part Replacement Cycle Data" from the database, and outputting personalized maintenance strategies, including recommended maintenance processes and estimated maintenance time.

[0048] In summary, one embodiment of the present invention provides a transformer fault diagnosis method. This method involves extracting image features from transformer image data to generate component image features, performing data conversion and mapping on dissolved gas analysis data in oil to generate two-dimensional image features, and semantically aligning and structuring historical maintenance data to generate text features. Furthermore, the component image features, the two-dimensional image features, and the text features are fused to generate a multimodal fusion feature vector. Therefore, by extracting and fusing features from multimodal transformer data, the present invention effectively overcomes the inaccuracies caused by traditional fault diagnosis methods that use single-modal data. Further, the fault diagnosis model diagnoses the transformer fault based on the multimodal fusion feature vector, and then uses a large language model to convert the transformer fault location and fault type output by the fault diagnosis model into a fault diagnosis report in natural language format. This avoids human error caused by complex system operations and data analysis, improving the efficiency and accuracy of transformer fault diagnosis.

[0049] See Figure 2 This is a fault diagnosis device for a transformer provided in one embodiment of the present invention, comprising: The fault data acquisition module is used to acquire transformer image data, dissolved gas analysis data in oil, and historical operation and maintenance data; The fault feature extraction module is used to extract image features from the transformer components in the image data, generate component image features, perform data mapping and transformation based on each dissolved gas component data in the dissolved gas analysis data in the oil, generate corresponding pixel values, generate two-dimensional image features based on the pixel values, and perform semantic alignment and structured processing on the historical operation and maintenance data to generate text features. The fault feature fusion module is used to fuse the component image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector. The transformer fault prediction module is used to input the multimodal fusion feature vector into a preset fault diagnosis model, so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, performs fault area localization and fault type analysis based on preset technical knowledge, and outputs the transformer fault location and transformer fault type. The transformer fault diagnosis module is used to output a fault diagnosis report of the transformer in natural language format based on the location and type of the transformer fault, using a preset language model.

[0050] Furthermore, the image data includes: infrared images and visible light images; The fault feature extraction module extracts image features from the transformer components in the image data to generate component image features, including: Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding bounding boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. The infrared image and the visible light image to be identified are divided into grids, and several target grids containing several bounding boxes are identified. A first grid position code is generated based on several target grids of the infrared image to be identified, and a second grid position code is generated based on several target grids of the visible light image to be identified; Visual features of several components of the transformer in the infrared image and the visible light image are extracted to generate visual features of each component; Based on the first grid position code, the second grid position code, the infrared image to be identified, the visible light image to be identified, and the visual features, the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified are determined. Based on the first image coordinates and the second image coordinates, a coordinate sequence of each element is generated, and the coordinate sequence of several elements is used as the element image feature.

[0051] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0052] See Figure 3 One embodiment of this application also provides a terminal device, including: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a transformer fault diagnosis method as described above.

[0053] The processor controls the overall operation of the terminal device to complete all or part of the steps of the aforementioned transformer fault diagnosis method. The memory stores various types of data to support the operation of the terminal device. This data may include, for example, instructions for any application or method operating on the terminal device, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0054] In an exemplary embodiment, the terminal device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform a transformer fault diagnosis method as described in any of the foregoing embodiments and achieve the same technical effects as the methods described above.

[0055] In another exemplary embodiment, a computer-readable storage medium including a computer program is also provided. When executed by a processor, the computer program implements the steps of a transformer fault diagnosis method as described in any of the foregoing embodiments. For example, the computer-readable storage medium may be the aforementioned memory including the computer program, which may be executed by a processor of a terminal device to complete the transformer fault diagnosis method as described in any of the foregoing embodiments and achieve the same technical effects as the aforementioned method.

[0056] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for diagnosing transformer faults, characterized in that, include: Acquire image data of the transformer, dissolved gas analysis data in the oil, and historical operation and maintenance data; Image features are extracted from the transformer components in the image data to generate component image features. Data mapping and transformation are performed on each dissolved gas component data in the dissolved gas analysis data in the oil to generate corresponding pixel values. Two-dimensional image features are generated based on the pixel values. Semantic alignment and structuring processing are performed on the historical operation and maintenance data to generate text features. The component image features, the two-dimensional image features, and the text features are fused to generate a multimodal fusion feature vector; The multimodal fusion feature vector is input into a preset fault diagnosis model, so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, and performs fault area localization and fault type analysis based on preset technical knowledge, outputting transformer fault location and transformer fault type. Using a pre-defined language model, a fault diagnosis report for the transformer is output in natural language format based on the location and type of the transformer fault.

2. The transformer fault diagnosis method as described in claim 1, characterized in that, The image data includes: infrared images and visible light images; Image features are extracted from the transformer components in the image data to generate component image features, including: Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding bounding boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. The infrared image and the visible light image to be identified are divided into grids, and several target grids containing several bounding boxes are identified. A first grid position code is generated based on several target grids of the infrared image to be identified, and a second grid position code is generated based on several target grids of the visible light image to be identified; Visual features of several components of the transformer in the infrared image and the visible light image are extracted to generate visual features of each component; Based on the first grid position code, the second grid position code, the infrared image to be identified, the visible light image to be identified, and the visual features, the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified are determined. Based on the first image coordinates and the second image coordinates, a coordinate sequence of each element is generated, and the coordinate sequence of several elements is used as the element image feature.

3. The transformer fault diagnosis method as described in claim 2, characterized in that, The step of performing data mapping and transformation on each dissolved gas component data in the dissolved gas analysis data of the oil to generate corresponding pixel values, and generating two-dimensional image features based on the pixel values, includes: The dissolved gas component data in the dissolved gas analysis data of the oil are normalized to generate normalized gas component data, and the polar angle and polar radius of each normalized gas component data are determined. Using Gram angle summation field and Gram angle difference field, based on the polar angle and polar radius of each normalized gas component data, the Gram angle summation matrix and Gram angle difference matrix of the dissolved gas analysis data in the oil are generated; The pixel values ​​of each normalized gas component data are formed based on the Gram angle summation matrix and the Gram angle difference matrix, and the two-dimensional image features are generated based on the pixel values.

4. The transformer fault diagnosis method as described in claim 3, characterized in that, The process of semantic alignment and structuring of the historical operation and maintenance data to generate text features includes: The historical operation and maintenance data is cleaned to generate operation and maintenance data to be processed; Based on preset keywords, target operation and maintenance data is extracted from the operation and maintenance data to be processed; The target operation and maintenance data is semantically aligned and structured to generate the text features.

5. The transformer fault diagnosis method as described in claim 4, characterized in that, The step of fusing the element image features, the two-dimensional image features, and the text features to generate a multimodal fused feature vector includes: The component image features and the two-dimensional image features are input into the image encoder of the preset feature fusion model, so that the image encoder adopts a multi-layer attention mechanism and nonlinear transformation to generate a first image embedding vector in a preset dimension based on the component image features, and a second image embedding vector in a preset dimension based on the two-dimensional image features. The text features are input into the text encoder of the feature fusion model so that the text encoder generates text embedding vectors of a preset dimension based on a self-attention mechanism. The first image embedding vector, the second image embedding vector, and the text embedding vector are concatenated to generate a concatenated feature vector. The concatenated feature vector is input into a linear connection layer, so that the linear connection layer maps the concatenated feature vector into the multimodal fusion feature vector through matrix multiplication and bias adjustment.

6. The transformer fault diagnosis method as described in claim 5, characterized in that, The method employs a preset language model to output a transformer fault diagnosis report in natural language format based on the transformer fault location and fault type, including: Using the aforementioned language model, the transformer fault type and the transformer fault location are converted into a standardized text format to generate an initial fault prompt text. Using the initial fault message text as a clue, several target maintenance solutions are matched among several maintenance solutions pre-stored in a preset industrial database. Using the aforementioned language model, the initial fault prompt text and several target repair solutions are converted into a fault diagnosis report and fault repair strategy in natural language format.

7. A fault diagnosis device for a transformer, characterized in that, include: The fault data acquisition module is used to acquire transformer image data, dissolved gas analysis data in oil, and historical operation and maintenance data; The fault feature extraction module is used to extract image features from the transformer components in the image data, generate component image features, perform data mapping and transformation based on each dissolved gas component data in the dissolved gas analysis data in the oil, generate corresponding pixel values, generate two-dimensional image features based on the pixel values, and perform semantic alignment and structured processing on the historical operation and maintenance data to generate text features. The fault feature fusion module is used to fuse the component image features, the two-dimensional image features, and the text features to generate a multimodal fusion feature vector. The transformer fault prediction module is used to input the multimodal fusion feature vector into a preset fault diagnosis model, so that the fault diagnosis model performs deep encoding and cross-modal mapping on the multimodal fusion feature vector based on an attention mechanism, generates cross-modal correlation features, performs fault area localization and fault type analysis based on preset technical knowledge, and outputs the transformer fault location and transformer fault type. The transformer fault diagnosis module is used to output a fault diagnosis report of the transformer in natural language format based on the location and type of the transformer fault, using a preset language model.

8. The transformer fault diagnosis device as described in claim 7, characterized in that, The image data includes: infrared images and visible light images; The fault feature extraction module extracts image features from the transformer components in the image data to generate component image features, including: Identify several components of the transformer in the infrared image and the visible light image, and attach corresponding bounding boxes to each component in the infrared image and the visible light image to generate an infrared image to be identified and a visible light image to be identified. The infrared image and the visible light image to be identified are divided into grids, and several target grids containing several bounding boxes are identified. A first grid position code is generated based on several target grids of the infrared image to be identified, and a second grid position code is generated based on several target grids of the visible light image to be identified; Based on the first grid position code, the second grid position code, the infrared image to be identified, and the visible light image to be identified, the first image coordinates of each element in the infrared image to be identified and the second image coordinates in the visible light image to be identified are determined. Based on the first image coordinates and the second image coordinates, a coordinate sequence of each element is generated, and the coordinate sequence of several elements is used as the element image feature.

9. A terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a transformer fault diagnosis method as described in any one of claims 1-6.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a transformer fault diagnosis method as described in any one of claims 1-6.

Images