A transformer fault diagnosis method and device based on a multi-modal self-attention mechanism

Abstract

The application provides a transformer fault diagnosis method and device based on a multi-modal self-attention mechanism, and the method comprises the following steps: collecting multi-modal monitoring data of a transformer to obtain a data sequence matrix of multiple modes; pre-processing the matrix to obtain pre-processed data; calculating the change rate and change point index to obtain a change rate and change point feature set; performing time sequence attention calculation on each mode based on the pre-processed data and the feature set to obtain single-mode feature representation; performing cross-modal attention interaction calculation based on the single-mode feature representation to obtain global cross-modal fusion feature representation; fusing the single-mode feature representation and the global cross-modal fusion feature representation to obtain fusion representation; and inputting the fusion representation into a pre-trained classifier to obtain a transformer fault diagnosis result. The application realizes reliable mutual verification of multi-modal information by extracting dynamic feature indexes and performing multi-modal self-attention mechanism processing, and improves the transformer fault diagnosis accuracy and early warning capability.

Description

Technical Field

[0001] This invention belongs to the field of power transformer fault diagnosis technology, specifically relating to a transformer fault diagnosis method and device based on a multimodal self-attention mechanism. Background Technology

[0002] In power systems, large oil-immersed power transformers are core assets of the transmission and distribution system. Faults such as overheating, partial discharge, low-energy discharge, winding loosening, core clamp detachment, insulation aging, and moisture absorption can not only cause significant economic losses but also trigger a series of chain reactions, severely impacting the stable operation of the power system. Therefore, to ensure reliable power supply, the industry is gradually shifting from the traditional "reactive maintenance" model to a "condition-based maintenance / predictive maintenance" model, which creates an urgent need for online monitoring and early warning technologies for power transformers.

[0003] Currently, to achieve transformer condition monitoring and fault diagnosis, a multimodal monitoring system has been established, covering dissolved gas analysis (DGA), mechanical / structural vibration and acoustic emission, electrical quantities, and thermal / environmental monitoring. However, these multimodal data suffer from numerous problems such as non-stationarity and multi-source heterogeneity. From the perspective of diagnostic method development, it has evolved from rule-based methods to data-driven and multimodal time-series modeling. Rule / graphical methods primarily rely on single-modal DGA; methods like the Rogers ratio method, while simple to implement, are insensitive to dynamic changes. The Duval triangle / pentagon method is acceptable for steady-state gas discrimination, but lacks sensitivity to early, slow changes or complex operating conditions. Statistical and signal processing methods focus on time-series dynamics, while machine learning / deep learning has evolved from single-modal to multimodal approaches, with various methods for multimodal fusion.

[0004] However, existing diagnostic technologies mostly rely on static thresholds and fail to systematically extract and utilize dynamic feature indicators. Furthermore, in multimodal data fusion, they cannot take into account both time dependence within a single mode and cross-modal mutual information synchronization, making it difficult to achieve reliable information verification. This results in poor diagnostic accuracy and early warning capabilities, failing to meet the requirements of transformer condition-based maintenance and predictive maintenance. Summary of the Invention

[0005] In view of this, the present invention provides a transformer fault diagnosis method and apparatus based on a multimodal self-attention mechanism, aiming to solve the above-mentioned problems existing in the existing diagnostic technology.

[0006] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0007] In a first aspect, the present invention provides a transformer fault diagnosis method based on a multimodal self-attention mechanism, comprising the following steps:

[0008] Multi-mode monitoring data of the transformer were collected to obtain a data sequence matrix of multiple modes;

[0009] Perform preprocessing operations on the data sequence matrix to obtain preprocessed data;

[0010] Calculate the rate of change and change point index of the preprocessed data to obtain the rate of change and change point feature set;

[0011] Based on the preprocessed data and the feature set of rate of change and change point, temporal attention is calculated for each mode to obtain the single-mode feature representation of each mode;

[0012] Based on the single-modal feature representations of each modality, cross-modal attention interaction calculations are performed between modalities to obtain a global cross-modal fusion feature representation;

[0013] A fusion representation is obtained by performing a representation fusion operation between the single-modal feature representation and the global cross-modal fusion feature representation;

[0014] The fused representations are input into a pre-trained classifier to obtain the fault diagnosis results for the transformer.

[0015] Furthermore, the rate of change and change point indices of the preprocessed data are calculated, including:

[0016] Calculate the first-order rate of change:

[0017]

[0018] In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively.

[0019] Calculate the second-order rate of change:

[0020]

[0021] In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp;

[0022] Calculate the relative rate of change:

[0023]

[0024] In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. To prevent division by zero constant;

[0025] Calculate the moving average rate of change:

[0026]

[0027] In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window.

[0028] The CUSUM change point detection index is calculated as follows:

[0029]

[0030]

[0031] In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

[0032] Furthermore, the expression for temporal attention computation is:

[0033]

[0034] In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, Let T be the key matrix, and the superscript T denotes the transpose operation. The signal dimension of the data sequence matrix. The value matrix, query matrix, key matrix, and value matrix are all obtained by linear transformation of a single-modal input sequence composed of preprocessed data, rate of change, and variable point feature sets, with a learnable parameter matrix. This is the normalization function.

[0035] Furthermore, the expression for cross-modal attention interaction computation is:

[0036]

[0037] In the formula, For global cross-modal fusion feature representation, Describes a set of single-modal feature representations containing m modalities. For cross-modal query matrix, This is a cross-modal bond matrix, where the superscript T indicates the transpose operation. This represents the number of consecutive timestamps in the single-modal input sequence. For the cross-modal value matrix, This is the normalization function.

[0038] Furthermore, a representation fusion operation is performed between the single-modal feature representation and the global cross-modal fusion feature representation, including:

[0039] The preprocessed data is multiplied by the corresponding global cross-modal fusion feature representation to obtain the first representation matrix;

[0040] The first representation matrix is ​​horizontally concatenated with the single-modal feature representation to obtain the fused representation.

[0041] Secondly, the present invention provides a transformer fault diagnosis device based on a multimodal self-attention mechanism, comprising:

[0042] The data acquisition module is used to collect multi-mode monitoring data of the transformer and obtain data sequences of multiple modes;

[0043] The preprocessing module is used to perform preprocessing operations on the data sequence to obtain preprocessed data;

[0044] The first calculation module is used to calculate the rate of change and change point index of the preprocessed data, and obtain the rate of change and change point feature set.

[0045] The second calculation module is used to perform temporal attention calculation on each mode based on the preprocessed data and the rate of change and the feature set of the change point, so as to obtain the single-mode feature representation of each mode;

[0046] The third computation module is used to perform cross-modal attention interaction computation between modalities based on the single-modal feature representations of each modality, so as to obtain a global cross-modal fusion feature representation;

[0047] The fusion computing module is used to perform a representation fusion operation on the single-modal feature representation and the global cross-modal fusion feature representation to obtain the fused representation;

[0048] The fault diagnosis module is used to input the fused representation into a pre-trained classifier to obtain the fault diagnosis results of the transformer.

[0049] Furthermore, in the first calculation module, the rate of change and change point index of the preprocessed data are calculated, including:

[0050] Calculate the first-order rate of change:

[0051]

[0052] In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively.

[0053] Calculate the second-order rate of change:

[0054]

[0055] In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp;

[0056] Calculate the relative rate of change:

[0057]

[0058] In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. To prevent division by zero constant;

[0059] Calculate the moving average rate of change:

[0060]

[0061] In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window.

[0062] The CUSUM change point detection index is calculated as follows:

[0063]

[0064]

[0065] In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

[0066] Furthermore, in the second calculation module, the expression for calculating temporal attention is:

[0067]

[0068] In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, Let T be the key matrix, and the superscript T denotes the transpose operation. The signal dimension of the data sequence matrix. The value matrix, query matrix, key matrix, and value matrix are all obtained by linear transformation of a single-modal input sequence composed of preprocessed data, rate of change, and variable point feature sets, with a learnable parameter matrix. This is the normalization function.

[0069] Thirdly, the present invention provides a computer device, the device including a processor and a memory:

[0070] The memory is used to store computer programs and send the instructions of the computer programs to the processor;

[0071] The processor executes, according to the instructions of the computer program, a transformer fault diagnosis method based on a multimodal self-attention mechanism, as described in the first aspect.

[0072] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a transformer fault diagnosis method based on a multimodal self-attention mechanism as described in the first aspect.

[0073] In summary, this invention provides a transformer fault diagnosis method and apparatus based on a multimodal self-attention mechanism. The method includes the following steps: collecting multimodal monitoring data of the transformer to obtain a data sequence matrix of multiple modes; preprocessing the data sequence matrix to obtain preprocessed data; calculating the rate of change and change point indices of the preprocessed data to obtain a set of rate of change and change point features; performing temporal attention calculation on each mode based on the preprocessed data and the rate of change and change point feature sets to obtain a single-modal feature representation for each mode; performing cross-modal attention interaction calculation between modes based on the single-modal feature representations of each mode to obtain a global cross-modal fusion feature representation; performing a representation fusion operation between the single-modal feature representation and the global cross-modal fusion feature representation to obtain a fused representation; and inputting the fused representation into a pre-trained classifier to obtain the transformer fault diagnosis result. This invention, by extracting dynamic features of the rate of change and change points and processing them using a multimodal self-attention mechanism, achieves reliable mutual verification of multimodal information, effectively improving the accuracy of transformer fault diagnosis and early warning capabilities, and meeting the requirements of condition-based maintenance and predictive maintenance. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A flowchart illustrating a transformer fault diagnosis method based on a multimodal self-attention mechanism, provided in an embodiment of the present invention.

[0076] Figure 2 A schematic diagram illustrating the principle of a transformer fault diagnosis method based on a multimodal self-attention mechanism provided in an embodiment of the present invention;

[0077] Figure 3 A block diagram illustrating the composition of a transformer fault diagnosis device based on a multimodal self-attention mechanism, provided in an embodiment of the present invention;

[0078] Figure 4 This is a block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0079] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0080] The background technology of this invention will be further introduced below.

[0081] Large oil-immersed power transformers are core assets of power transmission and distribution systems. Various faults such as overheating, partial discharge, winding loosening, and insulation aging can cause significant economic losses and potentially trigger cascading accidents. Therefore, the industry has gradually shifted from traditional "post-incident maintenance" to a "condition-based maintenance / predictive maintenance" model, creating an urgent need for online monitoring and early warning technologies. To achieve this goal, a multi-dimensional, multi-modal monitoring system has been established, covering various typical monitoring signals: First, dissolved gas analysis (DGA) in oil, which reflects the fault mechanisms of thermal faults and paper insulation pyrolysis through the temporal evolution of gas concentrations such as H2, CH4, and C2H6; second, mechanical / structural vibration and acoustic emission signals, which can capture characteristic frequency band energy changes caused by winding loosening and partial discharge impacts; third, electrical quantity signals such as voltage, current, and harmonics, used to characterize equipment operating conditions and excitation status; and fourth, thermal / environmental signals such as oil temperature, winding hot spot temperature, and ambient temperature and humidity, reflecting the environmental factors affecting equipment operation. However, these monitoring data present significant challenges due to their complexity, including non-stationarity caused by seasonal and load variations, multi-source heterogeneity resulting from differences in sampling frequencies and dimensions of different signals, noise and missing data caused by sensor drift or distortion, event sparsity caused by the scarcity of fault samples, and multi-scale dynamics where chronic aging and sudden change points coexist. This requires diagnostic algorithms to accurately capture the slow trend rate of change of the data, identify sudden transitions and change points in a timely manner, and achieve reliable information verification between multimodal data.

[0082] In the long-term operation of actual substations, transformer faults are mostly not instantaneous, but rather undergo a gradual evolution from normal operation, minor anomalies, localized degradation, to obvious faults. During this process, various monitoring signals often initially show weak amplitude and slow trends, gradually accumulating to form observable abnormal characteristics. Traditional diagnostic methods are insufficient in capturing this early, slowly changing, low-amplitude dynamic information, often only triggering alarms when signals exceed fixed thresholds. By this time, the equipment has already entered a significant fault stage, leaving maintenance personnel with extremely limited time to respond. Simultaneously, there is a natural physical coupling between multimodal monitoring signals. For example, partial discharge can simultaneously induce gas growth in the oil, abrupt changes in vibration signals, and electrical harmonic distortion. A single mode can only reflect the local manifestation of the fault. Only through the collaborative verification and cross-verification of multimodal information can misjudgments caused by single sensor noise, environmental interference, or load fluctuations be effectively eliminated, improving the reliability and confidence of diagnostic results.

[0083] With the comprehensive advancement of smart substation and digital grid construction, online transformer monitoring devices have gradually achieved full coverage, accumulating massive amounts of historical time-series data, providing a solid foundation for data-driven intelligent diagnosis. However, this massive data also brings new challenges: how to automatically extract key features highly correlated with fault evolution from high-dimensional, long-time-series, noisy, and multi-rate-sampled data; how to achieve end-to-end fault identification and early warning without relying on extensive manual feature engineering; and how to improve the interpretability and engineering practicality of the model while ensuring high accuracy—these have become key bottlenecks restricting the implementation of intelligent fault diagnosis technology for transformers. Most existing technologies remain at the level of single-modal analysis, simple feature splicing, or static threshold judgment, making it difficult to fully release the fault evolution information contained in multimodal time-series data, resulting in a significant gap between these technologies and the actual engineering requirements for high reliability, high sensitivity, interpretability, and early warning capabilities.

[0084] From a technological evolution perspective, transformer fault diagnosis technology has undergone a development process from rule-based methods to data-driven and multimodal time-series modeling. In the traditional stage, diagnostic techniques mainly relied on single-modal DGA rule / spectral methods, with typical methods including the Rogers ratio method, the IEC ratio method, and the Duval triangle / pentagon method. The former determines the fault type by the area where the gas concentration ratio falls, while the latter projects key hydrocarbon gases onto a specific area to locate the fault category. These methods are simple to implement and highly interpretable, but they lack sensitivity to dynamic changes in data (rate, acceleration) and cross-modal evidence, making them difficult to handle complex operating conditions. Subsequently, statistical and signal processing methods were introduced to capture time-series dynamic features. For example, first-order and second-order differences are used to enhance the dynamic components of the data, the slope of the sliding window linear fitting is used to characterize the average rate of change, and the CUSUM cumulative sum method is used to achieve early abnormal change point detection. These methods can effectively amplify trend features and suppress high-frequency noise, making them suitable for online threshold alarm scenarios. With the development of artificial intelligence technology, machine learning and deep learning methods have gradually become mainstream. Initially, models such as SVM, random forest, and LSTM / GRU were used for single-modal learning of DGA time series or vibration spectra, demonstrating powerful complex pattern recognition capabilities. Subsequently, multimodal fusion technology has gradually developed, successively going through three stages: early feature splicing, late decision-level voting / weighting, and mid-term attention fusion. Among them, the introduction of attention mechanism can adaptively weight different modalities and time segments, improving the algorithm's ability to focus on key information. Self-attention, as the core idea of ​​Transformer, has become a universal cornerstone for building intramodal and cross-modal attention models in the industrial time series field.

[0085] Despite significant progress in deep learning and multimodal fusion technologies, they still have obvious limitations when applied to real-world transformer fault scenarios. On one hand, existing deep learning models mostly use raw time-series data as input, lacking explicit representations of the physical mechanisms of faults. These models focus more on the statistical correlation of data than the causal relationship of fault evolution, resulting in insufficient generalization ability when faced with small samples, weak signals, and varying operating conditions. Furthermore, the diagnostic results are difficult for on-site maintenance personnel to understand and accept. On the other hand, most fusion structures adopt a unified modeling approach of "first splicing, then attention," mixing single-modal time-series dependencies with cross-modal correlations. This can easily dilute key temporal features within a mode and strengthen spurious cross-modal correlations, thereby reducing the model's sensitivity and robustness to early, minor faults.

[0086] Furthermore, early warning of faults essentially involves identifying "changing trends" and "abrupt points," rather than classifying static states. Existing technologies rarely embed dynamic indicators with clear physical meaning, such as rate of change, acceleration, relative change, slip trend, accumulation, and change points, as explicit features into deep networks. They also lack a systematic architecture of "dynamic feature enhancement—single-modal time-series focusing—cross-modal mutual verification fusion." This results in models exhibiting delayed response, low recognition rates, and high false alarm / false negative rates when facing typical early fault patterns such as slowly rising gas concentrations, gradually increasing vibration energy, and intermittent partial discharges. Consequently, they fail to truly meet the core requirements of predictive maintenance for early detection, early warning, and early intervention.

[0087] Although significant progress has been made in transformer fault diagnosis technology, existing research still has obvious shortcomings: most works have not specifically and systematically constructed explicit dynamic features such as "rate of change / point of change", and even fewer have carried out end-to-end collaborative training of "single-modal intra-temporal attention modeling" and "cross-modal coupling degree modeling" with rate of change features. It is difficult to give full play to the information mutual verification value of multimodal data and cannot meet the high-precision requirements of condition-based maintenance and predictive maintenance.

[0088] Although transformer fault diagnosis technology has gradually evolved towards data-driven and multimodal fusion, existing technologies still have many shortcomings that urgently need to be addressed. These shortcomings are precisely the core improvement directions of this invention: First, the sensitivity to dynamic features is severely insufficient. Existing methods mostly rely on static thresholds, static features, or raw time-series data for diagnosis, failing to systematically extract and utilize dynamic indicators that can accurately reflect "rate of change / abrupt change," such as first-order rate of change, second-order rate of change, relative rate of change, moving average slope, and CUSUM change points. This makes it difficult to capture the weak dynamic signs in the early stages of equipment faults. Second, the multimodal fusion capability has significant shortcomings. Using simple feature splicing or unified attention modeling methods cannot simultaneously and accurately characterize "the same mode." The system suffers from several shortcomings. Firstly, the data lacks robustness and interpretability. It lacks robust mechanisms to handle complex operating conditions such as transformer load fluctuations, temperature drift, sampling noise, and missing data. It also fails to effectively integrate interpretable statistical indicators with clear physical meaning, such as CUSUM variable points and moving average slopes, with the deep model, thus limiting the reliability and credibility of diagnostic results. Finally, the system has insufficient early warning capabilities. It is not sensitive to subtle, gradual changes in oil and gas concentrations or vibration energy at specific frequencies, making early fault identification and warning difficult and failing to meet the core requirements of predictive maintenance.

[0089] Furthermore, existing technologies generally suffer from the following shortcomings: First, the model structure is disconnected from the physical mechanism of the fault. Many deep models are purely data-driven structures, failing to incorporate prior knowledge such as transformer thermal characteristics, insulation aging mechanisms, and discharge development laws into the network design. This results in poor generalization ability and high transfer difficulty for the model in small sample and weak fault scenarios. Second, the utilization of multimodal information is superficial. Most methods only achieve simple fusion at the feature level or decision level, without fine-grained modeling of causal relationships, synchronicity, and hysteresis response relationships between modes, making it difficult to effectively suppress misjudgments caused by single-mode noise. Third, there is insufficient engineering implementation capability. Some complex models have high computational load, slow inference speed, and high hardware requirements, making them unsuitable for deployment on embedded edge monitoring devices and failing to meet the needs of online real-time diagnosis of substations. Fourth, there is a lack of a unified dynamic feature enhancement system. Various change rates and change point indicators are used in a scattered manner, without forming a standardized, scalable, end-to-end dynamic feature enhancement process, which limits the further improvement of early warning performance.

[0090] To address the aforementioned shortcomings of existing technologies, the core objective of this invention is to provide a high-precision and robust transformer fault diagnosis scheme. By extracting dynamic feature indicators (such as first-order rate of change, second-order rate of change, relative rate of change, moving average rate of change, and CUSUM change point indicators), the scheme enhances dual sensitivity to both chronic degradation trends and sudden fault change points. By learning the importance distribution of different temporal positions within each mode, it improves the ability to focus on key time segments and capture core fault information within each mode. By learning the interaction strength and synchronicity between different modes, it achieves adaptive allocation of mode weights and effective modeling of complementary information. Finally, it constructs an end-to-end multimodal diagnostic framework that organically integrates interpretable statistical indicators with a deep attention mechanism, balancing the interpretability of diagnostic results, robustness under complex operating conditions, and engineering feasibility, thereby significantly improving the accuracy of transformer fault diagnosis, early warning capabilities, and operational stability.

[0091] This invention also aims to address pain points in practical engineering, such as asynchronous multimodal data sampling, inconsistent dimensions, strong noise interference, and a small number of fault samples. It enhances weak signal perception capabilities through explicit dynamic feature construction, reduces information interference through decoupled single-modal and cross-modal attention structures, and ensures real-time inference at the edge through lightweight fusion design. This enables the model to maintain high accuracy on laboratory datasets and output diagnostic results stably, reliably, and interpretably during long-term online operation, truly achieving an upgrade from data-driven to knowledge and data fusion-driven approaches, and providing solid technical support for transformer condition-based maintenance and intelligent operation and maintenance.

[0092] The various embodiments of the present invention will be described in detail below.

[0093] Please see Figure 1 This embodiment provides a transformer fault diagnosis method based on a multimodal self-attention mechanism, including the following steps:

[0094] S1: Collect multi-mode monitoring data of the transformer to obtain a data sequence matrix of multiple modes.

[0095] It should be noted that multimodal monitoring data refers to a collection of monitoring information with different attributes and characteristics obtained from different monitoring dimensions of transformers. A data sequence matrix, on the other hand, refers to a two-dimensional matrix structure formed by organizing monitoring data of the same mode in sequential order of consecutive timestamps, with the rows and columns corresponding to the time dimension and the data feature dimension, respectively.

[0096] For example, corresponding sensor data can be collected through an online oil chromatography monitoring device (DGA sensor), vibration accelerometer / velocity sensor, sound sensor, electrical quantity acquisition module (current / voltage / harmonic), temperature / humidity probe, etc., and then the data sequences of each type of sensor are integrated in chronological order according to the timestamp to form a data sequence matrix.

[0097] In actual data acquisition, the hardware characteristics and working mechanisms of different modal sensors vary significantly. For example, DGA data has a long sampling period, typically ranging from several minutes to several hours, while vibration and electrical signals are sampled continuously at high frequencies, and temperature signals are slowly changing and stable. This step supports unified time axis alignment for heterogeneous signals with different sampling frequencies, data formats, and acquisition times, effectively avoiding intermodal correlation distortion caused by time misalignment.

[0098] S2: Perform preprocessing operations on the data sequence matrix to obtain preprocessed data.

[0099] It should be noted that preprocessing refers to data optimization operations performed on the original data sequence matrix to address issues such as noise, temporal misalignment, and missing data. Preprocessed data refers to data that has undergone optimization operations and possesses characteristics such as time alignment, noise reduction, and format uniformity.

[0100] For example, preprocessing operations may include time alignment, noise reduction, outlier handling, missing value completion, and normalization.

[0101] S3: Calculate the rate of change and change point index of the preprocessed data to obtain the rate of change and change point feature set.

[0102] It should be noted that rate of change and change point indicators refer to quantitative indicators that can reflect the trend, magnitude, and abrupt changes of data over time. The rate of change and change point feature set refers to a feature set composed of various rate of change indicators and change point indicators.

[0103] This step calculates the corresponding rate of change and change point index based on the preprocessed data, transforming the static data that could only reflect the operating status of the transformer at a certain moment into dynamic features that can reflect the trend of state change and abrupt change nodes, thereby capturing the weak signs of transformer faults in the early stage.

[0104] This step, by explicitly constructing a set of change rate and change point features, directly transforms the physical process of fault evolution into numerical features that the model can learn. This allows the model to focus directly on dynamic information highly correlated with the fault, rather than blindly learning trends from massive amounts of raw data. This dynamic feature enhancement significantly improves the model's ability to perceive early, slowly changing anomalies and intermittent, abruptly changing anomalies. Furthermore, because indicators such as change rate, slope, and cumulative sum have clear physical meanings, it greatly enhances the interpretability of the model's decision-making process, making it easier for operations and maintenance personnel to understand the basis for early warnings and determine the stage of fault development.

[0105] S4: Based on the preprocessed data and the feature set of rate of change and change point, perform temporal attention calculation for each mode to obtain the single-modal feature representation of each mode.

[0106] It should be noted that temporal attention computation is a feature extraction method that can adaptively focus on key time segments in a data sequence, assigning differentiated weights to data at different time points. Single-modal feature representation refers to the feature vector or matrix obtained after temporal attention computation of data from a single modality, which reflects the core fault information of that modality.

[0107] This step combines the preprocessed data with the rate of change and the feature set of change points as input for time series attention calculation. Through the attention mechanism, it automatically identifies and focuses on the key time segments related to the fault in the modal data, weakens the interference of irrelevant time point data, and finally generates a single modal feature representation that can characterize the fault features of the modality.

[0108] S5: Based on the single-modal feature representations of each modality, perform cross-modal attention interaction calculations between modalities to obtain a global cross-modal fusion feature representation.

[0109] It should be noted that cross-modal attention interaction computation is a computational method used to model the correlation between data from different modalities, and it can adaptively mine complementary information and mutual verification relationships between modalities. Global cross-modal fusion feature representation refers to a fusion feature formed by integrating feature information from all modalities, which has the ability to represent faults globally and in multiple dimensions.

[0110] This step takes the single-modal feature representations of each modality as input, and through cross-modal attention interaction calculation, it explores the intrinsic correlation between different modal data, adaptively assigns weights to different modalities, strengthens modal information that is valuable for fault diagnosis, weakens the influence of interfering modalities, and finally generates a global cross-modal fusion feature representation that can comprehensively reflect the correlation features of multimodal data.

[0111] S6: Perform a characterization fusion operation on the single-modal feature representation and the global cross-modal fusion feature representation to obtain the fused representation.

[0112] It should be noted that representation fusion operation refers to a processing method that integrates different types of feature representations, aiming to retain the advantages of different features simultaneously. Fusion representation, on the other hand, refers to the comprehensive feature obtained after representation fusion operation, which simultaneously contains single-modal core features and multimodal related features.

[0113] This step integrates single-modal feature representations with global cross-modal fusion feature representations through a representation fusion operation, which not only preserves the core fault features of each mode but also incorporates the correlation and complementary information between modes.

[0114] S7: Input the fused representation into the pre-trained classifier to obtain the fault diagnosis results of the transformer.

[0115] It should be noted that a pre-trained classifier refers to a model that has been trained in advance based on fault sample data and has the ability to identify fault types.

[0116] This step integrates the input representations into a pre-trained classifier. Based on the learned fault feature patterns, the classifier matches and identifies the integrated representations of the input, and finally outputs diagnostic results such as whether the transformer has a fault and the type of fault, thus completing the entire fault diagnosis process.

[0117] This embodiment provides a transformer fault diagnosis method based on a multimodal self-attention mechanism. Addressing the shortcomings of existing transformer fault diagnosis technologies in utilizing dynamic features and the difficulty in balancing single-modal time dependence and cross-modal mutual information in multimodal fusion, this method constructs an end-to-end multimodal self-attention diagnostic framework consisting of "dynamic feature enhancement - single-modal temporal focusing - cross-modal correlation modeling - dual-feature fusion." By extracting dynamic features of change rate and change points, the method enhances sensitivity to slow and abrupt fault signs. Then, by employing decoupled modeling of single-modal temporal attention and cross-modal attention, it achieves accurate focusing on key time segments within a single mode and deep mining of intermodal correlation information. Finally, through the representation fusion of single-modal features and global cross-modal fusion features, it balances the independence of single-modal features with the complementarity of multimodal features, effectively solving the problems of low diagnostic accuracy and weak early warning capability in traditional methods, and improving the overall performance of transformer fault diagnosis.

[0118] Compared with existing technologies, this embodiment has multiple advantages: First, dynamic features are made explicit, directly embedding indicators with clear physical meanings such as rate of change and change points into the model front end, significantly improving the sensitivity to early faults; second, the attention structure is decoupled, first focusing on single-modal time sequence and then fusing cross-modal associations, avoiding information aliasing and feature dilution; third, the fusion mechanism is refined, fully preserving information at all levels by weighting the original features and cross-modal representations and then splicing them with single-modal representations; fourth, it has strong engineering practicality, with a lightweight overall architecture, fast inference speed, and can be deployed at the edge, while also having good interpretability, making it easy to promote and apply in actual substation operation and maintenance systems.

[0119] Please see Figure 2 , Figure 2 This paper illustrates the implementation principle of a transformer fault diagnosis method based on a multimodal self-attention mechanism: Using multimodal time-series signals from transformer sensors as input, the raw data is first preprocessed and regularized. The preprocessed data then extracts dynamic features through rate of change calculation, followed by vector dimension filling and concatenation, and is then used for single-modal attention modeling (focusing on key time segment information within a single mode) and cross-modal coupling modeling (mining the correlation and complementarity between different modes). The results of these two modeling methods are then fused to integrate single-modal core features and cross-modal correlation features. After linear layer and Softmax normalization, the fault prediction result is obtained. Finally, the cross-entropy loss function is used to compare with the actual transformer fault categories, completing the supervised training and output of the fault diagnosis result. The following is based on... Figure 2 The principle illustrated will be used to introduce some other embodiments of the present invention.

[0120] In one embodiment of the present invention, calculating the rate of change and change point index of the preprocessed data includes:

[0121] Calculate the first-order rate of change:

[0122]

[0123] In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively.

[0124] Calculate the second-order rate of change:

[0125]

[0126] In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp;

[0127] Calculate the relative rate of change:

[0128]

[0129] In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements of two vectors are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. It is a very small constant to prevent division by zero;

[0130] Calculate the moving average rate of change:

[0131]

[0132] In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window.

[0133] The CUSUM change point detection index is calculated as follows:

[0134]

[0135]

[0136] In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

[0137] The five types of dynamic features described above depict the evolution of transformer operating states from different dimensions, collectively forming a systematic dynamic feature enhancement system: instantaneous rate of change captures rapid fluctuations, second-order rate of change identifies trend shifts, relative rate of change eliminates the influence of dimensions, moving average slope reflects long-term trends, and CUSUM change points accurately locate the onset of anomalies. This system can comprehensively cover the full-cycle dynamic features of transformers from slow aging to sudden failures, providing high-quality, highly discriminative, and highly interpretable input information for subsequent attention models, thereby improving fault diagnosis and early warning performance from the source.

[0138] This embodiment calculates the first-order rate of change, second-order rate of change, relative rate of change, moving average rate of change, and CUSUM change point detection index of the preprocessed data using specific formulas. The first-order rate of change characterizes the instantaneous magnitude of data change; the second-order rate of change reflects the trend of the magnitude of change; the relative rate of change eliminates dimensional differences between different modal data by dividing the elements and introducing a minimal constant to avoid division by zero, thus reflecting the proportion of change; the moving average rate of change smooths high-frequency noise through linear fitting over a time window, highlighting the long-term trend of the data; and the CUSUM change point detection index is based on the mean and standard deviation of the transformer's normal state data, combined with a sensitivity threshold to calculate the positive and negative cumulative sums, achieving accurate identification of data abrupt change nodes. By calculating these indices, the original static monitoring data is transformed into characteristic information reflecting the dynamic changes in the transformer's operating state, overcoming the shortcomings of existing technologies in utilizing dynamic features and enhancing the dual sensitivity to both early-stage gradual changes in transformer fault signs and sudden fault changes.

[0139] In one embodiment of the present invention, the expression for calculating temporal attention is:

[0140]

[0141] In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, , The key matrix, The superscript T indicates the transpose operation. The signal dimension of the data sequence matrix. For value matrices, The query matrix, key matrix, and value matrix are all obtained by linear transformation of the preprocessed data, the rate of change, and the feature set of change points, forming a single-modal input sequence, with the learnable parameter matrix. This is the normalization function.

[0142] In this embodiment, adaptive focusing on key time segments in a single-modal input sequence is achieved by constructing a query matrix, a key matrix, and a value matrix. The query matrix, key matrix, and value matrix are all generated from the single-modal input sequence, composed of preprocessed data, rate of change, and variable point feature sets, through a linear transformation with a learnable parameter matrix. The matrix transpose operation T is used to meet the dimensionality matching requirement of the inner product calculation between the query matrix and the key matrix. The inner product result is transformed into attention weights in the 0-1 interval using the Softmax normalization function, thereby increasing the weight of fault-related key time segments and weakening the weight of irrelevant time segments in the single-modal input sequence. The entire calculation process retains the basic information of the preprocessed data and the dynamic information of the rate of change and variable point feature sets, while accurately capturing the temporal dependencies within the single modality through the attention mechanism.

[0143] In one embodiment of the present invention, the expression for cross-modal attention interaction computation is:

[0144]

[0145] In the formula, For global cross-modal fusion feature representation, Describes a set of single-modal feature representations containing m modalities. For cross-modal query matrix, , For the cross-modal bond matrix, The superscript T indicates the transpose operation. This represents the number of consecutive timestamps in the single-modal input sequence. For the cross-modal value matrix, , This is the normalization function.

[0146] In this embodiment, cross-modal attention interaction computation can uncover the intrinsic correlations between features of different modalities, achieving complementary fusion of multimodal information. The input to the cross-modal attention interaction computation is the set of single-modal feature representations for all modalities. After passing through a Softmax normalization function, the inner product result is transformed into inter-modal attention weights, adaptively strengthening modal correlation information valuable for fault diagnosis and weakening interference from irrelevant modalities. The entire computation process models the mutual information and synchronization between modalities through an attention mechanism, breaking down the information barriers of single-modal features.

[0147] In one embodiment of the present invention, a representation fusion operation is performed between a single-modal feature representation and a global cross-modal fusion feature representation, including:

[0148] S61: Multiply the preprocessed data with the global cross-modal fusion feature representation to obtain the first representation matrix.

[0149] S62: The first representation matrix is ​​horizontally concatenated with the single-modal feature representation to obtain the fused representation.

[0150] Assuming the preprocessed data is , This represents the preprocessed input signal of a certain mode; it is a... The matrix has n rows representing the sensor readings after preprocessing n consecutive timestamps, and d represents the dimension of the sensor signal of the corresponding modality; the single-modal feature is represented as P (n rows and c columns, where the rows represent time points and the columns represent the sum of the dimensions of the potential representations of all modalities); the global cross-modal fusion feature is represented as E (k rows and r columns, where the rows represent the original features of all modalities and the columns represent the dimensions of the potential representations).

[0151] After the representation fusion operation, it becomes: [[X (1) ,X (2) ,…,X (m) [*E, P], which means multiplying the original feature matrix by the new representation matrix E of each modality to obtain an n*r representation matrix, and then concatenating the representation matrix and P horizontally.

[0152] The core technology of this invention is reflected in three levels. First, it systematically designs and integrates the calculation of rate of change and change point features end-to-end, combining first-order, second-order, relative, moving average rate of change, and CUSUM change point indicators. These indicators, which can characterize the dynamic characteristics of data, are used as explicit features of the trainable process and integrated into the same diagnostic framework. Second, it adopts a decoupled modeling strategy of single-modal temporal attention and cross-modal coupling. It first completes the learning and focusing of key time segments within a single modality, and then learns the coupling degree and weight allocation between different modalities, effectively avoiding the information dilution and noise amplification problems caused by early mixing of multimodal information. Third, it constructs an adaptive multimodal fusion method based on the coupling degree matrix, dynamically adjusting the information flow between modalities through mutual attention, thereby improving the robustness and generalization ability of the method under different working conditions.

[0153] The above-mentioned technical design has brought about significant application effects. On the one hand, the explicit rate of change and change point features have greatly improved the robustness of the model to slowly changing and abrupt signals. After focusing on key time slices through single-modal time-series attention, the information mutual verification between modes is enhanced through cross-modal coupling, which ultimately achieves a dual improvement in the accuracy of transformer fault diagnosis and early warning capability. On the other hand, compared with traditional diagnostic methods that rely solely on static rules or simple fusion, this invention can more sensitively capture the early abnormal state of the transformer and respond to potential fault signs of the equipment earlier, meeting the engineering needs of condition-based maintenance and predictive maintenance.

[0154] In actual engineering tests, compared with traditional single-modal methods and simple multimodal fusion methods, the present invention significantly improves the accuracy of comprehensive fault identification, significantly extends the early warning time of early anomalies, and greatly reduces the false alarm rate and missed alarm rate. It can effectively identify weak anomalies and samples with ambiguous boundaries that are difficult to detect by traditional methods. It is especially suitable for online monitoring and intelligent operation and maintenance scenarios of key equipment such as 220kV and above main transformers and important station service transformers.

[0155] Based on the above-described technical solutions of this invention, in order to further expand the applicable scenarios of the method and adapt to different engineering practice conditions and data characteristics, there are also a number of equivalent alternative technical solutions that do not depart from the core idea of ​​this invention. These solutions specifically include the following categories:

[0156] 1. Alternatives or extensions to the calculation of rate of change and change point characteristics

[0157] (1) Replace the sliding linear slope with:

[0158] Exponentially weighted moving slope (giving higher weight to recent data).

[0159] Robust regression slopes (such as Huber loss) resist the effects of spikes.

[0160] Multiscale difference / wavelet packet energy change rate to cover different time scales.

[0161] (2) Replace or parallelize CUSUM:

[0162] Bayesian change point detection, GLR (generalized likelihood ratio) detection, E-Divisive or Kernel-based change point methods.

[0163] (3) Introduce the rate of change of the derived ratio: calculate the rate of change of the DGA ratio (such as the Rogers ratio) to improve the sensitivity to specific mechanisms.

[0164] 2. Alternatives or extensions to unimodal temporal attention computation

[0165] (1) Use Transformer-style multi-head self-attention (in-modality) or temporal convolution + gating unit (TCN + gating).

[0166] (2) Introduce prior masks (such as operating condition labels and maintenance records) into the attention score to achieve prior guidance of attention.

[0167] 3. Alternatives or extensions to cross-modal attention-based interactive computation

[0168] (1) Graph structure modeling: Each modality is regarded as a graph node, the edge weight is the coupling degree, and the graph attention network (GAT) or graph convolutional network (GCN) is used.

[0169] (2) Maximize mutual information: Improve the extraction of shared information between modalities by contrastive learning methods such as InfoNCE.

[0170] (3) Dynamic routing (capsule concept) performs iterative routing between modalities to obtain a more robust fusion representation.

[0171] 4. Classification and Training Strategy Alternatives

[0172] (1) Loss function: Focal Loss handles class imbalance; multi-task joint (fault category + severity regression).

[0173] (2) Semi-supervised / self-supervised: Pre-training is performed using a large amount of unlabeled running data (mask time series prediction, contrastive learning), and then fine-tuned with a small amount of labeled data.

[0174] Based on the same inventive concept, this application also provides a transformer fault diagnosis device based on a multimodal self-attention mechanism for implementing the transformer fault diagnosis method based on the multimodal self-attention mechanism described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in the embodiments of the transformer fault diagnosis device based on the multimodal self-attention mechanism provided below can be found in the limitations of the transformer fault diagnosis method based on the multimodal self-attention mechanism described above, and will not be repeated here.

[0175] Please see Figure 3 This invention also provides a transformer fault diagnosis device based on a multimodal self-attention mechanism, comprising:

[0176] The data acquisition module is used to collect multi-mode monitoring data of the transformer and obtain data sequences of multiple modes;

[0177] The preprocessing module is used to perform preprocessing operations on the data sequence to obtain preprocessed data;

[0178] The first calculation module is used to calculate the rate of change and change point index of the preprocessed data, and obtain the rate of change and change point feature set.

[0179] The second calculation module is used to perform temporal attention calculation on each mode based on the preprocessed data and the rate of change and the feature set of the change point, so as to obtain the single-mode feature representation of each mode;

[0180] The third computation module is used to perform cross-modal attention interaction computation between modalities based on the single-modal feature representations of each modality, so as to obtain a global cross-modal fusion feature representation;

[0181] The fusion computing module is used to perform a representation fusion operation on the single-modal feature representation and the global cross-modal fusion feature representation to obtain the fused representation;

[0182] The fault diagnosis module is used to input the fused representation into a pre-trained classifier to obtain the fault diagnosis results of the transformer.

[0183] Furthermore, in the first calculation module, the rate of change and change point index of the preprocessed data are calculated, including:

[0184] Calculate the first-order rate of change:

[0185]

[0186] In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively.

[0187] Calculate the second-order rate of change:

[0188]

[0189] In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp;

[0190] Calculate the relative rate of change:

[0191]

[0192] In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. To prevent division by zero constant;

[0193] Calculate the moving average rate of change:

[0194]

[0195] In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window.

[0196] The CUSUM change point detection index is calculated as follows:

[0197]

[0198]

[0199] In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

[0200] Furthermore, in the second calculation module, the expression for calculating temporal attention is:

[0201]

[0202] In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, Let T be the key matrix, and the superscript T denotes the transpose operation. The signal dimension of the data sequence matrix. The value matrix, query matrix, key matrix, and value matrix are all obtained by linear transformation of a single-modal input sequence composed of preprocessed data, rate of change, and variable point feature sets, with a learnable parameter matrix. This is the normalization function.

[0203] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0204] Reference Figure 4 The present invention also provides a computer device, including: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the transformer fault diagnosis method based on a multimodal self-attention mechanism as described in any of the above methods.

[0205] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0206] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0207] In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0208] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the transformer fault diagnosis method based on a multimodal self-attention mechanism as described in any of the above methods.

[0209] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0210] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the transformer fault diagnosis method based on a multimodal self-attention mechanism as described in any of the above methods.

[0211] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0212] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0213] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0214] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A transformer fault diagnosis method based on a multimodal self-attention mechanism, characterized in that, Includes the following steps: Multi-mode monitoring data of the transformer were collected to obtain a data sequence matrix of multiple modes; The data sequence matrix is ​​preprocessed to obtain preprocessed data; Calculate the rate of change and change point index of the preprocessed data to obtain the rate of change and change point feature set; Based on the preprocessed data and the rate of change and change point feature set, temporal attention is calculated for each mode to obtain the single-mode feature representation of each mode; Based on the single-modal feature representations of each modality, cross-modal attention interaction calculations are performed to obtain a global cross-modal fusion feature representation; The single-modal feature representation and the global cross-modal fusion feature representation are subjected to a representation fusion operation to obtain a fused representation; The fused representation is input into a pre-trained classifier to obtain the fault diagnosis results of the transformer.

2. The transformer fault diagnosis method based on multimodal self-attention mechanism according to claim 1, characterized in that, Calculating the rate of change and change point index of the preprocessed data includes: Calculate the first-order rate of change: In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively. Calculate the second-order rate of change: In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp; Calculate the relative rate of change: In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. To prevent division by zero constant; Calculate the moving average rate of change: In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window. The CUSUM change point detection index is calculated as follows: In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

3. The transformer fault diagnosis method based on multimodal self-attention mechanism according to claim 1, characterized in that, The expression for calculating the temporal attention is: In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, Let T be the key matrix, and the superscript T denotes the transpose operation. Let be the signal dimension of the data sequence matrix. The query matrix, key matrix, and value matrix are all obtained by linear transformation of the preprocessed data, the rate of change, and the change point feature set into a single-modal input sequence, and the learnable parameter matrix. This is the normalization function.

4. The transformer fault diagnosis method based on multimodal self-attention mechanism according to claim 1, characterized in that, The expression for the cross-modal attention interaction calculation is: In the formula, This represents the global cross-modal fusion feature representation. Describes a set of single-modal feature representations containing m modalities. For cross-modal query matrix, This is a cross-modal bond matrix, where the superscript T indicates the transpose operation. This represents the number of consecutive timestamps in the single-modal input sequence. For the cross-modal value matrix, This is the normalization function.

5. The transformer fault diagnosis method based on multimodal self-attention mechanism according to claim 1, characterized in that, The representation fusion operation between the single-modal feature representation and the global cross-modal fusion feature representation includes: The preprocessed data is multiplied by the global cross-modal fusion feature representation to obtain the first representation matrix; The first representation matrix and the single-modal feature representation are horizontally concatenated to obtain the fused representation.

6. A transformer fault diagnosis device based on a multimodal self-attention mechanism, characterized in that, include: The data acquisition module is used to collect multi-mode monitoring data of the transformer and obtain data sequences of multiple modes; The preprocessing module is used to perform preprocessing operations on the data sequence to obtain preprocessed data; The first calculation module is used to calculate the rate of change and change point index of the preprocessed data, and obtain the rate of change and change point feature set. The second calculation module is used to perform temporal attention calculation on each mode based on the preprocessed data and the rate of change and change point feature set, so as to obtain the single-mode feature representation of each mode; The third calculation module is used to perform cross-modal attention interaction calculations between modalities based on the single-modal feature representations of each modality, so as to obtain a global cross-modal fusion feature representation; The fusion computing module is used to perform a representation fusion operation on the single-modal feature representation and the global cross-modal fusion feature representation to obtain a fused representation; The fault diagnosis module is used to input the fused representation into a pre-trained classifier to obtain the fault diagnosis results of the transformer.

7. The transformer fault diagnosis device based on multimodal self-attention mechanism according to claim 6, characterized in that, The first calculation module calculates the rate of change and change point index of the preprocessed data, including: Calculate the first-order rate of change: In the formula, Let be the first-order rate of change of the j-th mode at time t. and These are the preprocessed data for the j-th mode at timestamps t and t-1, respectively. Calculate the second-order rate of change: In the formula, Let be the second-order rate of change of the j-th mode at the t-th time stamp. This is the preprocessed data for the j-th mode at the (t-2)th timestamp; Calculate the relative rate of change: In the formula, Let be the relative rate of change of the j-th mode at time t. This indicates that corresponding elements are divided. Let be the absolute value of the data of the j-th mode after preprocessing at the (t-1)-th timestamp. To prevent division by zero constant; Calculate the moving average rate of change: In the formula, Let L be the moving average rate of change corresponding to the t-th timestamp, and L be the length of the time window. This refers to the preprocessed data corresponding to the i-th timestamp within the time window. The CUSUM change point detection index is calculated as follows: In the formula, Let the positive cumulative sum of the j-th mode be the sum of the j-th mode at the t-th timestamp. For the preprocessed data of the t-th timestamp, This represents the mean value of the modal data corresponding to the normal state of the transformer. This represents the standard deviation of the modal data corresponding to the transformer under normal conditions. This is the sensitivity threshold.

8. The transformer fault diagnosis device based on multimodal self-attention mechanism according to claim 6, characterized in that, In the second calculation module, the expression for calculating the temporal attention is: In the formula, This represents the single-modal feature representation of the i-th mode. For querying the matrix, Let T be the key matrix, and the superscript T denotes the transpose operation. The signal dimension of the data sequence matrix. The value matrix, query matrix, key matrix, and value matrix are all obtained by linear transformation of a single-modal input sequence composed of preprocessed data, rate of change, and variable point feature sets, with a learnable parameter matrix. This is the normalization function.

9. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store computer programs and send the instructions of the computer programs to the processor; The processor executes, according to the instructions of the computer program, a transformer fault diagnosis method based on a multimodal self-attention mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements a transformer fault diagnosis method based on a multimodal self-attention mechanism as described in any one of claims 1-7.

Images