Transformer fault prediction method based on multi-modal space-time diagram neural network
By integrating multi-source transformer data through a multimodal spatiotemporal graph neural network, the problem of insufficient diagnostic accuracy caused by a single data source is solved, enabling accurate prediction and graded early warning of transformer faults, and improving the intelligent operation and maintenance level of the power system.
Patent Information
- Application Number
- CN202511734393.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-24
- Publication Date
- 2026-03-06
AI Technical Summary
Existing transformer fault diagnosis methods rely on a single data source, which makes it difficult to fully reflect the operating status. Furthermore, they lack modeling of the spatiotemporal coupling relationship and long-term time-series dependency relationship of multimodal data, resulting in limited diagnostic accuracy and delayed response.
A multimodal spatiotemporal graph neural network is adopted to integrate dissolved gas concentration, electrical operating parameters and vibration spectrum data in transformer oil. Spatiotemporal features are captured through feature weighted fusion and Transformer architecture. Combined with graph neural network, fault reasoning and risk assessment are performed to trigger graded early warning.
It has achieved multimodal data integration and spatiotemporal dependency capture of transformer faults, improved fault prediction accuracy, optimized power equipment maintenance and scheduling, and reduced power outage risk.
Smart Images

Figure CN121614724A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of deep learning and power equipment fault diagnosis, and more specifically, to a transformer fault prediction method based on a multimodal spatiotemporal graph neural network. Background Technology
[0002] As a core component of the power system, the operating status of transformers directly affects the stability of the power grid and the reliability of power supply. With the continuous expansion of the power system and the improvement of its intelligence level, the need for early warning and accurate diagnosis of transformer faults is becoming increasingly urgent.
[0003] Dissolved gas analysis in oil is a common technique for transformer fault diagnosis. By detecting the concentration of gases in the oil, the fault type is determined in conjunction with the Duval triangle or the IEC 60599 standard. However, this method relies on a single data source, making it difficult to comprehensively reflect the transformer's operating status, and its diagnostic accuracy for complex faults is limited.
[0004] In recent years, machine learning technology has been applied to transformer fault diagnosis, achieving fault classification through training on historical data. However, traditional machine learning methods typically only process single-modal data, neglecting the spatiotemporal coupling relationship between gas concentration, electrical parameters, and vibration signals, resulting in insufficient feature extraction and limited prediction accuracy. Furthermore, existing methods lack the ability to model long-term temporal dependencies, making it difficult to capture the dynamic trends of fault development.
[0005] In response to the above situation, there is an urgent need for a transformer fault prediction method that can integrate multimodal data, capture spatiotemporal features, and realize dynamic fault reasoning and hierarchical early warning, so as to improve the intelligence level of power equipment operation and maintenance, optimize maintenance scheduling, and ensure the safe and stable operation of the power grid. Summary of the Invention
[0006] The purpose of this application is to address the issues of limited data and delayed response in traditional transformer fault early warning methods, to provide reliable technical support for intelligent transformer operation and maintenance, and to help power system operation and maintenance teams to more rationally schedule maintenance and repair times. The technical solution of this application is: to provide a transformer fault prediction method based on a multimodal spatiotemporal graph neural network, the method comprising: Step 1: Obtain dissolved gas concentration, electrical operating parameters, and vibration spectrum data in transformer oil. The dissolved gas concentration in transformer oil is acquired using a GC-PDHID gas chromatograph, and the electrical operating parameters and vibration spectrum data are acquired using corresponding sensors. Step 2 involves preprocessing the dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristic data collected in Step 1. This includes: filling in missing values at different time points for any modal data caused by sensor failure, communication interruption, or inconsistent sampling periods using linear interpolation; and detecting and removing or replacing outliers that significantly deviate from the normal range in the three types of multimodal data sequences (dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristic data) due to noise interference or transient faults using the Z-Score method. Step 3: The preprocessed dissolved gas concentration data, electrical operating parameter data, and vibration spectrum feature data are integrated into a unified feature matrix using feature weighted fusion technology to obtain multimodal features; Step 4: Using each multimodal feature dimension as a node, dynamically construct the spatiotemporal feature map G using a sliding time window. t =(V,E,F t The algorithm uses a learnable two-layer perceptron to nonlinearly fuse the absolute values of the Pearson correlation coefficients and the maximum mutual information coefficients of features i and j within the same window to generate initial edge weights. An adaptive thresholding strategy is then employed for sparsification to obtain the dynamic adjacency matrix A. t , to serve as a feature map that evolves in real time with the device state, where V is the node set, E is the edge set, and the The characteristic matrix at time t; Step 5: Using the Transformer architecture, the long-term temporal dependencies of the preprocessed dissolved gas concentration data, electrical operating parameter data, and vibration spectrum feature data are captured through a multi-head attention mechanism, and the deep spatiotemporal features in the spatiotemporal feature map are extracted. Step 6: On the transformer fault knowledge graph constructed based on historical data and domain knowledge, the deep spatiotemporal features extracted in Step 5 are injected as initial evidence into relevant symptom nodes, and a multi-layer graph neural network is used for message passing and reasoning to obtain the activation probability of various fault types and complete the risk assessment. Step 7: Trigger a tiered early warning response based on the risk assessment results.
[0007] In any of the above technical solutions, further, in step 1, the dissolved gas concentration in the oil includes the concentrations of seven gases: hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), acetylene (C2H2), carbon monoxide (CO), and carbon dioxide (CO2); the electrical operating parameters include load rate, winding temperature, oil temperature, and current harmonic content; the vibration spectrum data includes 128-dimensional spectral characteristics in the frequency band from 1kHz to 10kHz.
[0008] In any of the above technical solutions, further, in step 2, for missing values of any modal data at certain time points caused by sensor failure, communication interruption, or inconsistent sampling periods, linear interpolation is used to fill in the missing values at certain time points. Missing data, given known points and The values are respectively and : ; in, For time The interpolation result.
[0009] In any of the above technical solutions, further, in step 2, outliers that significantly deviate from the normal range due to noise interference or transient faults in the three types of multimodal data sequences—dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristic data—are identified and removed using the Z-Score method to ensure data quality. For data points... The Z-Score calculation formula is: ; in, The mean of the data series; Let be the standard deviation of the data series; if If the value is not found, the point is considered an outlier and is replaced using linear interpolation. Adjustments will be made based on the actual situation.
[0010] In any of the above technical solutions, further, in step 3, the preprocessed dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristic data are integrated into a unified feature matrix to obtain multimodal features, including: performing min-max normalization on each type of data, mapping it to the range [0,1]. ; Combine features into a matrix Where T is the number of time steps and D is the total number of features: ; Where G is the normalized gas concentration characteristic; E is the normalized electrical parameter characteristic; and V is the normalized vibration spectrum characteristic. The modal weights are denoted as , and their sum is 1.
[0011] In any of the above technical solutions, further, in step 4, the generated spatiotemporal feature map Where V is the set of nodes and E is the set of edges. The characteristic matrix at time t; the dynamic adjacency matrix. The edge weight calculation is as follows: within the sliding time window, first calculate the absolute value of the Pearson correlation coefficient between feature i and feature j. and maximum mutual information coefficient Then, the two are concatenated and input into a two-layer perceptron for learnable nonlinear fusion to obtain the initial edge weights. Subsequently, an adaptive threshold was adopted. right Perform sparsification, retaining only The edges are used to generate a dynamic adjacency matrix. , The elements in the table are the initial edge weights. .
[0012] In any of the above technical solutions, further, in step 5, the spatiotemporal feature map is... Flattened into a sequence of feature vectors, these are used as input to the Transformer for deep spatiotemporal features. ,in For output dimensions; ; in, These are the query, key, and value matrices, respectively, generated from the input features; Given the dimension of the key vector, the multi-head mechanism is as follows: ; in, h represents the number of attention heads.
[0013] In any of the above technical solutions, further, in step 6, fault types are represented by nodes, and causal or correlational relationships are represented by edges. A knowledge graph is constructed based on historical data and domain knowledge. The historical data includes oil chromatography, electrical, and vibration historical data. Fault types in the domain knowledge include partial discharge, low-energy discharge, high-energy discharge, low-temperature overheating, medium-temperature overheating, high-temperature overheating, and insulation aging. Symptom nodes include the concentrations of 7 key gases (H2, CH4, C2H6, C2H4, C2H2, CO, CO2), the ratios of 5 commonly used gases (CH4 / H2, C2H4 / C2H6, C2H2 / C2H4, C2H2 / H2, C2H2 / C2H6), the three coordinates of the Dewar's triangle, abnormal load rate, abnormal oil temperature, abnormal winding temperature, abnormal current harmonic content, abnormal vibration energy of 1-2kHz, abnormal vibration energy of 2-5kHz, and abnormal vibration energy of 5-10kHz. The inference part uses a graph neural network (GNN) to map features Z to fault types. ; in, For the first l Layer node embedding; This is the normalized adjacency matrix of the knowledge graph; The weight matrix is a learnable matrix; This is the activation function.
[0014] In any of the above technical solutions, further, in step 7, the risk score for each fault type is calculated through a softmax layer: ; in, For the feature representation of fault type i, define the risk thresholds as follows: low risk range is P<0.3, medium risk range is 0.3≤P≤0.7, and high risk range is P≥0.7.
[0015] The beneficial effects of this application are: The technical solution in this application acquires dissolved gas concentration, electrical operating parameters, and vibration spectrum data in transformer oil; it supplements missing data using linear interpolation and removes outliers using the Z-Score method; it integrates multi-source data into a unified feature matrix through feature weighted fusion technology; it dynamically establishes a node association network to quantify the spatiotemporal coupling relationship between gas concentration, electrical parameters, and vibration signals, generating a feature map characterizing the equipment status. Employing a Transformer architecture, it captures long-term temporal dependencies of multi-source data through a multi-head attention mechanism, extracting deep spatiotemporal features; it performs fault type inference and risk assessment based on a fault knowledge graph; and it triggers a graded early warning response based on the risk assessment results. Specific technical effects are as follows: (1) Integrate multimodal data (gas, electrical, vibration) to achieve comprehensive analysis; (2) Use graph neural networks and Transformers to capture spatiotemporal dependencies and improve prediction accuracy; (3) Dynamic knowledge graph and hierarchical early warning optimize maintenance and scheduling to reduce the risk of power outages. Attached Figure Description
[0016] The advantages of the above and / or additional aspects of this application will become apparent and readily understood in the description of the embodiments in conjunction with the following drawings, wherein: Figure 1 This is a schematic flowchart of a transformer fault prediction method based on a multimodal spatiotemporal graph neural network according to this application; Figure 2 This is a schematic diagram of a Transformer structure according to an embodiment of this application; Figure 3 This is a schematic diagram of a multi-head attention mechanism structure according to this application; Detailed Implementation To better understand the above-mentioned objectives, features, and advantages of this application, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other.
[0017] In the following description, many specific details are set forth in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of this application is not limited to the specific embodiments disclosed below.
[0018] like Figure 1 As shown, this embodiment provides a transformer fault prediction method based on a multimodal spatiotemporal graph neural network. The system includes: Step 1: Obtain dissolved gas concentration, electrical operating parameters, and vibration spectrum data in transformer oil. The dissolved gas concentration in transformer oil is acquired using a GC-PDHID gas chromatograph, and the electrical operating parameters and vibration spectrum data are acquired using corresponding sensors. Specifically, the dissolved gas concentration in the oil includes the concentrations of seven gases: hydrogen (H2), methane (CH4), ethane (C2H6), ethylene (C2H4), acetylene (C2H2), carbon monoxide (CO), and carbon dioxide (CO2); electrical operating parameters include load rate, winding temperature, oil temperature, and current harmonic content; vibration spectrum data includes 128-dimensional spectral characteristics in the frequency band from 1 kHz to 10 kHz.
[0019] Step 2 involves preprocessing the collected data on dissolved gas concentration in transformer oil, electrical operating parameters, and vibration spectrum characteristics. This includes: filling in missing values at different time points in any modal data due to sensor failure, communication interruption, or inconsistent sampling periods using linear interpolation; and detecting and removing or replacing outliers that significantly deviate from the normal range in the three types of multimodal data sequences (dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristics) due to noise interference or transient faults using the Z-Score method.
[0020] Specifically, for missing values of any modal data at certain time points caused by sensor failure, communication interruption, or inconsistent sampling periods, linear interpolation is used to fill in the missing values. Missing data, given known points and The values are respectively and : ; in, For time The interpolation result.
[0021] For three types of multimodal data sequences—dissolved gas concentration data, electrical operating parameter data, and vibration spectrum characteristic data—significant outliers deviating from the normal range due to noise interference or transient faults are identified and removed using the Z-Score method to ensure data quality. For data points... The Z-Score calculation formula is: ; in, The mean of the data series; Let be the standard deviation of the data series; if If the value is not found, the point is considered an outlier and is replaced using linear interpolation. Adjustments will be made based on the actual situation.
[0022] Step 3: The preprocessed dissolved gas concentration data, electrical operating parameter data, and vibration spectrum feature data are integrated into a unified feature matrix using feature weighted fusion technology to obtain multimodal features.
[0023] Specifically, through feature-weighted fusion technology, multimodal data such as gas concentration, electrical parameters, and vibration signals are integrated into a unified feature matrix. Min-max normalization is applied to each data type, mapping it to the range [0,1]. ; Combine features into a matrix Where T is the number of time steps and D is the total number of features: ; Where G is the normalized gas concentration characteristic; E is the normalized electrical parameter characteristic; and V is the normalized vibration spectrum characteristic. The modal weights are denoted as , and their sum is 1.
[0024] Step 4: Using each multimodal feature dimension as a node, dynamically construct a spatiotemporal feature map using a sliding time window. The method involves using a learnable two-layer perceptron to nonlinearly fuse the absolute values of the Pearson correlation coefficients and the maximum mutual information coefficients of features i and j within the same window to generate initial edge weights. An adaptive thresholding strategy is then employed for sparsification to obtain a dynamic adjacency matrix. , to serve as a feature map that evolves in real time with the device state, where V is the node set, E is the edge set, and the Let be the characteristic matrix at time t.
[0025] Specifically, the generated spatiotemporal feature map Where V is the set of nodes and E is the set of edges. The characteristic matrix at time t; the dynamic adjacency matrix. The edge weight calculation is as follows: within the sliding time window, first calculate the absolute value of the Pearson correlation coefficient between feature i and feature j. and maximum mutual information coefficient Then, the two are concatenated and input into a two-layer perceptron for learnable nonlinear fusion to obtain the initial edge weights. Subsequently, an adaptive threshold was adopted. right Perform sparsification, retaining only The edges are used to generate a dynamic adjacency matrix. , The elements in the table are the initial edge weights. .
[0026] Furthermore, in step 4, to overcome the shortcomings of traditional fixed thresholds or single correlation measures that cannot adapt to different operating conditions, this embodiment of the invention adopts a dynamic edge weight generation method based on learnable nonlinear fusion, the specific process of which is as follows: Within a sliding time window of length 144 (corresponding to 24 hours), the Pearson correlation coefficient matrix of all feature pairs is calculated for the normalized multimodal feature matrix. and the maximum mutual information coefficient matrix ; then Input a two-layer MLP, where the first layer has 64 units activated by ReLU and the second layer has 1 unit activated by Sigmoid; output the initial edge weight matrix. Then, an adaptive threshold is used. Sparsification is performed to obtain the dynamic adjacency matrix A.
[0027] Step 5: Using the Transformer architecture, the long-term temporal dependencies of the preprocessed dissolved gas concentration data, electrical operating parameter data, and vibration spectrum feature data are captured through a multi-head attention mechanism, and the deep spatiotemporal features in the spatiotemporal feature map are extracted.
[0028] like Figure 2 , Figure 3 As shown. Specifically, the feature map Flattened into a sequence of feature vectors, this serves as the input to the Transformer, with a time step T = 144 (corresponding to 24 hours, gas data sampled every 10 minutes) for deep spatiotemporal features. ,in For output dimensions; ; in, These are the query, key, and value matrices, respectively, generated from the input features; Given the dimension of the key vector, the multi-head mechanism is as follows: ; in, h represents the number of attention heads. A 4-layer Transformer encoder with 8 attention heads, each with a dimension of 64, is used, and positional encoding is added to preserve the time series order.
[0029] Step 6: On the transformer fault knowledge graph constructed based on historical data and domain knowledge, the deep spatiotemporal features extracted in Step 5 are injected as initial evidence into relevant symptom nodes, and a multi-layer graph neural network is used for message passing and reasoning to obtain the activation probability of various fault types and complete the risk assessment.
[0030] Specifically, nodes represent fault types, and edges represent causal or correlational relationships. A knowledge graph is constructed based on historical data and domain knowledge. The fault types include: partial discharge, low-energy discharge, high-energy discharge, low-temperature overheating, medium-temperature overheating, high-temperature overheating, and insulation aging. The inference part uses a graph neural network (GNN) to map features Z to fault types. ; in, For the first l Layer node embedding; This is the normalized adjacency matrix of the knowledge graph; The weight matrix is a learnable matrix; This is the activation function.
[0031] Step 7: Trigger a tiered early warning response based on the risk assessment results.
[0032] Specifically, the risk score for each fault type is calculated using the softmax layer: ; in, For the feature representation of fault type i, define the risk thresholds as follows: low risk range is P<0.3, medium risk range is 0.3≤P≤0.7, and high risk range is P≥0.7.
[0033] The steps in this application can be rearranged, combined, or deleted according to actual needs.
[0034] The units in the device of this application can be merged, divided, or deleted according to actual needs.
[0035] Although this application has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and not intended to limit the application of this application. The scope of protection of this application is defined by the appended claims and may include various variations, modifications, and equivalents of the invention without departing from the scope and spirit of this application.
Claims
1. A transformer fault prediction method based on a multi-modal spatio-temporal graph neural network, characterized in that, The method comprises: Step 1, obtaining the concentration of dissolved gas in transformer oil, electrical operation parameters and vibration spectrum data, wherein the concentration of dissolved gas in transformer oil is collected by GC-PDHID gas chromatograph, and the electrical operation parameters and vibration spectrum data are collected by corresponding sensors; Step 2, preprocessing the dissolved gas concentration data, electrical operation parameter data and vibration spectrum feature data collected in step 1, including: for any missing value of modal data at different time points caused by sensor failure, communication interruption or inconsistent sampling period, linear interpolation method is used for filling; for the abnormal values of the dissolved gas concentration data, the electrical operation parameter data and the vibration spectrum feature data in the three types of multi-modal data sequences due to noise interference or transient fault, Z-Score method is used for detection and elimination or replacement; Step 3, integrating the preprocessed dissolved gas concentration data, electrical operation parameter data and vibration spectrum feature data into a unified feature matrix by feature weighting fusion technology to obtain multi-modal features; Step 4, taking each multimodal feature dimension as a node, a sliding time window is used to dynamically construct a spatiotemporal feature graph Wherein, the absolute value of the Pearson correlation coefficient and the maximum mutual information coefficient of the features i and j in the same window are nonlinearly fused by a learnable two-layer perception machine to generate an initial edge weight, and an adaptive threshold strategy is used for sparse processing to obtain a dynamic adjacency matrix A t As a feature map evolving in real time with the device state, the V is a node set, the E is an edge set, the is a feature matrix of time t; Step 5, using the Transformer architecture, capturing the long time sequence dependence of the preprocessed dissolved gas concentration data, electrical operation parameter data and vibration spectrum feature data through multi-head attention mechanism, and extracting deep space-time features in the space-time feature map; Step 6, on the transformer fault knowledge graph constructed based on historical data and domain knowledge, the deep space-time features extracted in step 5 are injected into the relevant symptom nodes as initial evidence, and a multi-layer graph neural network is used for message passing and reasoning to obtain the activation probability of each type of fault and complete risk assessment; Step 7, triggering a hierarchical early warning response according to the risk assessment result.
2. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network according to claim 1, wherein, In step 1, the dissolved gas concentration in oil includes the concentrations of hydrogen, methane, ethane, ethylene, acetylene, carbon monoxide and carbon dioxide; the electrical operation parameters include load rate, winding temperature, oil temperature and current harmonic content; the vibration spectrum data includes 128-dimensional spectrum features in the frequency band of 1kHz to 10kHz.
3. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 2, for any missing values of modal data at some time points due to sensor failure, communication interruption or inconsistent sampling period, linear interpolation method is used to fill in the missing values. For missing data at time point , the values of given points and are and respectively: ; wherein is the interpolation result of time .
4. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 2, the Z-Score method is used to identify and eliminate the abnormal values in the three types of multi-modal data sequences, i.e., the dissolved gas concentration data, the electrical operating parameter data, and the vibration spectrum feature data, which deviate significantly from the normal range due to noise interference or transient faults. For data points , the Z-Score calculation formula is as follows: ; where, is the mean of the data sequence; is the standard deviation of the data sequence; if , the point is considered as an outlier and is replaced by linear interpolation, Adjust according to the actual situation.
5. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 3, the preprocessed dissolved gas concentration data, electrical operation parameter data and vibration spectrum feature data are integrated into a unified feature matrix to obtain multi-modal features, including: minimum-maximum normalization is performed on each type of data to map to the range of [0, 1]: ; Combining features into a matrix where T is the number of time steps and D is the total number of features: ; where G is a normalized gas concentration feature; E is a normalized electrical parameter feature; and V is a normalized vibrational spectrum feature; are the modal weights, and 1.
6. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 4, the generated spatiotemporal feature map , wherein V is a node set, E is an edge set, is a feature matrix of time t; a dynamic adjacency matrix The edge weight calculation is specifically as follows: in the sliding time window, first, the absolute value of the Pearson correlation coefficient of features i and j and the maximum mutual information coefficient are calculated. Then, the two are spliced and input into a two-layer perception machine for learnable nonlinear fusion to obtain an initial edge weight ; subsequently, an adaptive threshold is used to perform sparse processing on , and only the edges of are retained to generate a dynamic adjacency matrix , , wherein the elements in are initial edge weights .
7. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 5, the space-time feature map is flattened into a sequence of feature vectors as the input of the Transformer, and the deep space-time feature is flattened into a sequence of feature vectors as the input of the Transformer, and the deep space-time feature wherein is the output dimension; ; wherein, are query, key, and value matrices generated from input features, respectively; is the dimension of the key vectors, and the multi-head mechanism is: ; wherein, h is the number of attention heads.
8. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 6, the node represents the fault type, and the edge represents the causal or correlation relationship, and a knowledge graph is constructed based on historical data and domain knowledge, wherein the historical data includes oil chromatogram, electrical, and vibration historical data; the fault types in the domain knowledge include partial discharge, low-energy discharge, high-energy discharge, low-temperature overheating, medium-temperature overheating, high-temperature overheating, and insulation aging; the symptom nodes include seven kinds of key gas concentrations, five kinds of commonly used gas ratios, three coordinates of the Duval triangle, load rate abnormalities, oil temperature abnormalities, winding temperature abnormalities, current harmonic content abnormalities, 1-2 kHz vibration energy abnormalities, 2-5 kHz vibration energy abnormalities, and 5-10 kHz vibration energy abnormalities; and the reasoning part uses a graph neural network (GNN) to map the features Z to the fault type: ; wherein, is the l node embedding of the is the normalized adjacency matrix of the knowledge graph; is the learnable weight matrix; is the activation function.
9. The transformer fault prediction method based on the multi-modal spatio-temporal graph neural network of claim 1, wherein, In step 7, the risk scores of various fault types are calculated through a softmax layer: ; wherein, For the feature representation of the fault type i, define the risk threshold: the low-risk range is P < 0.3, the medium-risk range is: 0.3≤P≤0.7, and the high-risk range is: P≥0.7.