A Transformer Fault Diagnosis Method Based on Multimodal Collaboration and Triplet Metric Learning

Abstract

This invention discloses a transformer fault diagnosis method based on multimodal collaboration and triplet metric learning. The method collects acoustic signals, vibration signals, and temperature time-series signals from the transformer during operation, and performs synchronization alignment and two-dimensional input representation transformation to obtain a three-modal two-dimensional input representation. A DenseNet-Transformer collaborative feature extraction network is used to obtain a deep collaborative feature map of the three modes. A Poisson gradient domain fusion method is employed, using the vibration mode deep collaborative feature map as the background field, and injecting the acoustic anomaly gradient and temperature time-series gradient into the background field to obtain a cross-modal joint feature map. Online hard sample mining, triplet metric learning, cross-entropy loss, and fourth-order moment distribution constraints are used to construct a joint loss function for training and optimization, outputting the transformer fault diagnosis result. This invention can enhance the consistency and discriminative power of multimodal fault feature fusion, improving the accuracy and robustness of transformer fault diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of transformer condition fault diagnosis technology, specifically involving a transformer fault diagnosis method based on multimodal collaboration and triplet metric learning. Background Technology

[0002] Transformers are critical equipment in power systems, and their operating status directly affects the reliability and security of power supply. During long-term operation, transformers may experience faults or abnormal conditions such as loose windings, core abnormalities, partial discharge, overheating, and mechanical shock. Failure to identify these faults in a timely manner can lead to insulation degradation, decreased operating efficiency, and even power outages. Therefore, accurate, stable, and timely fault diagnosis of transformer operating conditions is of paramount importance.

[0003] Existing transformer fault diagnosis methods typically rely on single-mode data analysis. For example, they may use acoustic signals to identify partial discharge or abnormal noise, vibration signals to identify mechanical loosening or impact anomalies, or temperature time-series signals to determine overheating trends. However, single-mode data is easily affected by the acquisition environment, noise interference, changes in operating conditions, and sensor placement, leading to insufficient expression of fault characteristics and instability in diagnostic results.

[0004] With the development of multi-source sensing technology, comprehensive diagnosis using multimodal data such as acoustics, vibration, and temperature has become an important direction for improving transformer fault identification capabilities. However, existing multimodal fusion methods mostly employ simple splicing, weighted fusion, or attention fusion approaches, which struggle to fully preserve the main mechanical structure reflected by the vibration modes and effectively incorporate acoustic anomalies and temporal temperature variations into the fused features. Furthermore, existing classification models typically rely primarily on cross-entropy loss for training, which fails to adequately constrain intra-class compactness, inter-class separability, and the stability of higher-order distributions of similar fault features in the fused feature space. This results in decreased diagnostic performance under noisy samples, similar fault categories, or modal perturbations.

[0005] Therefore, it is necessary to propose a new transformer fault diagnosis method. While ensuring the temporal consistency of the three-modal data, this method improves the fusion consistency, class discrimination, and diagnostic robustness of multimodal fault features through deep collaborative feature extraction, Poisson gradient domain fusion, and joint loss optimization. Summary of the Invention

[0006] To address the shortcomings of existing transformer fault diagnosis methods, such as insufficient single-modal feature representation, weak consistency in multimodal feature fusion, insufficient class discrimination in the fused feature space, and poor diagnostic stability under noise disturbances, this invention proposes a transformer fault diagnosis method based on multimodal collaboration and triplet metric learning. This method achieves accurate identification of transformer fault types by synchronously aligning acoustic signals, vibration signals, and temperature time-series signals, extracting collaborative features, fusing them in the Poisson gradient domain, and optimizing the joint loss.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A transformer fault diagnosis method based on multimodal collaboration and triplet metric learning includes the following steps: S1: Acquire acoustic signals, vibration signals, and temperature timing signals during transformer operation, and perform synchronization alignment and conversion processing to obtain acoustic two-dimensional input characterization, vibration two-dimensional input characterization, and temperature timing two-dimensional input characterization; S2: The acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are respectively input into a three-way DenseNet-Transformer collaborative feature extraction network. Local feature extraction is performed through the DenseNet network, and global dependency modeling and gated adaptive aggregation processing are performed through the Transformer network to obtain acoustic modal deep collaborative feature maps, vibration modal deep collaborative feature maps, and temperature time-series modal deep collaborative feature maps. S3: The cross-modal fusion module employs the Poisson gradient domain fusion method to perform cross-modal fusion on the acoustic modal deep collaborative feature map, vibration modal deep collaborative feature map, and temperature time-series modal deep collaborative feature map. The vibration modal deep collaborative feature map is used as the background field. Gradient information representing acoustic anomaly modes in the acoustic modal deep collaborative feature map and gradient information representing temperature rise trends and time-series fluctuation characteristics in the temperature time-series modal deep collaborative feature map are injected into the background field as guiding information to construct a cross-modal joint guiding field. By solving the Poisson fusion equation, the gradient of the fused cross-modal joint feature map is made consistent with the cross-modal joint guiding field, thus obtaining the cross-modal joint feature map. S4: The cross-modal fusion module, the three-way DenseNet-Transformer collaborative feature extraction network, the fusion feature mapping layer, and the classification mapping layer together form the fault diagnosis model. In the fusion feature space output by the fusion feature mapping layer, online hard sample mining and triplet metric learning methods are used to construct feature triplets. Based on the feature triplets, the triplet metric loss is calculated, and a joint loss function is constructed by combining cross-entropy classification loss and fourth-order moment distribution constraint loss. The fault diagnosis model is trained and optimized to obtain the optimal fault diagnosis model, which is used for transformer fault diagnosis and outputs the diagnosis results.

[0008] Furthermore, step S1 specifically includes: S11: Using a timestamped data acquisition method, acoustic signals, vibration signals, and temperature time-series signals during transformer operation are obtained to acquire the raw time-series data of the three modes; S12: Using a unified timestamp correction method, the acoustic signal, vibration signal and temperature time series signal are processed to unify the time coordinates, and the acquisition time of the vibration signal is used as a reference to obtain three-modal data under a unified time coordinate system. S13: The vibration reference sliding time window division method is adopted to divide the vibration signal under the unified time coordinate system into multiple vibration reference time windows, and each vibration reference time window is used as the synchronization alignment boundary of acoustic signal, vibration signal and temperature time sequence signal. S14: Using the time window matching method, the acoustic signal and temperature time series signal are matched to the corresponding vibration reference time window according to the start and end times of each vibration reference time window, so as to obtain the acoustic signal segment, vibration signal segment and temperature time series segment under the same time window; S15: Using time interpolation, resampling, and proximity-preserving completion methods, the acoustic signal segment, vibration signal segment, and temperature time series segment are sampled scale matched to obtain acoustic synchronization sequence, vibration synchronization sequence, and temperature synchronization sequence within the same time window. S16: Using the sample binding method, the acoustic synchronization sequence, vibration synchronization sequence and temperature synchronization sequence within the same vibration reference time window are bound according to the same sample number to obtain a three-mode time-aligned data segment with the vibration signal as the time reference. S17: A two-dimensional input representation conversion method is adopted to perform two-dimensional processing on the three-modal time-aligned data segments according to the data type of each modality. Specifically, the acoustic synchronization sequence is processed by short-time Fourier transform and Mel-filter mapping to obtain the acoustic two-dimensional input representation; the vibration synchronization sequence is processed by short-time Fourier transform to obtain the vibration two-dimensional input representation; and the temperature synchronization sequence is processed by sliding subsequence stacking to obtain the temperature time-series two-dimensional input representation. S18: Using a size normalization method, the acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are subjected to scale resampling, boundary clipping, zero-value filling, and channel dimension normalization processing to unify the three two-dimensional input representations into a preset input size and preset channel form, thus obtaining a three-modal two-dimensional input representation group.

[0009] Furthermore, step S2 specifically includes: S21: The DenseNet network is used to perform local feature reuse and convolutional feature extraction on the acoustic two-dimensional input representation, vibration two-dimensional input representation and temperature time-series two-dimensional input representation respectively, so as to extract the local texture anomaly features in the acoustic two-dimensional input representation, the local time-frequency impact features in the vibration two-dimensional input representation and the local temperature rise trend features in the temperature time-series two-dimensional input representation, and obtain the local feature map of the corresponding mode. S22: The local feature map extracted by the DenseNet network and the two-dimensional input representation guidance feature of the corresponding modality are jointly input into the Transformer network. Global correlation features in the two-dimensional representation space are obtained through position encoding and self-attention modeling, so that acoustic anomaly mode, vibration response mode and temperature time series change mode can form global dependency expression in their respective modes, and a global dependency feature map is obtained. S23: Perform gated adaptive aggregation processing on the local feature map, global dependency feature map, and two-dimensional input representation guidance features. Based on the local feature contribution and global dependency contribution of different modes under the current vibration reference time window, adaptively adjust the fusion weight of local features and global features to obtain the deep collaborative feature map of the corresponding mode.

[0010] Furthermore, step S3 specifically includes: S31: The vibration mode depth co-feature map used to characterize the mechanical structure state, transient impact, and mechanical disturbance distribution of the transformer is used as the background field for Poisson gradient domain fusion; the feature map spatial region corresponding to the background field is defined, and the spatial position of the feature map spatial region located on the periphery of the feature map is defined as the boundary region; the features of the background field on the boundary region are used as the boundary constraints in the Poisson gradient domain fusion process to provide the main mechanical structure and boundary preservation conditions for the subsequent cross-modal joint feature map; S32: Perform spatial gradient mapping on the acoustic modal depth co-feature map, background field, and temperature time-series modal depth co-feature map respectively to obtain the corresponding gradient tensors; perform gradient magnitude normalization on the gradient tensors to obtain acoustic gradient features, vibration background gradient features, and temperature time-series gradient features. S33: Construct an acoustic residual gradient guiding term based on the difference between the acoustic gradient features and the vibration background gradient features; construct a temperature time-series residual gradient guiding term based on the difference between the temperature time-series gradient features and the vibration background gradient features. S34: Construct a cross-modal joint guiding field based on the vibration background gradient characteristics, acoustic residual gradient guiding term, and temperature time-series residual gradient guiding term; S35: Using the cross-modal joint guiding field as the target gradient field and the background field boundary features as the boundary preservation condition, establish the Poisson fusion equation; S36: Discretize the Poisson fusion equation and combine it with the background field boundary constraints to organize the discretized Poisson fusion equation into a system of linear equations. S37: The linear equation system is numerically solved using an iterative solution method to obtain the cross-modal joint feature vectors corresponding to each channel. The cross-modal joint feature vectors corresponding to each channel are then restored according to their spatial positions in the feature map to obtain the cross-modal joint feature map.

[0011] Further optimization, in step S34, the cross-modal joint guiding field is constructed according to the following formula: ; in, This represents the cross-modal joint steering field corresponding to the k-th vibration reference time window. These are identifiers representing acoustic, vibration, and temperature time sequences, respectively. Indicates the characteristics of the vibration background gradient. This represents the acoustic residual gradient guiding term. This represents the temperature-series residual gradient guiding term. This represents the injection weights of the acoustic residual gradient guiding term. The injection weights represent the temperature time-series residual gradient guiding terms; the acoustic residual gradient guiding terms are constructed based on the differences between the acoustic gradient features and the vibration background gradient features, and the temperature time-series residual gradient guiding terms are constructed based on the differences between the temperature time-series gradient features and the vibration background gradient features.

[0012] Further preferred, the injection weights of the acoustic residual gradient guiding term and the injection weights of the temperature time-series residual gradient guided term satisfy: ; Furthermore, the injection weight is dynamically adjusted based on prior fault knowledge within the current vibration reference time window: it increases when there is a tendency for partial discharge. Increase when there is a tendency to overheat. .

[0013] Further preferred, in step S32, the spatial gradient mapping adopts the central difference method, and the one-sided difference method is adopted in the boundary region of the feature map; the gradient magnitude normalization process adopts the L2 normalization method, and the gradient tensor of each channel is normalized channel by channel.

[0014] Further preferred, the fourth-order moment distribution constraint loss in step S4 is obtained by: constructing the fourth-order central moment based on the fusion feature vector of the same fault category in the current training batch, and calculating the fourth-order moment distribution constraint loss based on the fourth-order central moment.

[0015] Further optimization involves constructing a transformer fault dataset containing acoustic signals, vibration signals, temperature time-series signals, and corresponding fault type labels. This dataset is then hierarchically and randomly divided according to fault categories to obtain a training set, a validation set, and a test set. The fault diagnosis model is trained using the training set based on the joint loss function. The optimal fault diagnosis model is selected from multiple candidate models obtained during training using the validation set. Finally, the trimodal samples to be diagnosed from the test set are input into the optimal fault diagnosis model, which outputs the transformer fault type. The training process aims to minimize the joint loss function, enabling the fault diagnosis model to simultaneously optimize its fault category prediction capability, the spatial distance structure of the fused feature space, and the stability of the distribution of similar fault features during training.

[0016] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program, implements the transformer fault diagnosis method based on multimodal collaboration and triplet metric learning.

[0017] The beneficial effects of this invention include: First, this invention uses vibration signals as a time reference to synchronize and align acoustic signals and temperature time-series signals, so that the three modal data of acoustic, vibration and temperature time-series correspond to the same transformer operating state time window, thereby improving the consistency of multimodal input data.

[0018] Second, this invention employs a DenseNet-Transformer collaborative feature extraction network, which combines the local feature extraction capability of the DenseNet network with the global dependency modeling capability of the Transformer network, and forms a deep collaborative feature map through gated adaptive aggregation processing, thereby improving the ability to express single-mode fault features.

[0019] Third, the present invention adopts the Poisson gradient domain fusion method, using the vibration mode depth collaborative feature map as the background field and boundary constraint, and injecting acoustic anomaly gradient information and temperature time-series gradient information as residual guiding information into the vibration background gradient field, so that the fused cross-modal joint feature map can absorb the anomaly information of acoustic mode and temperature time-series mode while maintaining the main structure of mechanical state.

[0020] Fourth, this invention employs a joint loss function constructed using triplet metric loss, cross-entropy classification loss, and fourth-order moment distribution constraint loss. This allows the fused feature space to be simultaneously subject to classification prediction constraints, intra-class compactness and inter-class separation constraints, as well as higher-order distribution stability constraints, thereby improving the accuracy and robustness of the transformer fault diagnosis model. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the transformer fault diagnosis method based on multimodal collaboration and triplet metric learning; Figure 2 A schematic diagram illustrating the transformation effect of two-dimensional input representations before and after processing acoustic signals, vibration signals, and temperature time-series signals; Figure 3 This is a schematic diagram of the DenseNet-Transformer collaborative feature extraction network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the cross-modal fusion process in the Poisson gradient domain provided in an embodiment of the present invention; Detailed Implementation

[0022] The present invention will be further described below with reference to embodiments. It should be noted that the following embodiments are only for further illustration of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-essential improvements and adjustments made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

[0023] like Figure 1 As shown, a transformer fault diagnosis method based on multimodal collaboration and triplet metric learning includes the following steps: S1: Acquire acoustic signals, vibration signals, and temperature timing signals during transformer operation, and perform synchronization alignment and conversion processing to obtain acoustic two-dimensional input characterization, vibration two-dimensional input characterization, and temperature timing two-dimensional input characterization; S2: The acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are respectively input into a three-way DenseNet-Transformer collaborative feature extraction network. Local feature extraction is performed through the DenseNet network, and global dependency modeling and gated adaptive aggregation processing are performed through the Transformer network to obtain acoustic modal deep collaborative feature maps, vibration modal deep collaborative feature maps, and temperature time-series modal deep collaborative feature maps. S3: The cross-modal fusion module employs the Poisson gradient domain fusion method to perform cross-modal fusion on the acoustic modal deep collaborative feature map, vibration modal deep collaborative feature map, and temperature time-series modal deep collaborative feature map. The vibration modal deep collaborative feature map is used as the background field. Gradient information representing acoustic anomaly modes in the acoustic modal deep collaborative feature map and gradient information representing temperature rise trends and time-series fluctuation characteristics in the temperature time-series modal deep collaborative feature map are injected into the background field as guiding information to construct a cross-modal joint guiding field. By solving the Poisson fusion equation, the gradient of the fused cross-modal joint feature map is made consistent with the cross-modal joint guiding field, thus obtaining the cross-modal joint feature map. S4: The cross-modal fusion module, the three-way DenseNet-Transformer collaborative feature extraction network, the fusion feature mapping layer, and the classification mapping layer together form the fault diagnosis model. In the fusion feature space output by the fusion feature mapping layer, online hard sample mining and triplet metric learning methods are used to construct feature triplets. Based on the feature triplets, the triplet metric loss is calculated, and a joint loss function is constructed by combining cross-entropy classification loss and fourth-order moment distribution constraint loss. The fault diagnosis model is trained and optimized to obtain the optimal fault diagnosis model, which is used for transformer fault diagnosis and outputs the diagnosis results.

[0024] In this embodiment, step S1 specifically includes: S11: Using a timestamped data acquisition method, acoustic signals, vibration signals, and temperature time-series signals during transformer operation are obtained to acquire the raw time-series data of the three modes; S12: Using a unified timestamp correction method, the acoustic signal, vibration signal and temperature time series signal are processed to unify the time coordinates, and the acquisition time of the vibration signal is used as a reference to obtain three-modal data under a unified time coordinate system. S13: The vibration reference sliding time window division method is adopted to divide the vibration signal under the unified time coordinate system into multiple vibration reference time windows, and each vibration reference time window is used as the synchronization alignment boundary of acoustic signal, vibration signal and temperature time sequence signal. S14: Using the time window matching method, the acoustic signal and temperature time series signal are matched to the corresponding vibration reference time window according to the start and end times of each vibration reference time window, so as to obtain the acoustic signal segment, vibration signal segment and temperature time series segment under the same time window; S15: Using time interpolation, resampling, and proximity-preserving completion methods, the acoustic signal segment, vibration signal segment, and temperature time series segment are sampled scale matched to obtain acoustic synchronization sequence, vibration synchronization sequence, and temperature synchronization sequence within the same time window. S16: Using the sample binding method, the acoustic synchronization sequence, vibration synchronization sequence and temperature synchronization sequence within the same vibration reference time window are bound according to the same sample number to obtain a three-mode time-aligned data segment with the vibration signal as the time reference. S17: A two-dimensional input representation conversion method is adopted to perform two-dimensional processing on the three-modal time-aligned data segments according to the data type of each modality. Specifically, the acoustic synchronization sequence is processed by short-time Fourier transform and Mel-filter mapping to obtain the acoustic two-dimensional input representation; the vibration synchronization sequence is processed by short-time Fourier transform to obtain the vibration two-dimensional input representation; and the temperature synchronization sequence is processed by sliding subsequence stacking to obtain the temperature time-series two-dimensional input representation. S18: Using a size normalization method, the acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are subjected to scale resampling, boundary clipping, zero-value filling, and channel dimension normalization processing to unify the three two-dimensional input representations into a preset input size and preset channel form, thus obtaining a three-modal two-dimensional input representation group.

[0025] In this embodiment, a piezoelectric accelerometer is installed on the transformer tank wall or core clamp, with a sampling frequency of 12.8 kHz, to collect vibration signals. A microphone array is placed 0.5 m away from the transformer casing, with a sampling frequency of 25.6 kHz, to collect acoustic signals. PT100 temperature sensors and fiber optic temperature sensors are respectively arranged at the top oil temperature and winding hot spots of the transformer, with a sampling frequency of 1 Hz, to collect temperature timing signals. All sensors are clock-synchronized via GPS or the IEEE 1588 precision time protocol, and each data point is accompanied by a microsecond-level timestamp.

[0026] Using the vibration signal as a time reference, the length of the sliding time window is set. The sampling point has a window overlap rate of 50%. For the first sampling point... A vibration reference time window, with its start time and end time The sampling time axis of the vibration signal is determined. Then, the timestamps in the acoustic signal and temperature time sequence signal are placed on... Data segments within the range are extracted to obtain acoustic signal segments, vibration signal segments, and temperature time series segments within the same time window.

[0027] Because the acoustic sampling rate frequency is 25.6 kHz and the vibration sampling rate frequency is 12.8 kHz, the acoustic segment needs to be downsampled to 12.8 kHz, or the vibration segment needs to be upsampled to 25.6 kHz. In this embodiment, the acoustic segment is uniformly downsampled to 12.8 kHz to match the vibration signal sampling rate. The temperature time series signal is a low-frequency sequence of 1 Hz, which typically has only one or zero temperature points within a window of approximately 0.32 s. Therefore, the nearest neighbor filling method is used to extend the temperature values ​​within the window to the same length as the vibration sequence. Finally, the first... Acoustic synchronization sequence under a vibration reference time window Vibration synchronization sequence and temperature synchronization sequence To form a three-mode synchronization sequence Each sequence has a length of 4096. These are identifiers representing acoustic, vibration, and temperature timing, respectively.

[0028] Acoustic synchronization sequence Perform a short-time Fourier transform with a window size of 256 points and a step size of 128 points to obtain the time spectrum, which is then mapped through a Mel filter bank. The Mel spectrogram, as a two-dimensional acoustic input representation: ; In the formula, Indicates the feature dimension.

[0029] Vibration synchronization sequence Perform a short-time Fourier transform directly with a window size of 256 points and a step size of 128 points to obtain... The vibration time-frequency spectrum is used as a two-dimensional input representation of vibration: ; Temperature synchronization sequence Containing 4096 points, using a sliding subsequence stacking method: a temperature synchronization sequence of length 4096 is... The system is divided into 8 main time periods, each containing 512 sampling points. Each main time period is further divided into 8 temperature subsequences, each containing 64 sampling points, and the mean of each temperature subsequence is calculated. Then, the mean values ​​of the 8×8 temperature subsequences obtained within the current vibration reference time window are arranged in chronological order to form a two-dimensional temperature time series representation corresponding to the k-th vibration reference time window, as shown below: ; And uniform upsampling is performed using bilinear interpolation. To maintain the same spatial scale as acoustic and vibration characterization, the processing only uses data within the current k-th vibration reference time window, without depending on previous or subsequent windows.

[0030] The three types of two-dimensional representations are clipped or filled respectively, and then unified as follows: , The number of channels is 1 for each channel. The final trimodal two-dimensional input representation set is obtained, expressed as: ; in, Indicates the first The three-mode two-dimensional input characterization set corresponding to each vibration reference time window Indicates the first Acoustic two-dimensional input characterization under a vibration reference time window Indicates the first Two-dimensional input characterization of vibration under a vibration reference time window Indicates the first A two-dimensional input characterization of temperature timing within a vibration reference time window.

[0031] like Figure 2 As shown, this embodiment provides a comparison example of three modal signals before and after processing. The left side shows the original time-domain waveforms: the acoustic signal amplitude ranges from approximately -0.75 to 0.75, with a time interval of 0 to 120 s; the vibration signal amplitude ranges from approximately 0.090 to 0.104, with a time interval of 0 to 120 s; and the temperature time-series signal amplitude ranges from approximately 62.0 to 62.25℃. The right side shows the corresponding two-dimensional input representations: the acoustic two-dimensional input representation is the spectrum after short-time Fourier transform and Mel-filter processing; the vibration two-dimensional input representation is the time-spectrum obtained from short-time Fourier transform; and the temperature time-series two-dimensional input representation is the heatmap obtained from stacked sliding subsequences. Figure 2 This illustrates the correspondence between the original signal and the two-dimensional input representation.

[0032] like Figure 3 As shown, in this embodiment, step S2 includes: S21: The DenseNet network is used to perform local feature reuse and convolutional feature extraction on the acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation, respectively, to extract local texture anomaly features in the acoustic two-dimensional input representation, local time-frequency impact features in the vibration two-dimensional input representation, and local temperature rise trend features in the temperature time-series two-dimensional input representation, to obtain the local feature maps of the corresponding modes; for each mode's two-dimensional input representation ,in The DenseNet network used in this embodiment is a simplified DenseNet-BC (bottleneck + compression) structure, containing 3 dense blocks, each containing 4 convolutional layers, with a growth rate of 16, and a transition layer compression factor of 0.5, outputting local feature maps. The size is .

[0033] S22: The local feature map extracted by the DenseNet network and the two-dimensional input representation guidance feature of the corresponding modality are jointly input into the Transformer network. Global correlation features in the two-dimensional representation space are obtained through position encoding and self-attention modeling, so that acoustic anomaly mode, vibration response mode and temperature time series change mode can form global dependency expression in their respective modes, and a global dependency feature map is obtained. right and Perform feature mapping: First, Through a core size of Convolutional layers compressed to , and then with The tokens are concatenated along the channel dimension and then flattened using Flatten to obtain a token sequence of length 1024. Position encoding employs learnable sine and cosine coding. The Transformer encoder consists of 6 layers, each with 8 attention heads, and the feedforward network has a dimension of 512. The output is a globally dependent feature map. Size is The computational expression for the Transformer global dependency modeling process guided by the two-dimensional input representation is as follows: ; ; in, Indicates the first Two-dimensional input representation of various modalities guides the features. Indicates the first The vibration reference time window under the first Two-dimensional input representation of a modality, Indicates the first The two-dimensional input representation guided mapping function corresponding to each mode. This indicates a batch normalization operation; Represents a non-linear activation function. Indicates the first The corresponding modes Convolutional layer and They represent the first The step size of the first and second layers corresponding to this mode is 2. Convolutional layer Indicates the first The vibration reference time window under the first Global dependency feature maps of various modalities Represents the feature mapping function. Indicates the first The vibration reference time window under the first Local feature maps of various modes Indicates the first Position encoding of the modality This represents the Transformer global dependency modeling mapping for the corresponding modality. This represents the operation of flattening the feature map into a one-dimensional sequence. This represents the network parameters for the corresponding Transformer branch.

[0034] S23: Perform gated adaptive aggregation processing on the local feature map, global dependency feature map, and two-dimensional input representation guidance features. Based on the local feature contribution and global dependency contribution of different modes under the current vibration reference time window, adaptively adjust the fusion weight of local features and global features to obtain the deep collaborative feature map of the corresponding mode. Will , and guiding features Input gating aggregation unit. The gating aggregation unit first processes the three separately. Convolution, then calculate the gating weight vector: ; in, For the gated weight vector, It is the Sigmoid activation function. This is a learnable weight matrix used to calculate the gating weights. Then, the computational expression for the gated adaptive aggregation process guided by the two-dimensional input representation is: ; in, Indicates the first The vibration reference time window under the first Deep collaborative feature maps of various modalities , , represents the gating weights corresponding to the local feature map, the global dependency feature map, and the two-dimensional input representation guided feature, respectively, and ⊙ represents element-wise multiplication.

[0035] Through three parallel processing paths, acoustic modal deep collaborative feature maps are obtained respectively. Vibration mode depth co-feature map Deep co-feature map of temperature time-series modes All sizes The acoustic modal deep collaborative feature map, the vibration modal deep collaborative feature map, and the temperature time-series modal deep collaborative feature map are depth feature maps obtained based on the same vibration reference time window. The three have corresponding position indices and channel representation relationships in the feature map space, which are used for calculating gradients at the same position, background field constraints, and constructing cross-modal joint guiding fields in the subsequent Poisson gradient domain fusion process. like Figure 4 As shown, step S3 in this embodiment specifically includes: S31: The vibration mode depth co-feature map used to characterize the mechanical structural state, transient impact, and mechanical disturbance distribution of the transformer is used as the background field for Poisson gradient domain fusion, so that the cross-modal joint feature map obtained by subsequent fusion is based on the main structure of the mechanical state characterized by the vibration modes. The background field is expressed as: ; in, Indicates the first The background field is fused using the Poisson gradient domain corresponding to each vibration reference time window; Define the feature map spatial region corresponding to the background field, and define the spatial position of the feature map spatial region located on the periphery of the feature map as the boundary region. The features of the background field in the boundary region are used as boundary constraints in the Poisson gradient domain fusion process to provide the mechanical state main structure and boundary preservation conditions for the subsequent cross-modal joint feature map. The feature map spatial region corresponding to the background field is represented as follows: ; in, Represents the spatial region of the feature map. Represents the spatial coordinates of the feature map. and These represent the height and width of the deep collaborative feature map, respectively, both with a value of 32; Boundary constraints are expressed as: ; in, Indicates the first Background field boundary features corresponding to each vibration reference time window This represents the boundary projection operator used to extract features from the boundary region of the feature map. The background field and background field boundary features represent the boundary region of the feature map; the background field and background field boundary features are used to provide the mechanical state main structure and boundary preservation conditions for the subsequent cross-modal joint feature map.

[0036] S32: After determining the vibration mode depth collaborative feature map as the background field for Poisson gradient domain fusion, in order to enable acoustic anomaly changes, vibration main structure changes, and temperature time series changes to participate in the subsequent cross-modal joint guidance field construction within a unified gradient space, the first... The acoustic modal deep co-feature map under each vibration reference time window, the Poisson gradient domain fused background field and temperature time-series modal deep co-feature map are represented as three types of features to be gradientified, as follows:

[0037] in, This represents the acoustic features to be gradient-processed. This represents the vibration background features to be gradient-processed. The temperature time series features to be gradientd are represented; all three originate from the same vibration reference time window and have corresponding spatial location indices and channel representation relationships.

[0038] For any feature to be gradient-processed ,in The spatial gradient of the feature map is calculated using the spatial difference gradient mapping method in the height and width directions, respectively; at the spatial location inside the feature map, the central difference method is used to calculate: ; ; in, For the first The vibration reference time window under the first The height gradient of the mode, For the first The vibration reference time window under the first The width gradient of the modality. Indicates the feature map height orientation index. Indicates the feature map width direction index. The feature channel index is represented. For spatial locations located in the boundary region of the feature map, the spatial gradient in the corresponding direction is calculated using forward one-sided difference or backward one-sided difference methods to ensure that the gradient features at the boundary location can correspond to the aforementioned background field boundary constraints.

[0039] Based on the height gradient and width gradient, construct the first... The vibration reference time window under the first The gradient tensor corresponding to each mode: ; in, Indicates the first The vibration reference time window under the first The gradient tensor of each mode, where , Represents the acoustic modal gradient tensor, used to characterize the local abrupt changes in acoustic anomaly modes in the feature space; Represents the vibration background gradient tensor, used to characterize the changes in the mechanical state of the main structure in the vibration background field; It represents the temperature time-series gradient tensor, used to characterize temperature rise trends, temperature abrupt changes, and time-series fluctuations.

[0040] To reduce the impact of differences in modal feature amplitudes on the subsequent construction of the cross-modal joint guiding field, the gradient tensor of each mode is normalized according to the channel, resulting in normalized gradient features: ; in, Indicates the first The vibration reference time window under the first Normalized gradient features corresponding to each mode and These represent the height and width of the feature map, respectively. To prevent constants with a denominator of zero.

[0041] Finally, the acoustic gradient features are obtained. Vibration background gradient characteristics Temperature time-series gradient characteristics Among them, acoustic gradient features Vibration background gradient features serve as guiding information for acoustic anomaly gradients. Temperature temporal gradient characteristics serve as gradient information for the main vibrating structure. As temperature change gradient guidance information, the three together serve as gradient inputs for the subsequent construction of a cross-modal joint guidance field.

[0042] S33: Using the vibration background gradient features as the main gradient field of the cross-modal joint guiding field, and using the acoustic gradient features and temperature time-series gradient features as anomalous residual guiding information relative to the main gradient field; constructing an acoustic residual gradient guiding term based on the difference between the acoustic gradient features and the vibration background gradient features, and constructing a temperature time-series residual gradient guiding term based on the difference between the temperature time-series gradient features and the vibration background gradient features: ; ; in, This represents the acoustic residual gradient guiding term. This represents the temperature time-series residual gradient guiding term.

[0043] S34: Construct a cross-modal joint guiding field based on the vibration background gradient characteristics, acoustic residual gradient guiding term, and temperature time-series residual gradient guiding term: ; in, Indicates the first The cross-modal joint guiding field corresponding to each vibration reference time window This represents the injection weights of the acoustic residual gradient guiding term. The injected weights represent the temperature-series residual gradient-guided terms, and these injected weights satisfy the following: ; Inject weights and Dynamically adjust based on prior knowledge of faults within the current time window: if there is no particular fault tendency, take... , If partial discharge is known to be possible, then increase... Reduce to 0.8; if overheating is known to be a possibility, increase to 0.8. Up to 0.8.

[0044] The injection weights are used to adjust the intensity of the acoustic anomaly gradient information and temperature time-series gradient information injected into the vibration background gradient field. Thus, the cross-modal joint guiding field takes the vibration background gradient characteristics as the main structural basis, and injects acoustic anomaly mode changes through acoustic residual gradient guiding terms, and injects temperature rise trends and time-series fluctuation changes through temperature time-series residual gradient guiding terms, so that the subsequent Poisson fusion equation can absorb the anomaly gradient information of acoustic modes and temperature time-series modes while maintaining the main structure of the vibration mechanical state.

[0045] S35: Using the cross-modal joint guiding field as the target gradient field and the background field boundary features as the boundary preservation condition, establish the Poisson fusion equation: Let the first The joint feature map of the unsolved cross-modal modes corresponding to each vibration reference time window To ensure that the gradient of the cross-modal joint feature map is consistent with the cross-modal joint guiding field, the Poisson fusion equation is expressed as: ; And satisfy the background field boundary constraints: ;

[0046] in, Represents the Laplace operator. Denotes the divergence operator, Represents the position coordinates within the feature map spatial region. Represents the spatial region of the feature map. Indicates the boundary region of the feature map. Represents the boundary projection operator. Indicates the first Background field boundary features corresponding to each vibration reference time window; S36: Discretize the Poisson fusion equation and, in conjunction with the background field boundary constraints, rearrange the discretized Poisson fusion equation into a system of linear equations: ; in, Let represent the coefficient matrix consisting of the discrete Laplace operator and boundary conditions. Indicates the first The cross-modal joint feature vector to be solved in each channel This represents the right-hand term, which is composed of the cross-modal joint steering field divergence term and the background field boundary features. Indicates the feature map channel index; S37: The linear equation system is numerically solved using an iterative solution method, including initializing the cross-modal joint feature vector to the feature vector of the corresponding channel of the background field, iteratively updating it based on the coefficient matrix and right-hand side, and stopping the iteration when the iteration error meets a preset threshold or the number of iterations reaches a preset number, to obtain the cross-modal joint feature vector corresponding to each channel; the cross-modal joint feature vector corresponding to each channel is restored according to the spatial position of the feature map to obtain the cross-modal joint feature map: ; ; in, This indicates the solution obtained for the first... The cross-modal joint feature vector corresponding to each channel This represents the iterative solution function for a system of linear equations. This indicates an operation that restores the feature map based on its spatial location and channel dimensions. This represents the number of channels in the cross-modal joint feature map.

[0047] The cross-modal joint feature map is based on the mechanical state of the main structure provided by the vibration background field. It absorbs acoustic anomaly gradient information and temperature time-series gradient information through the cross-modal joint guiding field and uses them as inputs for subsequent fusion feature space mapping, triplet metric learning and fault classification.

[0048] Further, step S4 specifically includes: S41: Joint feature maps across modalities Input fusion feature mapping layer: First, global average pooling is performed to obtain a 128-dimensional feature vector; then, a fully connected layer is used to reduce the dimensionality to 64 dimensions to obtain the fusion feature vector. ; There are a total of A number of samples are used to map the cross-modal joint features obtained by Poisson gradient domain fusion to a fusion feature space used for metric learning and fault classification. S42: Using any fused feature vector in the current training batch as an anchor sample, among the samples with the same fault category label as the anchor sample, select the fused feature vector with the largest Euclidean distance as a difficult positive sample; among the samples with different fault category labels as the anchor sample, select the fused feature vector with the smallest Euclidean distance as a difficult negative sample, thereby constructing feature triples, and setting the triple interval to 0.5 to constrain the clustering of similar fault features in the fused feature space and the separation of dissimilar fault features in the fused feature space. S43: Input the fused feature vector into the classification mapping layer to obtain the fault category prediction probability, and calculate the cross-entropy classification loss based on the fault category prediction probability and the true fault category label to constrain the consistency between the fault category prediction result and the true fault category label; wherein, the cross-entropy classification loss adopts the standard cross-entropy. form; S44: Construct fourth-order central moments based on the fused feature vectors of the same fault category in the current training batch, and calculate the fourth-order moment distribution constraint loss based on the fourth-order central moments. The fourth-order central moment is used to characterize the higher-order distribution form of the fusion features of the same fault category, and the fourth-order moment distribution constraint loss is used to constrain the stability of the higher-order statistical distribution of the fusion features of the same fault category during the training process. S45: Construct a joint loss function based on the cross-entropy classification loss, triplet metric loss, and fourth-order moment distribution constraint loss: ; in, Denotes the joint loss function. Represents the cross-entropy classification loss. This indicates that the triplet measures the loss. This represents the fourth-order moment distributed constraint loss. , and These represent the weight coefficients of the corresponding loss terms, and the joint loss weights are set to... , , ; By minimizing the joint loss function, the learnable parameters in the fault diagnosis model are optimized. The learnable parameters include the network parameters of the three-way DenseNet-Transformer collaborative feature extraction network, the fusion feature mapping layer, and the classification mapping layer. In this embodiment, the Poisson gradient domain fusion method in step S3 is encapsulated as a cross-modal fusion module in the forward inference process of the fault diagnosis model. The cross-modal fusion module constructs a cross-modal joint feature map based on the acoustic modal deep collaborative feature map, the vibration modal deep collaborative feature map, and the temperature time series modal deep collaborative feature map. The cross-modal joint feature map is then input into the fusion feature mapping layer and the classification mapping layer to output the fault diagnosis result. The model training uses the Adam optimizer with an initial learning rate of 0.001, which decays to 0.9 times every 20 rounds for a total of 100 rounds. The model is evaluated using a validation set after each round. Three types of transformer operating conditions were simulated on a 10 kV / 0.4 kV transformer in the laboratory: normal operation, partial discharge abnormality, and overload / overheating abnormality. For each operating condition, 1500 sets of three-mode samples were collected. Each set of three-mode samples included synchronously aligned acoustic, vibration, and temperature time-series signals within the same time window, resulting in a total of 4500 sets of three-mode transformer fault diagnosis samples.

[0049] The 4500 sets of trimodal samples were randomly partitioned according to fault category into training, validation, and test sets in a 7:1:2 ratio. The training set consisted of 3150 samples (1050 samples for each operating state); the validation set consisted of 450 samples (150 samples for each operating state); and the test set consisted of 900 samples (300 samples for each operating state). The training set was used for parameter learning of the fault diagnosis model, the validation set was used for model parameter tuning and optimal model determination, and the test set was used to evaluate the recognition performance of the optimal fault diagnosis model after training.

[0050] During the testing phase, the trimodal samples to be diagnosed in the test set were input into the trained optimal fault diagnosis model to obtain the predicted operating state for each test sample. To evaluate the model's ability to identify different types of faults, the accuracy, precision, recall, and harmonic mean of precision and recall (F1 score) were calculated for different operating state categories. The test results are shown in Table 1.

[0051] Table 1 Performance Indicators for Fault Identification of Different Categories

[0052] As shown in Table 1, the fault diagnosis model achieves good performance under normal operating conditions, partial discharge abnormal conditions, and overload and overheating abnormal conditions. This indicates that after three-modal synchronous alignment, DenseNet-Transformer collaborative feature extraction, Poisson gradient domain fusion, and hybrid loss joint optimization, the model can effectively distinguish between normal conditions and different types of fault conditions, and improve the stability and category differentiation ability of transformer fault diagnosis under complex operating conditions.

[0053] Table 1 is used to evaluate the classification and recognition performance of the method of the present invention under different operating state categories. To further verify the effectiveness of the Poisson gradient domain fusion method and the three-modal collaborative input, comparative experiments are conducted from the perspectives of different fusion strategies and different input modalities. The accuracy, precision, recall, and F1 score in Tables 2 and 3 are the overall evaluation results on 900 sets of test samples.

[0054] To further verify the effectiveness of the Poisson gradient domain fusion method described in this invention in cross-modal feature fusion, this embodiment maintains the same dataset partitioning, DenseNet-Transformer collaborative feature extraction network, fusion feature mapping layer, classification layer, and joint loss function, but only replaces the cross-modal fusion strategy in S3 for comparative experiments. The comparative methods include feature concatenation fusion, weighted average fusion, attention fusion, and the Poisson gradient domain fusion described in this invention. Specifically, feature concatenation fusion directly concatenates the acoustic, vibration, and temperature temporal depth collaborative feature maps along the channel dimension and inputs them into the fusion feature mapping layer; weighted average fusion weights and sums the three modal feature maps according to preset modal weights to form a unified fusion feature map; attention fusion calculates the contributions of acoustic, vibration, and temperature temporal modes through channel attention and performs weighted fusion according to attention weights; and the Poisson gradient domain fusion described in this invention uses the vibration modal feature map as the background field and boundary constraints, injecting the acoustic anomaly gradient and temperature temporal gradient as residual guiding information into the background gradient field. The test results of different cross-modal fusion strategies are shown in Table 2.

[0055] Table 2. Experimental results comparing different cross-modal fusion strategies with Poisson gradient domain fusion.

[0056] As shown in Table 2, under the same test set and training conditions, the Poisson gradient domain fusion method adopted in this invention achieves an accuracy of 96.56% and an F1 score of 96.55%, which are 3.78, 2.56, and 1.45 percentage points higher than feature splicing fusion, weighted average fusion, and attention fusion, respectively. This result demonstrates that Poisson gradient domain fusion can effectively inject acoustic anomaly gradient and temperature temporal gradient information while maintaining the main structure of the vibration modal mechanical state, thereby enhancing the discriminative ability of cross-modal joint features.

[0057] Meanwhile, to verify the performance advantage of the three-modal collaborative input over the single-modal input, this embodiment further constructs acoustic single-modal, vibration single-modal, and temperature-temporal single-modal diagnostic models. The acoustic single-modal method uses only two-dimensional acoustic input representation for feature extraction and fault classification, mainly used to characterize acoustic anomalies such as partial discharge and abnormal noise; the vibration single-modal method uses only two-dimensional vibration input representation for feature extraction and fault classification, mainly used to characterize mechanical loosening, impact disturbance, and structural vibration anomalies; the temperature-temporal single-modal method uses only two-dimensional temperature-temporal input representation for feature extraction and fault classification, mainly used to characterize overload, overheating, temperature rise trends, and temporal fluctuations. The three-modal Poisson gradient domain fusion method of this invention simultaneously uses acoustic, vibration, and temperature-temporal three-modal inputs, and forms a cross-modal joint feature map through Poisson gradient domain fusion. The test results of the single-modal diagnostic method and the three-modal Poisson gradient domain fusion method of this invention are shown in Table 3.

[0058] Table 3. Comparative experimental results of single-modal diagnostic methods and Poisson gradient domain fusion.

[0059] Table 3 shows that the recognition performance of single acoustic mode, single vibration mode, and single temperature time-series mode is lower than that of the three-mode Poisson gradient domain fusion method. Among them, the recognition performance of the vibration single-mode method is relatively high, but it is still 5.78 percentage points lower than that of the method of this invention. The acoustic single-mode and temperature time-series single-mode methods are relatively unstable due to their susceptibility to environmental noise, sampling frequency differences, or incomplete fault characterization. This indicates that the present invention, through three-mode synchronous alignment, collaborative feature extraction, and Poisson gradient domain fusion, can fully utilize the complementary fault information between different modes to improve the accuracy and robustness of transformer fault diagnosis.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A transformer fault diagnosis method based on multimodal collaboration and triplet metric learning, characterized in that, Includes the following steps: S1: Acquire acoustic signals, vibration signals, and temperature timing signals during transformer operation, and perform synchronization alignment and conversion processing to obtain acoustic two-dimensional input characterization, vibration two-dimensional input characterization, and temperature timing two-dimensional input characterization; S2: The acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are respectively input into a three-way DenseNet-Transformer collaborative feature extraction network. Local feature extraction is performed through the DenseNet network, and global dependency modeling and gated adaptive aggregation processing are performed through the Transformer network to obtain acoustic modal deep collaborative feature maps, vibration modal deep collaborative feature maps, and temperature time-series modal deep collaborative feature maps. S3: The cross-modal fusion module employs the Poisson gradient domain fusion method to perform cross-modal fusion on the acoustic modal deep collaborative feature map, vibration modal deep collaborative feature map, and temperature time-series modal deep collaborative feature map. The vibration modal deep collaborative feature map is used as the background field. Gradient information representing acoustic anomaly modes in the acoustic modal deep collaborative feature map and gradient information representing temperature rise trends and time-series fluctuation characteristics in the temperature time-series modal deep collaborative feature map are injected into the background field as guiding information to construct a cross-modal joint guiding field. By solving the Poisson fusion equation, the gradient of the fused cross-modal joint feature map is made consistent with the cross-modal joint guiding field, thus obtaining the cross-modal joint feature map. S4: The cross-modal fusion module, the three-way DenseNet-Transformer collaborative feature extraction network, the fusion feature mapping layer, and the classification mapping layer together form the fault diagnosis model. In the fusion feature space output by the fusion feature mapping layer, online hard sample mining and triplet metric learning methods are used to construct feature triplets. Based on the feature triplets, the triplet metric loss is calculated, and a joint loss function is constructed by combining cross-entropy classification loss and fourth-order moment distribution constraint loss. The fault diagnosis model is trained and optimized to obtain the optimal fault diagnosis model, which is used for transformer fault diagnosis and outputs the diagnosis results.

2. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: Using a timestamped data acquisition method, acoustic signals, vibration signals, and temperature time-series signals during transformer operation are obtained to acquire the raw time-series data of the three modes; S12: Using a unified timestamp correction method, the acoustic signal, vibration signal and temperature time series signal are processed to unify the time coordinates, and the acquisition time of the vibration signal is used as a reference to obtain three-modal data under a unified time coordinate system. S13: The vibration reference sliding time window division method is adopted to divide the vibration signal under the unified time coordinate system into multiple vibration reference time windows, and each vibration reference time window is used as the synchronization alignment boundary of acoustic signal, vibration signal and temperature time sequence signal. S14: Using the time window matching method, the acoustic signal and temperature time series signal are matched to the corresponding vibration reference time window according to the start and end times of each vibration reference time window, so as to obtain the acoustic signal segment, vibration signal segment and temperature time series segment under the same time window; S15: Using time interpolation, resampling, and proximity-preserving completion methods, the acoustic signal segment, vibration signal segment, and temperature time series segment are sampled scale matched to obtain acoustic synchronization sequence, vibration synchronization sequence, and temperature synchronization sequence within the same time window. S16: Using the sample binding method, the acoustic synchronization sequence, vibration synchronization sequence and temperature synchronization sequence within the same vibration reference time window are bound according to the same sample number to obtain a three-mode time-aligned data segment with the vibration signal as the time reference. S17: A two-dimensional input representation conversion method is adopted to perform two-dimensional processing on the three-modal time-aligned data segments according to the data type of each modality. Specifically, the acoustic synchronization sequence is processed by short-time Fourier transform and Mel-filter mapping to obtain the acoustic two-dimensional input representation; the vibration synchronization sequence is processed by short-time Fourier transform to obtain the vibration two-dimensional input representation; and the temperature synchronization sequence is processed by sliding subsequence stacking to obtain the temperature time-series two-dimensional input representation. S18: Using a size normalization method, the acoustic two-dimensional input representation, vibration two-dimensional input representation, and temperature time-series two-dimensional input representation are subjected to scale resampling, boundary clipping, zero-value filling, and channel dimension normalization processing to unify the three two-dimensional input representations into a preset input size and preset channel form, thus obtaining a three-modal two-dimensional input representation group.

3. A transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 1 or 2, characterized in that, Step S2 specifically includes the following steps: S21: The DenseNet network is used to perform local feature reuse and convolutional feature extraction on the acoustic two-dimensional input representation, vibration two-dimensional input representation and temperature time-series two-dimensional input representation respectively, so as to extract the local texture anomaly features in the acoustic two-dimensional input representation, the local time-frequency impact features in the vibration two-dimensional input representation and the local temperature rise trend features in the temperature time-series two-dimensional input representation, and obtain the local feature map of the corresponding mode. S22: The local feature map extracted by the DenseNet network and the two-dimensional input representation guidance feature of the corresponding modality are jointly input into the Transformer network. Global correlation features in the two-dimensional representation space are obtained through position encoding and self-attention modeling, so that acoustic anomaly mode, vibration response mode and temperature time series change mode can form global dependency expression in their respective modes, and a global dependency feature map is obtained. S23: Perform gated adaptive aggregation processing on the local feature map, global dependency feature map, and two-dimensional input representation guidance features. Based on the local feature contribution and global dependency contribution of different modes under the current vibration reference time window, adaptively adjust the fusion weight of local features and global features to obtain the deep collaborative feature map of the corresponding mode.

4. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 1, characterized in that, Step S3 specifically includes: S31: The vibration mode depth co-feature map used to characterize the mechanical structure state, transient impact, and mechanical disturbance distribution of the transformer is used as the background field for Poisson gradient domain fusion; the feature map spatial region corresponding to the background field is defined, and the spatial position of the feature map spatial region located on the periphery of the feature map is defined as the boundary region; the features of the background field on the boundary region are used as the boundary constraints in the Poisson gradient domain fusion process to provide the main mechanical structure and boundary preservation conditions for the subsequent cross-modal joint feature map; S32: Perform spatial gradient mapping on the acoustic modal depth co-feature map, background field, and temperature time-series modal depth co-feature map respectively to obtain the corresponding gradient tensors; perform gradient magnitude normalization on the gradient tensors to obtain acoustic gradient features, vibration background gradient features, and temperature time-series gradient features. S33: Construct an acoustic residual gradient guiding term based on the difference between the acoustic gradient features and the vibration background gradient features; construct a temperature time-series residual gradient guiding term based on the difference between the temperature time-series gradient features and the vibration background gradient features. S34: Construct a cross-modal joint guiding field based on the vibration background gradient characteristics, acoustic residual gradient guiding term, and temperature time-series residual gradient guiding term; S35: Using the cross-modal joint guiding field as the target gradient field and the background field boundary features as the boundary preservation condition, establish the Poisson fusion equation; S36: Discretize the Poisson fusion equation and combine it with the background field boundary constraints to organize the discretized Poisson fusion equation into a system of linear equations. S37: The linear equation system is numerically solved using an iterative solution method to obtain the cross-modal joint feature vectors corresponding to each channel. The cross-modal joint feature vectors corresponding to each channel are then restored according to their spatial positions in the feature map to obtain the cross-modal joint feature map.

5. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 4, characterized in that, In step S34, the cross-modal joint guiding field is constructed according to the following formula: ； in, This represents the cross-modal joint steering field corresponding to the k-th vibration reference time window. These are identifiers representing acoustic, vibration, and temperature time sequences, respectively. Indicates the characteristics of the vibration background gradient. This represents the acoustic residual gradient guiding term. This represents the temperature-time residual gradient guiding term. This represents the injection weights of the acoustic residual gradient guiding term. The injection weights represent the temperature time-series residual gradient guiding terms; the acoustic residual gradient guiding terms are constructed based on the differences between the acoustic gradient features and the vibration background gradient features, and the temperature time-series residual gradient guiding terms are constructed based on the differences between the temperature time-series gradient features and the vibration background gradient features.

6. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 5, characterized in that, The injection weights of the acoustic residual gradient guidance term and the injection weights of the temperature time-series residual gradient guided term satisfy: ； Furthermore, the injection weight is dynamically adjusted based on prior fault knowledge within the current vibration reference time window: it increases when there is a tendency for partial discharge. Increase when there is a tendency to overheat. .

7. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 4, characterized in that, In step S32, the spatial gradient mapping adopts the central difference method, and the one-sided difference method is used in the boundary region of the feature map; the gradient magnitude normalization process adopts the L2 normalization method, and the gradient tensor of each channel is normalized channel by channel.

8. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 1, characterized in that, The fourth-order moment distribution constraint loss in step S4 is obtained by constructing the fourth-order central moment based on the fusion feature vector of the same fault category in the current training batch, and calculating the fourth-order moment distribution constraint loss based on the fourth-order central moment.

9. The transformer fault diagnosis method based on multimodal collaboration and triplet metric learning according to claim 1, characterized in that, A transformer fault dataset containing acoustic signals, vibration signals, temperature time-series signals, and corresponding fault type labels is constructed. The transformer fault dataset is then hierarchically and randomly divided according to the fault category to obtain a training set, a validation set, and a test set. The fault diagnosis model is trained using the training set based on the joint loss function, and the optimal fault diagnosis model is selected from multiple candidate fault diagnosis models obtained during the training process using the validation set. Then, the three-modal samples to be diagnosed in the test set are input into the optimal fault diagnosis model, and the transformer fault type is output. The training process takes the minimization of the joint loss function as the optimization objective.

10. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the transformer fault diagnosis method based on multimodal collaboration and triplet metric learning as described in any one of claims 1 to 9.

Images