Transformer state identification method and device based on multi-scale time-frequency characteristics

Abstract

The invention discloses a transformer state recognition method and device based on multi-scale time-frequency characteristics. The method comprises the steps that voiceprint signals generated when a transformer operates are collected; processing the voiceprint signal, and extracting a multi-scale time-frequency feature to obtain a time-frequency tensor; the time-frequency tensor is input into a pre-constructed multi-scale time-frequency model, and the multi-scale time-frequency model is constructed based on multi-scale convolution, a Transform neural network and a double-branch attention mechanism; and outputting a transformer state identification result through the multi-scale time-frequency model. According to the method, the voiceprint feature expression capability can be remarkably improved by extracting the multi-scale time-frequency features and the multi-scale time-frequency model, so that the transformer state recognition precision and speed are improved, and stable real-time recognition of the transformer state can still be realized in a complex noise environment.

Description

Technical Field

[0001] This invention relates to the field of transformer technology, and more specifically, to a method and apparatus for transformer state identification based on multi-scale time-frequency features. Background Technology

[0002] Transformers are extremely critical equipment in power systems, and their operating status directly affects the safety and stability of the power grid. During normal operation or the early stages of a fault, transformers generate specific acoustic signals. These signals are formed by factors such as core magnetostriction, winding vibration, tank resonance, and partial discharge, exhibiting distinct condition characteristics.

[0003] Traditional transformer condition monitoring mainly relies on electrical quantities (such as voltage and current) or infrared thermography, which makes it difficult to detect early mechanical and electromagnetic coupling anomalies in a timely manner. However, acoustic fingerprint signals are extremely sensitive to structural changes and discharge anomalies. Therefore, intelligent monitoring based on acoustic fingerprints is considered an important direction for realizing "auscultatory" fault detection of transformers.

[0004] Currently, most mainstream voiceprint recognition algorithms employ CNN (Convolutional Neural Network) or LSTM (Long Short-Term Memory) structures to extract features, while some studies introduce SE (Squeeze-and-Excitation) or CBAM (Convolutional Block Attention Module) attention mechanisms. However, these models have the following shortcomings: (1) The model has poor adaptability to changes in time-frequency characteristics and cannot capture transient sound patterns under complex background noise; (2) In edge deployment scenarios (such as substation field terminals), the computational overhead is large and the inference latency is high.

[0005] There is currently no effective solution to the problems of low accuracy and high latency in existing transformer voiceprint recognition models. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a transformer condition identification method and apparatus based on multi-scale time-frequency features. A multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural networks, and a dual-branch attention mechanism. Multi-scale time-frequency features are extracted from the collected transformer acoustic signature signals and combined with this model for transformer condition identification. This significantly improves the expressive power of acoustic signature features, enhancing the accuracy and speed of transformer condition identification. Stable real-time identification of transformer conditions can still be achieved even in complex noise environments. This enables real-time identification of transformer operating conditions and early fault warning, solving the problems of low accuracy, high latency, and poor transferability of traditional acoustic signature recognition models in complex noise environments.

[0007] In a first aspect, the present invention provides a transformer state identification method based on multi-scale time-frequency features, comprising: Acquire acoustic signature signals during transformer operation; By processing the voiceprint signal, multi-scale time-frequency features are extracted to obtain the time-frequency tensor; The time-frequency tensor is input into a pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism; The transformer state identification result is output through the multi-scale time-frequency model.

[0008] Furthermore, by processing the voiceprint signal, multi-scale time-frequency features are extracted to obtain a time-frequency tensor, including: The voiceprint signal is preprocessed, including pre-emphasis, framing, and windowing; The preprocessed voiceprint signal is subjected to a short-time Fourier transform to obtain the first time-frequency diagram; Continuous wavelet transform is performed on the preprocessed voiceprint signal to extract feature changes at different time scales, resulting in a second time-frequency map. The frequency characteristics of the preprocessed acoustic signature signal are converted into Mel frequencies using the Mel filter bank to obtain the Mel spectrum. The first time-frequency graph, the second time-frequency graph, and the Mel spectrum graph are concatenated into a three-channel input tensor to obtain the time-frequency tensor.

[0009] Furthermore, the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural networks, and a two-branch attention mechanism, including: Training data is collected in the cloud. The training data includes acoustic fingerprint data under different transformer conditions, including normal, loose core, winding fault, partial discharge, and mechanical looseness. The initial network for transformer state recognition is constructed in the cloud, including a multi-scale convolutional front-end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer. The time-frequency cross-attention module includes a time attention branch, a frequency attention branch, and a cross-fusion layer. The cloud platform iteratively trains the initial network until convergence based on the training data to obtain the multi-scale time-frequency model. The multi-scale convolutional front end is used to extract multi-scale features from the training data using convolutional kernels of different sizes; the time-frequency cross-attention module is used to reorganize the multi-scale features and perform self-attention calculation through time attention branches and frequency attention branches, and to perform feature fusion through a cross-fusion layer; the lightweight Transformer encoder is used to encode through a factorization multi-head attention mechanism; the edge SE fusion module is used to recalibrate the channel weights; and the output layer is used to perform global average pooling on the time-frequency features output by the edge SE fusion module and output the transformer state category probability.

[0010] Furthermore, the cloud platform iteratively trains the initial network until convergence based on the training data, including: Step 1: Extract the time-frequency tensor from the training data, input it into the multi-scale convolution front end, use convolution kernels of different sizes to extract multi-scale features from the training data in parallel, output the time-frequency features under different frequency bands, perform channel concatenation, and activate them with BatchNorm and ReLU to form a fused feature tensor; Step 2: Input the fused feature tensor into the time-frequency cross-attention module, process it through the time attention branch and the frequency attention branch, and perform feature fusion through the cross-fusion layer; Step 3: Input the output of the time-frequency cross-attention module into the lightweight Transformer encoder and process it through the factorization multi-head attention mechanism; Step 4: Input the output of the lightweight Transformer encoder into the edge SE fusion module, and recalibrate the channel weights using the squeeze excitation mechanism; Step 5: Input the output of the edge SE fusion module into the output layer, perform global average pooling, and then map it to the transformer state category through a fully connected layer and Softmax function to output the transformer state category probability; Step Six: Repeat steps one through five until convergence, completing the training.

[0011] Furthermore, after iteratively training the initial network in the cloud using the training data until convergence to obtain the multi-scale time-frequency model, the process further includes: The cloud compresses the multi-scale time-frequency model to adapt it to edge nodes; The cloud distributes the compressed multi-scale time-frequency model to the edge nodes of each substation.

[0012] Furthermore, the above-mentioned transformer status identification method also includes: Local storage of voiceprint data that has not been labeled within the most recent preset time period; The unlabeled voiceprint data is grouped using a sliding window of a preset size; The transformer state category is identified for each group of grouped data using a locally stored multi-scale time-frequency model, resulting in pseudo-label data for each group. The multi-scale time-frequency model stored locally is trained and its parameters are updated based on pseudo-labeled data and real labeled data.

[0013] Furthermore, after outputting the transformer state identification result through the multi-scale time-frequency model, the process also includes: The transformer status identification results and voiceprint summaries are uploaded to the cloud. The cloud uses a clustering algorithm to filter data based on the voiceprint summaries and transformer status identification results of the edge nodes of multiple substations to participate in model retraining. When the performance indicators of the new model are better than the current version of the model in the cloud, a new model file is generated and distributed to the edge nodes of each substation.

[0014] Furthermore, the above-mentioned transformer status identification method also includes: Receive the new model file from the cloud locally; The new model file is decrypted and its signature verified to obtain the new model. Determine whether the prediction accuracy of the new model is better than that of the current local model. If so, the verification passes, and the current local version of the model is hot-replaced to complete the model update.

[0015] Furthermore, after uploading the transformer status identification results and voiceprint summary to the cloud, the cloud also provides a voiceprint feature heatmap, status classification probability, and anomaly trend curve.

[0016] Secondly, the present invention provides an apparatus for implementing the above-mentioned transformer state identification method based on multi-scale time-frequency features, comprising: The acquisition module is used to acquire acoustic fingerprint signals during transformer operation; The feature extraction module is used to process the voiceprint signal to extract multi-scale time-frequency features and obtain a time-frequency tensor. The input module is used to input the time-frequency tensor into a pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism; The output module is used to output the transformer state identification result through the multi-scale time-frequency model.

[0017] The transformer state identification method and apparatus based on multi-scale time-frequency features provided by this invention have at least the following beneficial effects: (1) A multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism. Multi-scale time-frequency features are extracted from the collected transformer acoustic signature signal to obtain time-frequency tensor and input into the model for transformer status identification. By extracting multi-scale time-frequency features and multi-scale time-frequency model, the acoustic signature feature expression capability can be significantly improved, thereby improving the accuracy and speed of transformer status identification. Stable real-time identification of transformer status can still be achieved in complex noise environment.

[0018] (2) Multi-scale time-frequency features are extracted by short-time Fourier transform, continuous wavelet transform and Mel filter bank respectively to form a three-channel time-frequency tensor, which can significantly improve the voiceprint feature expression capability and help improve the accuracy and speed of transformer status identification.

[0019] (3) The constructed multi-scale time-frequency model includes a multi-scale convolutional front-end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer. Combining the advantages of multi-scale convolution, Transformer, and attention mechanisms, it can capture local textures and learn global temporal dependencies, thereby improving the accuracy and speed of transformer state recognition. The multi-scale time-frequency model is constructed in the cloud and distributed to edge nodes for local real-time recognition, realizing near-source processing and rapid response of voiceprint signals, avoiding network congestion and privacy risks caused by uploading a large amount of raw audio to the cloud.

[0020] (4) Edge nodes perform lightweight online fine-tuning of local model parameters, complete on-site adaptive model updates, realize edge-side adaptive learning and scene adaptation, and improve model performance. Moreover, on-site adaptive updates do not require retraining, resulting in low maintenance costs.

[0021] (5) The model is automatically updated and optimized in the cloud and promptly distributed to the edge nodes to ensure the accuracy and speed of transformer status identification. Attached Figure Description

[0022] Figure 1 This is a flowchart of a transformer state identification method based on multi-scale time-frequency features provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the transformer state identification architecture provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of feature extraction provided in an embodiment of the present invention; Figure 4 This is the initial network architecture diagram of the multi-scale time-frequency model provided in this embodiment of the invention; Figure 5 This is a schematic diagram of the time-frequency cross-attention module structure provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the architecture of the cloud-edge collaborative transformer acoustic signature intelligent monitoring system provided in an embodiment of the present invention; Figure 7 This is a flowchart of cloud-edge collaborative operation provided in an embodiment of the present invention; Figure 8 This is a structural block diagram of a transformer state identification device based on multi-scale time-frequency features provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0024] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0025] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0026] To address the issues of low accuracy and high latency in existing transformer voiceprint recognition models, this invention provides a transformer state recognition method based on multi-scale time-frequency features. This method can be executed by edge nodes deployed in substations, such as... Figure 1 As shown, the method includes the following steps: S101 collects acoustic signature signals during transformer operation.

[0027] S102, by processing the voiceprint signal, extracts multi-scale time-frequency features to obtain the time-frequency tensor.

[0028] S103, input the time-frequency tensor into the pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism.

[0029] S104 outputs transformer condition identification results through a multi-scale time-frequency model. Transformer conditions include normal, loose core, winding fault, partial discharge, and mechanical loosening.

[0030] This invention constructs a multi-scale time-frequency model based on multi-scale convolution, Transformer neural network, and dual-branch attention mechanism. Multi-scale time-frequency features are extracted from the collected transformer acoustic signature signal to obtain a time-frequency tensor, which is then input into the model. The model outputs the transformer state recognition result. By extracting multi-scale time-frequency features and using the multi-scale time-frequency model, the ability to express acoustic signature features can be significantly improved, thereby improving the accuracy and speed of transformer state recognition. Stable real-time recognition of transformer state can still be achieved in complex noise environments, solving the problems of low accuracy and high latency of existing transformer acoustic signature recognition models in complex noise environments.

[0031] like Figure 2 As shown, the architecture for transformer status identification can include: an acoustic signature acquisition layer, a feature extraction layer, an intelligent identification layer, and a result visualization layer.

[0032] (1) The acoustic signature acquisition layer is used to acquire acoustic signature signals during transformer operation. Specifically, it can be acquired in real time by an industrial-grade acoustic sensor installed on the surface of the transformer casing. For example, a high-sensitivity microphone array can be used to acquire acoustic signature signals during transformer operation, with a sampling frequency of 48kHz. In order to suppress environmental noise interference, the acoustic signature acquisition layer can integrate adaptive filtering and multi-channel beamforming algorithms to extract the signal components of the optimal sound source direction.

[0033] (2) The feature extraction layer is used to process the voiceprint signal, extract multi-scale time-frequency features, and obtain the time-frequency tensor.

[0034] refer to Figure 3 By processing the voiceprint signal, multi-scale time-frequency features are extracted to obtain the time-frequency tensor, including: S11: Preprocess the voiceprint signal, including pre-emphasis, framing, and windowing; pre-emphasis can enhance the high-frequency components of the signal; framing can effectively reduce the non-stationarity of the signal, for example, framing the voiceprint signal with a sampling frequency of 48kHz with a frame length of 25ms and a frame shift of 10ms; windowing can reduce spectral leakage, for example, by using a Hamming window. S12: Perform a Short-time Fourier Transform (STFT) on the preprocessed voiceprint signal to obtain the first time-frequency diagram; specifically, first perform a Fast Fourier Transform (FFT) on each frame of the preprocessed voiceprint signal to obtain a linear spectrum diagram, then arrange the spectra of all frames in chronological order to form a two-dimensional time-frequency diagram, and then compress the dynamic range through a logarithmic transformation to improve the distinguishability of weak signals; S13: Perform a continuous wavelet transform (CWT) on the preprocessed voiceprint signal to extract feature changes at different time scales and obtain a second time-frequency map; specifically, the scaling and translation of the Morlet wavelet basis function can be used to extract feature changes at different time scales and obtain a second time-frequency map, thereby enhancing the sensitivity to non-stationary signals and realizing time-frequency localization analysis of the signal. S14: The frequency characteristics of the preprocessed acoustic signature signal are converted into Mel frequencies through the Mel filter bank to obtain the Mel spectrum; that is, the frequency characteristics of the signal are converted to the human ear auditory scale through the Mel filter bank to better reflect the mechanical and electromagnetic coupling effects inside the transformer. S15: Concatenate the first time-frequency plot, the second time-frequency plot, and the Mel spectrum plot into a three-channel input tensor to obtain a time-frequency tensor. For example, through multi-channel fusion, a time-frequency tensor with a shape of [128×128×3] is obtained, corresponding to STFT, CWT, and Mel spectrum information, respectively.

[0035] After preprocessing the voiceprint signal through the above steps, multi-scale time-frequency features are extracted by short-time Fourier transform, continuous wavelet transform and Mel filter bank to form a three-channel time-frequency tensor, which can significantly improve the voiceprint feature expression capability and help improve the accuracy and speed of transformer state identification.

[0036] (3) The intelligent recognition layer is used to identify the transformer state through the local stored multi-scale time-frequency model. The time-frequency tensor is input into the multi-scale time-frequency model, and the transformer state recognition result is output through the multi-scale time-frequency model.

[0037] The multi-scale time-frequency model is the MS-TF-Transformer (Multi-Scale Time-Frequency Transformer), capable of deep modeling and classification of voiceprint features. Combining the advantages of multi-scale convolution, Transformer, and attention mechanisms, the multi-scale time-frequency model can capture local textures and learn global temporal dependencies, thereby improving recognition accuracy and speed. Specifically, the intelligent recognition layer can employ the TensorRT inference engine with a built-in MS-TF-Transformer model.

[0038] (4) The results visualization layer is used to display visualization results related to the transformer status, such as: acoustic signature feature heatmap, status classification probability, anomaly trend curve, etc. The above visualization results can also be uploaded to the cloud for information synchronization to realize a cloud-edge collaborative acoustic signature intelligent monitoring network. If the transformer status identification result is abnormal, alarm information can be output, such as audible and visual alarms or display alarm information.

[0039] The construction of the multi-scale time-frequency model will be explained below.

[0040] This invention constructs a multi-scale time-frequency model in the cloud and distributes it to edge nodes. The edge nodes then use the received and stored multi-scale time-frequency model locally to identify the transformer status based on the real-time collected voiceprint signals. This enables near-source processing and rapid response of voiceprint signals, avoiding network congestion and privacy risks caused by uploading large amounts of raw audio to the cloud.

[0041] The multi-scale time-frequency model is built upon multi-scale convolution, Transformer neural networks, and a two-branch attention mechanism, including: Training data is collected in the cloud. The training data includes acoustic fingerprint data under different transformer conditions, including normal, loose core, winding fault, partial discharge and mechanical looseness. For example, the acoustic fingerprint dataset of transformers in State Grid substations and experimental platforms can be collected as training data. The sampling time for each type of transformer condition is no less than 10 hours, and the total data volume exceeds 500GB. The initial network for transformer state recognition is constructed in the cloud, including a multi-scale convolutional front-end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer. The time-frequency cross-attention module includes a time attention branch, a frequency attention branch, and a cross-fusion layer. The cloud platform iteratively trains the initial network until convergence based on the training data, resulting in a multi-scale time-frequency model.

[0042] The multi-scale convolutional front end is used to extract multi-scale features from the training data using convolutional kernels of different sizes; the time-frequency cross-attention module is used to reorganize multi-scale features and perform self-attention calculation through time attention branches and frequency attention branches, and to perform feature fusion through the cross-fusion layer; the lightweight Transformer encoder is used to encode through a factorization multi-head attention mechanism; the edge SE fusion module is used to recalibrate the channel weights; and the output layer is used to perform global average pooling on the time-frequency features output by the edge SE fusion module and output the transformer state class probability.

[0043] The multi-scale time-frequency model constructed in this invention includes a multi-scale convolutional front-end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer. Combining the advantages of multi-scale convolution, Transformer, and attention mechanisms, it can capture local textures and learn global temporal dependencies, thereby improving the accuracy and speed of transformer state recognition.

[0044] The cloud-based system iteratively trains the initial network until convergence based on the training data, including: S21: Extract the time-frequency tensor from the training data, input it into the multi-scale convolution front end, use convolution kernels of different sizes to extract multi-scale features from the training data in parallel, output the time-frequency features under different frequency bands, perform channel concatenation, and activate them with BatchNorm and ReLU to form a fused feature tensor; S22: Input the fused feature tensor into the time-frequency cross-attention module, perform recombination and self-attention calculation through time attention branch and frequency attention branch, and perform feature fusion through cross-fusion layer; S23: Input the output of the time-frequency cross-attention module into the lightweight Transformer encoder and process it through the factorization multi-head attention mechanism; S24: Input the output of the lightweight Transformer encoder into the edge SE fusion module, and recalibrate the channel weights using the squeeze excitation mechanism; S25: Input the output of the edge SE fusion module into the output layer, perform global average pooling, and then map it to the transformer state category through the fully connected layer and the Softmax function and output the transformer state category probability; S26: Repeat steps S21 to S25 above until the performance metrics meet the requirements, reaching convergence and completing the training. The performance metrics include at least one of the following: cross-entropy loss, F1 score, and prediction accuracy. Prediction accuracy refers to the model's accuracy in voiceprint recognition.

[0045] Understandably, the cloud platform first preprocesses the collected training data, extracting multi-scale time-frequency features through short-time Fourier transform, continuous wavelet transform, and Mel filter bank to form a three-channel time-frequency tensor, which is then input into the initial network. The cloud platform then performs the aforementioned iterative training process on the initial network based on the training data, enabling the construction of a multi-scale time-frequency model. This model combines the advantages of multi-scale convolution, Transformer, and attention mechanisms, capturing both local textures and learning global temporal dependencies, thus improving the accuracy of transformer state recognition. Furthermore, this model reduces the number of parameters and computational cost, increasing the speed of transformer state recognition.

[0046] like Figure 4 As shown, the initial network of the MS-TF-Transformer model, as a lightweight Transformer structure, includes: a multi-scale convolutional front end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer.

[0047] The multi-scale convolutional front-end (MSC Block) consists of three convolutional kernels with different receptive fields (3×3, 5×5, and 7×7) operating in parallel. These kernels capture time-frequency features across different frequency bands. After channel concatenation, they are activated using BatchNorm and ReLU to form a fused feature tensor. In a specific example, the MSC Block uses a 3×3 convolution to capture fine-grained textures at 100Hz, a 5×5 convolution to capture medium-grained temporal trends at 100Hz, and a 7×7 convolution to capture coarse-grained spatiotemporal distributions at 100Hz. The output features of the three convolutional kernels are concatenated and then activated to form a fused feature tensor, which serves as the input to the time-frequency cross-attention module, providing multi-scale local texture features of the speaker signal.

[0048] A dual-branch attention mechanism is introduced before the Transformer encoder, consisting of a temporal attention (T-Attention) branch and a frequency attention (F-Attention) branch. Together with the cross-fusion layer, this forms a temporal-frequency cross-attention module (TF-Attention), such as... Figure 5 As shown, this module processes signals through temporal and frequency attention, and calculates the interaction correlation between the temporal and frequency dimensions using a cross-attention matrix, enabling the model to adaptively focus on energy transitions and spectral variation regions of the voiceprint. Specifically, the temporal attention branch expands the fused feature tensor into a sequence X over time frames. T This model aims to model the dynamic evolution of the same frequency point across different time frames; the frequency attention branch expands the fused feature tensor into a sequence X based on frequency points. F This is used to model the correlation between different frequency points within the same time frame. Then, the temporal attention branch is applied to sequence X. T Calculate temporal attention and output temporal correlation feature A.T Frequency attention branch pairs with sequence X F Calculate frequency attention and output frequency domain correlation feature A. F Next, the cross-fusion layer uses temporal correlation features as the query source and frequency domain correlation features as the key source to generate a cross-attention matrix, where each element m... ij This represents the interaction strength between frequency point i and time frame j. Finally, the frequency domain correlation features are weighted and aggregated using a cross-attention matrix, and then reconstructed into a three-dimensional tensor for output.

[0049] In this example, time attention captures the temporal continuity of transient events (e.g., the complete cycle of partial discharge sound from generation to decay), frequency attention locates fault characteristic frequency bands (e.g., specific frequency band harmonic energy accumulation caused by core loosening), and cross-attention models time-frequency nonlinear coupling relationships (e.g., harmonic energy changes with time load due to winding deformation). Compared to traditional time-frequency joint analysis, this avoids cross-interference and significantly improves the ability to identify complex faults (e.g., mechanical loosening accompanied by partial discharge).

[0050] The lightweight Transformer Encoder consists of two encoder layers and employs an improved factorized multi-head attention mechanism. It performs factor analysis on the input data, extracts common factors, determines the number of heads based on the factor analysis results, and splits the input data into multiple heads. Each head independently learns different feature representations of the input data, and finally, the outputs of multiple heads are aggregated. Through a two-layer stacked structure (128-dimensional hidden layer and 256-dimensional feedforward layer), it learns deeper global temporal dependencies and time-frequency coupling relationships. Factorized attention reduces computational cost, two-layer stacking enhances feature representation, and residual connections and layer normalization improve training stability. This achieves a balance between global temporal modeling of acoustic signature signals and edge device adaptation, improving the real-time performance and accuracy of intelligent transformer acoustic signature monitoring.

[0051] The first encoder layer receives the feature sequence output by the cross-attention module. Each sequence element corresponds to a time-frequency feature of the voiceprint signal, representing the high-dimensional semantic information of that time-frequency point (such as the preliminary features of "normal" or "fault"). The feature sequence is first linearly transformed, generating query Q, key K, and value V through three linear layers (in this example, there are 8 heads, each with a dimension of 16). Then, key K is decomposed into a time-low-rank matrix and a frequency-low-rank matrix, and after reconstructing key K using the Kronecker Product, attention weights are calculated and weighted to fuse the value V. The outputs of each head are then merged to obtain the attention output. After residual connection and layer normalization, the input is fed to the feedforward layer for nonlinear transformation, residual connection, and layer normalization to obtain the output of the first encoder layer.

[0052] The output of the first encoder layer is input to the second encoder layer, and the same processing procedure is performed as for the first encoder layer, ultimately outputting a time-frequency feature sequence encoded by both layers. Each time-frequency feature integrates global temporal dependence and time-frequency coupling, providing more discriminative features for the edge SE fusion module to recalibrate channel weights.

[0053] The Edge-SE Block employs a squeeze-and-excitation mechanism on the convolutional output channel dimension to recalibrate the channel-level weights of the time-frequency features encoded by the Transformer. This emphasizes fault-related channels (such as the 100Hz harmonic channel for a loose iron core) and suppresses environmental noise channels (such as high-frequency channels for wind noise). Weight coefficients can be pre-stored in the edge deployment version using a lookup table to reduce real-time computational burden. Squeezing involves performing average pooling on each channel of the input feature tensor, calculating the average of all elements within that channel to obtain the global context scalar for that channel. This reflects the channel's contribution to the current voiceprint state, providing a basis for subsequent channel weight recalibration. Excitation involves multiplying each channel of the input feature tensor by its corresponding weight coefficient (obtainable through a lookup table) to obtain the recalibrated feature value.

[0054] The output layer compresses the time-frequency features (two-dimensional features in this example) output by the edge SE fusion module into a one-dimensional vector using global average pooling. Then, it outputs the transformer state category probability through a fully connected layer and a softmax function. Specifically, the output layer averages all time-frequency points for each channel of the time-frequency features and integrates these averages into a one-dimensional feature vector. The fully connected layer maps the one-dimensional feature vector to the transformer state category, representing it as an unnormalized predicted value. The softmax function then converts this unnormalized predicted value into a probability distribution, i.e., the transformer state category probability.

[0055] The entire model has approximately 0.68M parameters and 43MFLOPs of computation. The inference latency on the device is less than 25ms, which effectively reduces the number of parameters and computation, and improves the speed of transformer state identification.

[0056] After iteratively training the initial network to convergence using training data in the cloud to obtain the multi-scale time-frequency model, the process also includes: compressing the multi-scale time-frequency model in the cloud to adapt it to edge nodes; and distributing the compressed multi-scale time-frequency model to the edge nodes of each substation in the cloud. Model compression reduces the model size, thereby reducing storage requirements and computational complexity while maintaining accuracy.

[0057] In practice, data augmentation strategies can be used to enhance the robustness of the model. For example, SpecAugment can be used to randomly mix environmental noise (such as wind noise, bird calls, equipment humming, etc.) with random temporal and frequency occlusion.

[0058] Specifically, compression techniques such as pruning, fixed-point quantization, and structural distillation can be used to compress the trained multi-scale time-frequency model to a size that fits the edge nodes. For example, by removing 30% of low-weight connections based on a global pruning strategy and using INT8 dynamic quantization, the model size can be compressed from 15MB to 3.2MB.

[0059] This invention deploys a multi-scale time-frequency model on edge nodes. The deployment format supports TensorRT and ONNXRuntime. Both C++ and Python API versions are automatically generated. The model runs independently on the edge nodes, which possess the ability to identify transformer states in real time and adaptively update the local model. Regarding local model adaptive updates, the above transformer state identification method further includes the following steps: Local storage of voiceprint data that has not been labeled within the most recent preset time period; The unlabeled voiceprint data is grouped using a sliding window of a preset size; The transformer state category is identified for each group of grouped data using a locally stored multi-scale time-frequency model, resulting in pseudo-label data for each group. The multi-scale time-frequency model stored locally is trained and its parameters are updated based on pseudo-labeled data and real labeled data.

[0060] The recent preset time can be set according to the actual situation, such as caching voiceprint data that has not been labeled in the last 24 hours at the edge node.

[0061] In this embodiment, edge nodes cache unlabeled voiceprint samples from the recent past. A semi-supervised pseudo-labeling method is used to perform lightweight online fine-tuning of the local model parameters, updating only the weights of the last two layers—the edge SE fusion module and the output layer. This achieves adaptive updating of the model in the current environment, enabling adaptive learning at the edge and adaptation to different scenarios, thus improving model performance. Furthermore, this adaptive updating eliminates the need for retraining, resulting in low maintenance costs and avoiding the poor transferability of traditional voiceprint recognition models in complex noisy environments.

[0062] Specifically, a sliding window strategy can be used to update the weights of the last two layers, improving the model's adaptability to data. Experiments showed a local inference latency of 22ms, an average CPU utilization of 38%, a power consumption of 3.2W, and continuous operation for 30 days without performance degradation.

[0063] This invention proposes a cloud-edge collaborative intelligent monitoring system for transformer voiceprints. By deploying lightweight edge nodes in each substation and forming a distributed collaborative architecture with the cloud central server, it realizes an integrated voiceprint diagnostic system that combines "local real-time recognition + cloud-based centralized learning + dynamic model synchronization".

[0064] like Figure 6 As shown, the intelligent transformer acoustic signature monitoring system based on cloud-edge collaboration includes the following three layers: edge layer, communication layer, and cloud layer.

[0065] A. Edge layer The edge layer refers to the embedded edge computing nodes deployed at various substation sites. The edge layer includes the following modules: The voiceprint acquisition module is used to acquire the voiceprint signal of the transformer during operation, and can use a high-sensitivity microphone array; The feature extraction module is used to generate and fuse STFT, CWT and Mel spectra to obtain time-frequency tensors; A lightweight identification model for identifying transformer states can be used with the TensorRT inference engine, which has a built-in MS-TF-Transformer network. The local alarm module is used to output alarm information when the transformer is identified as being in an abnormal state.

[0066] The edge layer may also include a data caching and pseudo-label module, which is used to cache the latest 24-hour voiceprint data for subsequent self-learning and to achieve local model self-updating.

[0067] B. Communication Layer The communication layer is responsible for data interaction and model synchronization between edge nodes and the cloud platform, employing a highly reliable, low-latency, lightweight communication protocol. The transmission protocol can use MQTT (Message Queuing Telemetry Transport) and SSL (Secure Sockets Layer) encrypted channels. A publish-subscribe mechanism is used for asynchronous message interaction. It features breakpoint resumption and automatic reconnection mechanisms to ensure network reliability. The data transmitted by the communication layer includes: transformer status identification results (also known as diagnostic results), voiceprint summaries (e.g., MFCC feature packets), model version numbers, and alarm events. The voiceprint summary is a low-dimensional, high-information feature representation of the voiceprint signal. By extracting key semantic information (such as frequency distribution, energy changes, and temporal features) from the voiceprint signal, the original audio data is compressed into a compact feature packet.

[0068] It adopts a modular C++ and Python hybrid development architecture, with the inference end encapsulated as a callable dynamic library (.so); it communicates bidirectionally with the cloud server via the MQTT protocol to achieve state synchronization, model updates and log uploads; it can seamlessly interface with SCADA (Supervisory Control And Data Acquisition) or substation monitoring systems, and supports Modbus-TCP data interface.

[0069] To ensure reliability and data security, this invention designs a multi-level protection mechanism at the communication layer: (1) Encrypted communication: All message transmissions use the MQTTS protocol and employ TLS 1.3 encryption and a two-way authentication mechanism; (2) Hierarchical access control: The cloud platform assigns independent keys and access tokens to different edge nodes; (3) Data anonymization: The voiceprint summary data uploaded by the edge node only contains spectral features and does not contain reversible audio; (4) Model signature verification: Each distributed model package is accompanied by an SHA256 signature file and a timestamp to prevent tampering.

[0070] C. Cloud layer The cloud layer is a voiceprint intelligent analysis platform deployed in data centers or power grid operation and maintenance master stations, possessing large-scale data management and unified model scheduling capabilities. The cloud layer mainly includes the following modules: The voiceprint database management module is used to store historical voiceprint samples and diagnostic logs; The training module is used for global model training and fusion based on a distributed GPU cluster. The model update module is used to automatically evaluate the accuracy and generalization ability of the model version, and distribute the model to each edge node after it passes the evaluation. The visualization monitoring module is used to display global voiceprint maps, device health indices, and alarm statistics via a web interface.

[0071] The cloud can perform global optimization and model evolution, and periodically distribute optimized model parameters to edge nodes, thereby enabling intelligent migration and adaptive updates between different scenarios.

[0072] After outputting the transformer status identification results through the multi-scale time-frequency model, the above method also includes: uploading the transformer status identification results and voiceprint summaries to the cloud; the cloud uses a clustering algorithm to select data to participate in model retraining based on the voiceprint summaries and transformer status identification results of the edge nodes of multiple substations; when the performance index of the new model is better than the current version of the model in the cloud, a new model file is generated and sent to the edge nodes of each substation.

[0073] The voiceprint summary only contains spectral features and does not include reversible audio, thus achieving data anonymization and ensuring security. For example, transformer status identification results and key voiceprint summaries can be reported to the cloud in JSON format via MQTT.

[0074] The cloud-based system periodically receives voiceprint summaries and diagnostic results from multiple substations. Clustering algorithms (such as K-Means and DBSCAN) are used to assess sample quality and screen for abnormal voiceprints within this data. The filtered data is then used for global model retraining. The cloud-based training module comprehensively evaluates the performance of the new model based on metrics such as cross-entropy loss, F1 score, and prediction accuracy. When the new model outperforms the current model, a new model file (e.g., a signature package) is automatically generated and pushed to each edge node, completing model evaluation and distribution. This ensures automatic cloud-based model updates and optimization, guaranteeing both accuracy and speed in transformer status identification.

[0075] In practical implementation, the following model optimization strategies can be used: the optimizer uses Adam with an initial learning rate of 1e-3, combined with cosine annealing scheduling to improve model training efficiency; Label Smoothing cross-entropy loss and contrastive learning regularization are used for joint optimization. Experiments show that the recognition accuracy on the training set is 98.6%, and on the validation set it is 97.8%, demonstrating significant noise resistance and few-shot transfer capability.

[0076] Specifically, the edge node receives a new model file from the cloud; it decrypts and verifies the signature of the new model file to obtain the new model; it then determines whether the prediction accuracy of the new model is better than that of the current local model; if so, the verification passes, and the current local model is hot-replaced, the new model is loaded, and the model update is completed. In this embodiment, the edge node performs local verification on the received new model file, which improves security and reliability, and allows for hot replacement updates without stopping the monitoring process.

[0077] After uploading the transformer status identification results and voiceprint summaries to the cloud, the process also includes: displaying visualization results related to the transformer status on the cloud, such as providing voiceprint feature heatmaps, status classification probabilities, and abnormal trend curves to achieve visualization. The cloud-based web platform can display the operating status, voiceprint spectrograms, and health indices of all transformers in real time, and link with the power monitoring system through the SCADA interface to trigger operation and maintenance early warnings, achieving visualization and alarm linkage.

[0078] like Figure 7 As shown, the cloud-edge collaborative operation process includes: S601, edge nodes acquire transformer acoustic signature signals in real time.

[0079] S602, edge nodes perform feature extraction on voiceprint signals.

[0080] S603, edge nodes use locally stored multi-scale time-frequency models to identify transformer status.

[0081] In S604, edge nodes upload transformer status identification results and voiceprint summaries to the cloud. The cloud updates the model based on data from multiple edge nodes and distributes the updates to each edge node. Edge nodes can cache data from a recent period locally for local model adaptive updates.

[0082] After a 60-day trial run at multiple substations, the overall system performance is shown in Table 1 below: Table 1 Performance Indicators of Multi-Scale Time-Frequency Models

[0083] During the trial operation, the low-frequency noise changes caused by the loosening of the iron core were successfully identified in advance, and an early warning was issued 6 days in advance; the partial discharge anomaly identification rate reached 99.1%, and the consistency of manual verification was >98%; after updating through the unified model in the cloud, the self-learning accuracy of the edge nodes was improved by 2.3%.

[0084] This invention provides a transformer status identification method based on multi-scale time-frequency features. By monitoring transformer acoustic signatures, it achieves real-time identification of transformer operating status and early fault warning. The multi-scale time-frequency fusion Transformer structure significantly improves the acoustic signature feature representation capability, supports edge adaptive learning, enables on-site model self-updating, and constructs an end-to-end lightweight intelligent monitoring system. This cloud-edge collaborative solution achieves high-precision distributed monitoring and intelligent diagnosis of transformer acoustic signatures, maintaining stability, real-time performance, and scalability even in complex environments. It solves the problems of low accuracy, high latency, and poor transferability of traditional acoustic signature recognition models in complex noise environments.

[0085] The present invention also provides an apparatus for implementing any of the transformer state identification methods based on multi-scale time-frequency features as described above, such as... Figure 8 As shown, the device includes: Acquisition module 701 is used to acquire acoustic fingerprint signals during transformer operation; The feature extraction module 702 is used to process the voiceprint signal to extract multi-scale time-frequency features and obtain a time-frequency tensor. The input module 703 is used to input the time-frequency tensor into a pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism; Output module 704 is used to output transformer state identification results through the multi-scale time-frequency model.

[0086] The transformer state identification method and apparatus based on multi-scale time-frequency features provided by this invention have the following advantages: High real-time performance: inference latency <25ms, meeting the real-time monitoring requirements of substations; Highly robust: Maintains an accuracy of >90% even with a signal-to-noise ratio of -5dB; High portability: The model can run on a variety of embedded devices; Low maintenance cost: adaptive updates in the field, no retraining required; Enhanced security: Enables local decision-making and reduces reliance on the cloud.

[0087] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A transformer state identification method based on multi-scale time-frequency features, characterized in that, include: Acquire acoustic signature signals during transformer operation; By processing the voiceprint signal, multi-scale time-frequency features are extracted to obtain the time-frequency tensor; The time-frequency tensor is input into a pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism; The transformer state identification result is output through the multi-scale time-frequency model.

2. The method according to claim 1, characterized in that, By processing the voiceprint signal, multi-scale time-frequency features are extracted to obtain a time-frequency tensor, including: The voiceprint signal is preprocessed, including pre-emphasis, framing, and windowing; The preprocessed voiceprint signal is subjected to a short-time Fourier transform to obtain the first time-frequency diagram; Continuous wavelet transform is performed on the preprocessed voiceprint signal to extract feature changes at different time scales, resulting in a second time-frequency map. The frequency characteristics of the preprocessed acoustic signature signal are converted into Mel frequencies using the Mel filter bank to obtain the Mel spectrum. The first time-frequency graph, the second time-frequency graph, and the Mel spectrum graph are concatenated into a three-channel input tensor to obtain the time-frequency tensor.

3. The method according to claim 1, characterized in that, The multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural networks, and a two-branch attention mechanism, including: Training data is collected in the cloud. The training data includes voiceprint data under different transformer conditions, including normal, loose core, winding fault, partial discharge, and mechanical looseness. The initial network for transformer state recognition is constructed in the cloud, including a multi-scale convolutional front-end, a time-frequency cross-attention module, a lightweight Transformer encoder, an edge SE fusion module, and an output layer. The time-frequency cross-attention module includes a time attention branch, a frequency attention branch, and a cross-fusion layer. The cloud platform iteratively trains the initial network until convergence based on the training data to obtain the multi-scale time-frequency model. The multi-scale convolutional front end is used to extract multi-scale features from the training data using convolutional kernels of different sizes; the time-frequency cross-attention module is used to reorganize the multi-scale features and perform self-attention calculation through time attention branches and frequency attention branches, and to perform feature fusion through a cross-fusion layer; the lightweight Transformer encoder is used to encode through a factorization multi-head attention mechanism; the edge SE fusion module is used to recalibrate the channel weights; and the output layer is used to perform global average pooling on the time-frequency features output by the edge SE fusion module and output the transformer state category probability.

4. The method according to claim 3, characterized in that, The cloud platform iteratively trains the initial network until convergence based on the training data, including: Step 1: Extract the time-frequency tensor from the training data, input it into the multi-scale convolution front end, use convolution kernels of different sizes to perform multi-scale feature extraction in parallel, output time-frequency features under different frequency bands, perform channel concatenation, and then activate it with BatchNorm and ReLU to form a fused feature tensor; Step 2: Input the fused feature tensor into the time-frequency cross-attention module, perform recombination and self-attention calculation through the time attention branch and the frequency attention branch, and perform feature fusion through the cross-fusion layer; Step 3: Input the output of the time-frequency cross-attention module into the lightweight Transformer encoder and process it through the factorization multi-head attention mechanism; Step 4: Input the output of the lightweight Transformer encoder into the edge SE fusion module, and recalibrate the channel weights using the squeeze excitation mechanism; Step 5: Input the output of the edge SE fusion module into the output layer, perform global average pooling, and then map it to the transformer state category through a fully connected layer and Softmax function to output the transformer state category probability; Step Six: Repeat steps one through five until convergence, completing the training.

5. The method according to claim 3, characterized in that, After iteratively training the initial network in the cloud using the training data until convergence to obtain the multi-scale time-frequency model, the process further includes: The cloud compresses the multi-scale time-frequency model to adapt it to edge nodes; The cloud distributes the compressed multi-scale time-frequency model to the edge nodes of each substation.

6. The method according to any one of claims 1 to 5, characterized in that, Also includes: Local storage of voiceprint data that has not been labeled within the most recent preset time period; The unlabeled voiceprint data is grouped using a sliding window of a preset size; The transformer state category is identified for each group of grouped data using a locally stored multi-scale time-frequency model, resulting in pseudo-label data for each group. The multi-scale time-frequency model stored locally is trained and its parameters are updated based on pseudo-labeled data and real labeled data.

7. The method according to any one of claims 1 to 5, characterized in that, After outputting the transformer state identification result through the multi-scale time-frequency model, the method further includes: The transformer status identification results and voiceprint summaries are uploaded to the cloud. The cloud uses a clustering algorithm to select data to participate in model retraining based on the voiceprint summaries and transformer status identification results of the edge nodes of multiple substations. When the performance indicators of the new model are better than the current version of the model in the cloud, a new model file is generated and sent to the edge nodes of each substation.

8. The method according to claim 7, characterized in that, Also includes: Receive the new model file from the cloud locally; The new model file is decrypted and its signature verified to obtain the new model. Determine whether the prediction accuracy of the new model is better than that of the current local model. If so, the verification passes, and the current local version of the model is hot-replaced to complete the model update.

9. The method according to claim 7, characterized in that, After uploading the transformer status identification results and voiceprint summary to the cloud, the cloud also provides a voiceprint feature heatmap, status classification probability, and anomaly trend curve.

10. An apparatus for implementing the transformer state identification method based on multi-scale time-frequency features as described in any one of claims 1 to 9, characterized in that, include: The acquisition module is used to acquire acoustic fingerprint signals during transformer operation; The feature extraction module is used to process the voiceprint signal to extract multi-scale time-frequency features and obtain a time-frequency tensor. The input module is used to input the time-frequency tensor into a pre-constructed multi-scale time-frequency model, wherein the multi-scale time-frequency model is constructed based on multi-scale convolution, Transformer neural network and dual-branch attention mechanism; The output module is used to output the transformer state identification result through the multi-scale time-frequency model.

Images