Transformer voiceprint fault recognition method and system based on three-dimensional cross-attention mechanism

Abstract

The application discloses a transformer voiceprint fault identification method and system based on a three-dimensional cross attention mechanism, comprising: preprocessing collected transformer voiceprint signal data, obtaining frame signals, extracting feature vectors of the frame signals, and performing regularization processing on the feature vectors, then calculating the transverse cross-branch attention weight between different branch feature vectors, so as to obtain the transverse attention output of the multi-branch feature vectors. Then, taking the same feature point of the different branch feature vectors as a granularity vector, the longitudinal cross-granularity attention weight between the unit vectors is calculated to obtain the longitudinal cross-granularity attention output. The transverse and longitudinal attention outputs are linearly added to obtain a three-dimensional cross attention output result. The application can improve the noise resistance performance of the identification model, has good generalization and adaptability, and effectively supports the voiceprint fault identification demand of the transformer.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology and relates to a method and system for identifying transformer acoustic fingerprint faults based on a three-dimensional cross-attention mechanism. Background Technology

[0002] Transformers are core hub equipment in power systems, and their stable operation is crucial to the power system. Under the influence of complex and diverse power operation scenarios, transformers may experience material degradation, component wear, and decreased insulation performance, leading to a series of mechanical faults such as abnormal vibration and partial discharge, and in severe cases, even causing large-scale power grid blackouts. Therefore, research on transformer mechanical fault identification is particularly urgent. Due to the effects of magnetostriction, electromagnetic forces, and other factors, potential mechanical defects in transformers are often accompanied by abnormal acoustic signatures. Therefore, identifying transformer mechanical faults based on transformer acoustic signature data has become a feasible solution. However, current identification methods still suffer from problems such as relatively simple identification models, low model accuracy, poor robustness, and performance being highly susceptible to interference from external environmental noise. In addition, how to generalize and adapt the model for different types and scales of transformers also requires in-depth research. Summary of the Invention

[0003] The purpose of this invention is to solve the problems of simple identification models, low model accuracy, poor robustness, and performance being easily affected by external environmental noise in the existing technology. At the same time, the problem of how to generalize and adapt the model for transformers of different models and scales also needs in-depth research. This invention provides a transformer acoustic fingerprint fault identification method and system based on a three-dimensional cross-attention mechanism.

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] A transformer acoustic fingerprint fault identification method based on a three-dimensional cross-attention mechanism includes:

[0006] The collected transformer acoustic signature data is preprocessed to obtain frame signals;

[0007] Feature vectors are extracted from the frame signal to obtain several branch feature vectors. Regularization and normalization are performed on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors.

[0008] The lateral cross-branch attention weights between the feature vectors of different branches after regularization are calculated to obtain the lateral attention output of the multi-branch feature vectors.

[0009] For the regularized feature vectors of different branches, the same feature point of the feature vectors of different branches is used as the granularity vector. The vertical cross-granularity attention weight between unit vectors is calculated to obtain the vertical cross-granularity attention output of the multi-branch feature vectors.

[0010] Based on the horizontal attention output and vertical cross-granularity attention output of multi-branch feature vectors, the mechanical fault identification results of transformers are obtained.

[0011] A further improvement of the present invention is that:

[0012] Furthermore, the acquired transformer acoustic signature signal data is preprocessed to obtain frame signals. Specifically, the preprocessing includes pre-emphasis, framing, and windowing. The pre-emphasis is used to suppress the spectral tilt of the acoustic signature signal and achieve a balanced spectrum. The framing is used to suppress the time-varying characteristics of the transformer acoustic signature signal data. The windowing is used to reduce signal abrupt changes and spectral leakage at frame edges.

[0013] Furthermore, feature vectors are extracted from the frame signal to obtain several branch feature vectors. Specifically, the multi-branch feature vectors of the frame signal include MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening features. Let the frame signal be x(n), n = 1, 2, ..., N.

[0014] Feature extraction is performed on the MFCC coefficients, specifically as follows:

[0015] Step 1.1: Perform Fast Fourier Transform on the frame signal

[0016]

[0017] Step 1.2: Calculate the energy spectrum of the frequency domain signal after Fourier transform.

[0018] Step 1.3: Convert the frequency f to Mel scale

[0019] Step 1.4: Divide the Mel scale axis into K+1 segments, and let the Mel scale frequency at each point be f(m);

[0020] Step 1.5: Configure a Mel-triangle filter H between every two segments. m (k), a total of K are configured:

[0021]

[0022] Step 1.6: Calculate H for each filter m The logarithmic energy s(m) of (k):

[0023]

[0024] Step 1.7: Perform Discrete Cosine Transform to obtain the MFCC coefficients:

[0025]

[0026] The LPC coefficients are extracted, specifically by calculating the autocorrelation function of the frame signal.

[0027]

[0028] The P-order LPC coefficients of the framed signal are calculated iteratively based on the Levinson-Durbin recursive algorithm.

[0029] Vibration entropy reflects the complexity of sound vibration. After calculating the Fourier transform of the framed signal, the vibration entropy is obtained from the frequency domain signal, as shown in formula (6):

[0030]

[0031] The spectral center is used to describe the energy concentration range of the acoustic signal, as shown in formula (7):

[0032]

[0033] Spectral broadening is used to describe the width of the distribution of spectral energy in an audio signal, as shown in formula (8):

[0034]

[0035] Among them, A k The amplitude of the frequency domain signal.

[0036] Furthermore, the feature vectors of different branches are regularized and normalized to obtain the regularized and normalized feature vectors of different branches. Specifically, based on the obtained MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening, let the vector lengths of the MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening be D1, D2, D3, D4, and D5, respectively, and let the number of frames be N; the branch vectors with different lengths are normalized to a uniform length of D.

[0037] D = max{D1,D2,D3,D4,D5} (9)

[0038] Vectors that are too short are padded with zeros; after normalization, the five branch vectors are denoted as follows:

[0039] F1,F2,F3,F4,F5∈R N×D (10).

[0040] Furthermore, the lateral cross-branch attention weights between the different branch feature vectors after regularization are calculated to obtain the lateral attention output of the multi-branch feature vectors. Specifically, let the different multi-branch feature vectors be F. n n = 1, 2, ..., 5

[0041] Step 2.1: Calculate the attention value query vector q for different multi-branch feature vectors. n Key vector k n Sum vector v n ;

[0042]

[0043] Among them, W q W k W v ∈R 512×N These are the feature parameters that need to be learned in the network;

[0044] Step 2.2: Calculate the lateral cross-branch attention weight 'a' n,n′ ∈R 512×512 ,a n,n′ It is the branch vector F n and F n' Cross-attention weights between them;

[0045]

[0046] Step 2.3: Calculate the cross-branch attention value vector:

[0047]

[0048] Step 2.4: Cross-branch attention branch output After passing through a linear output layer, it is transformed into a multi-branch feature vector F. n Same dimensions, then through residual layers and F n Direct computation is performed, and finally, the cross-branch attention output is obtained through an average pooling layer.

[0049] Step 2.4.1: Output the cross-branch attention branch. Transform into F n Same dimensions:

[0050]

[0051] Step 2.4.2: After passing through the residual layer and pooling layer, the final lateral cross-branch attention output is obtained.

[0052]

[0053] in, Dimensions and F n Maintain consistency.

[0054] Furthermore, for the regularized feature vectors of different branches, using the same feature point as the granularity vector, the vertical cross-granularity attention weights between unit vectors are calculated to obtain the vertical cross-granularity attention output of the multi-branch feature vectors, specifically:

[0055] Step 3.1: For the feature vectors F of different branches n Feature points at the same pixel coordinates (i,j) are denoted as follows:

[0056] F n (i,j) i=1,2,…,N,j=1,2,…,D (16)

[0057] The feature points at the same pixel coordinate point (i,j) are used to form the granularity vector f of that pixel. i,j :

[0058] f i,j =[F1(i,j),F2(i,j),…,F5(i,j)],f i,j ∈R 5 (17)

[0059] Step 3.2: Apply granularity vector f i,j Perform feature activation; let the activated feature granularity vector be e. i,j ∈R 512 The activation feature weight vector is W. i,j ∈R 5×512 :

[0060] e i,j =f i,j ·W i,j (18)

[0061] Wherein, the activation feature weight vector W i,j It is the parameter vector that is subsequently updated by the backpropagation algorithm;

[0062] Step 3.3: Calculate the longitudinal cross-granularity cross-attention weight b i,j,i′,j′ ∈R 512×512 b i,j,i′,j′ It is the feature granularity vector e at pixel coordinates (i,j). i,j and the feature granularity vector e at pixel coordinates (i′, j′) i′,j′ The cross-attention weights are shown in Equation (19):

[0063] b i,j,i′ ,j′ =softmax(ReLU(e i,j ,e i′,j′ (19)

[0064] Step 3.4: Calculate the longitudinal cross-granularity cross-attention output:

[0065]

[0066] Step 3.5: Restore the activated feature granularity vector, maintaining the same scale as the input granularity vector; similar to the lateral cross-branch attention output, it also undergoes residual calculation and pooling processing to obtain...

[0067]

[0068] Where S i,j ∈R 512×5 The feature recovery vector, and W i,j Similarly, both are parameter vectors that need to be learned; the last one... and Maintain consistency in feature dimensions; The granular vector data is reorganized and presented as a multi-branch vector. The new multi-branch feature vector is denoted as...

[0069] Furthermore, based on the lateral attention output and vertical cross-granularity attention output of the multi-branch feature vectors, the transformer mechanical fault identification results are obtained, specifically as follows:

[0070] The horizontal cross-branch cross-attention output and the vertical cross-granularity cross-attention output are linearly added together to form the three-dimensional cross-attention output M. n :

[0071]

[0072] The 3D cross-attention output M n The data is integrated and passed through a multilayer perceptron to obtain the final recognition result. Let the multilayer perceptron function be MLP(·) and the concatenation function be concat(·). The output expression is shown below:

[0073] Output = MLP(concat(M n ),n=1,2,…,5 (23).

[0074] A transformer acoustic fingerprint fault identification system based on a three-dimensional cross-attention mechanism includes:

[0075] The preprocessing module preprocesses the acquired transformer acoustic signature data to obtain frame signals;

[0076] The regularization module extracts feature vectors from the frame signal, obtains several branch feature vectors, and performs regularization and normalization on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors.

[0077] The first acquisition module calculates the lateral cross-branch attention weights between different branch feature vectors after regularization and normalization, and obtains the lateral attention output of the multi-branch feature vectors.

[0078] The second acquisition module calculates the longitudinal cross-granularity attention weight between unit vectors by taking the same feature point of the different branch feature vectors after regularization as the granularity vector, and obtains the longitudinal cross-granularity attention output of the multi-branch feature vector.

[0079] The third acquisition module acquires the transformer mechanical fault identification results based on the horizontal attention output and vertical cross-granularity attention output of the multi-branch feature vector.

[0080] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the method described above.

[0081] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] This invention collects acoustic signature signals from transformers using an acoustic signature sensor. After preprocessing the acoustic signature signals, it extracts multi-branch feature vectors and regularizes them. Then, it calculates the lateral cross-branch attention weights between different branch feature vectors, thus obtaining the lateral attention output of the multi-branch feature vectors. Next, using the same feature point of different branch feature vectors as the granularity vector, it calculates the longitudinal cross-granularity attention weights between unit vectors, obtaining the longitudinal cross-granularity attention output. The lateral and longitudinal attention outputs are linearly added to obtain the three-dimensional cross-attention output. This invention improves the noise resistance of the identification model by introducing a three-dimensional cross-attention mechanism, while also exhibiting good generalization and adaptability, effectively supporting the acoustic signature fault identification requirements of transformers. Attached Figure Description

[0084] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 The flowchart shows the transformer acoustic fingerprint fault identification method based on the three-dimensional cross-attention mechanism of the present invention.

[0086] Figure 2 Flowchart of preprocessing operations for voiceprint signals;

[0087] Figure 3 A schematic diagram illustrating the process of extracting multi-branch feature vectors from frame signals;

[0088] Figure 4 A flowchart illustrating the process of obtaining cross-branch attention output.

[0089] Figure 5 A flowchart illustrating the process of obtaining vertical cross-granularity cross-attention output;

[0090] Figure 6 A diagram illustrating the integration of cross-branch attention outputs to obtain the final recognition result;

[0091] Figure 7 This is a schematic diagram of the transformer acoustic fingerprint fault identification system based on the three-dimensional cross-attention mechanism of the present invention.

[0092] Figure 8 Schematic diagram of different types of voiceprint samples;

[0093] Figure 9 This is a schematic diagram illustrating the accuracy of the present invention on the training and test sets; wherein, Figure 9 (a) is a schematic diagram of the accuracy of the present invention on the training set; Figure 9 (b) is a schematic diagram of the accuracy of the present invention on the test set. Detailed Implementation

[0094] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0095] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0096] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0097] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0098] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0099] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0100] The present invention will now be described in further detail with reference to the accompanying drawings:

[0101] See Figure 1 This invention discloses a transformer acoustic signature fault identification method based on a three-dimensional cross-attention mechanism, comprising:

[0102] S101 preprocesses the collected transformer acoustic signature data to obtain frame signals.

[0103] See Figure 2The preprocessing includes pre-emphasis, framing, and windowing; the pre-emphasis is used to suppress the spectral tilt of the acoustic signature signal and achieve a balanced spectrum; the framing is used to suppress the time-varying characteristics of the transformer acoustic signature signal data; and the windowing is used to reduce signal abrupt changes and spectral leakage at frame edges.

[0104] Common window functions include the Hamming window and the Hanning window. Let M be the length of the Hamming window, L be the length of the audio signal, and S be the step size of the Hamming window. Then the relationship between the number of frames N after the voiceprint signal is segmented and M, L, and S is as follows:

[0105]

[0106] After preprocessing, the voiceprint signal is divided into several frame signals.

[0107] S102, extract feature vectors from the frame signal, obtain several branch feature vectors, and perform regularization and normalization on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors.

[0108] See Figure 3 The multi-branch feature vector of the frame signal includes MFCC coefficients, LPC coefficients, vibration entropy, spectral center and spectral broadening features; let the frame signal be x(n), n=1,2,…,N;

[0109] S102.1 performs feature extraction on the MFCC coefficients. MFCC is a technique that converts an audio signal into a set of coefficients that can capture spectral features, and it is widely used in speech and audio processing. Specifically:

[0110] S102.1.1 Perform Fast Fourier Transform on the frame signal

[0111]

[0112] S102.1.2 Calculate the energy spectrum of the frequency domain signal after Fourier transform.

[0113] S102.1.3 Convert the frequency f to Mel scale.

[0114] S102.1.4 Divide the Mel scale axis into K+1 segments (K+2 points in total), and let the Mel scale frequency at each point be f(m);

[0115] S102.1.5, A Mel-triangle filter H is configured between every two segments. m (k), a total of K are configured:

[0116]

[0117] S102.1.6 Calculate H for each filter m The logarithmic energy s(m) of (k):

[0118]

[0119] S102.1.7. Perform discrete cosine transform to obtain the MFCC coefficients:

[0120]

[0121] S102.2 Extracting the LPC coefficients, specifically: Calculating the autocorrelation function of the frame signal:

[0122]

[0123] The P-order LPC coefficients of the framed signal are calculated iteratively based on the Levinson-Durbin recursive algorithm.

[0124] S102.3. Vibration entropy reflects the complexity of sound vibration. After calculating the Fourier transform of the framed signal, the vibration entropy is obtained from the frequency domain signal, as shown in formula (6):

[0125]

[0126] S102.4, The spectrum center is used to describe the energy concentration range of the voiceprint signal, as shown in formula (7):

[0127]

[0128] S102.5, Spectral broadening is used to describe the width of the spectral energy distribution of an audio signal, as shown in formula (8):

[0129]

[0130] Among them, A k The amplitude of the frequency domain signal.

[0131] S102.6. Based on the obtained MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening, let the vector lengths of the MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening be D1, D2, D3, D4, and D5, respectively, and let the number of frames be N; standardize the branch vectors with different lengths, and unify the length to D:

[0132] D = max{D1,D2,D3,D4,D5} (9)

[0133] Vectors that are too short are padded with zeros; after normalization, the five branch vectors are denoted as follows:

[0134] F1,F2,F3,F4,F5∈R N×D (10).

[0135] S103 calculates the lateral cross-branch attention weights between different branch feature vectors after regularization to obtain the lateral attention output of the multi-branch feature vectors.

[0136] See Figure 4 Let F be the different multi-branch feature vectors. n n = 1, 2, ..., 5;

[0137] S103.1: Calculate the attention value query vector q for different multi-branch feature vectors. n Key vector k n Sum vector v n ;

[0138]

[0139] Among them, W q W k W v ∈R 512×N These are the feature parameters that need to be learned in the network;

[0140] S103.2: Calculate the lateral cross-branch attention weight a n,n′ ∈R 512×512 ,a n,n′ It is the branch vector F n and F n' Cross-attention weights between them;

[0141]

[0142] S103.3: Calculate the cross-branch attention value vector:

[0143]

[0144] S103.4: Cross-branch attention branch output After passing through a linear output layer, it is transformed into a multi-branch feature vector F. n Same dimensions, then through residual layers and F n Direct computation is performed, and finally, the cross-branch attention output is obtained through an average pooling layer.

[0145] S103.4.1: Output cross-branch attention branches Transform into F n Same dimensions:

[0146]

[0147] S103.4.2: The final lateral cross-branch attention output is obtained after passing through the residual layer and pooling layer.

[0148]

[0149] in, Dimensions and F n Maintaining consistency. After the above operations, the cross-branch attention output is obtained. Traditional attention mechanisms mainly focus on information interaction within a single branch, while cross-branch attention allows information exchange between different branches. This allows for the sharing and integration of different feature representations across multiple branches, thereby improving the model's ability to model complex relationships. Cross-branch attention expands the expressive power of traditional attention mechanisms. By introducing interaction between branches, it allows the model to more flexibly control and adjust the importance and weights of different branches, thus better adapting to complex tasks and data distributions.

[0150] S104: For the regularized feature vectors of different branches, the same feature point of the feature vectors of different branches is used as the granularity vector. The vertical cross-granularity attention weight between unit vectors is calculated to obtain the vertical cross-granularity attention output of the multi-branch feature vector.

[0151] See Figure 5 S104.1: For different branch feature vectors F n Feature points at the same pixel coordinates (i,j) are denoted as follows:

[0152] F n (i,j) i=1,2,…,N,j=1,2,…,D (16)

[0153] The feature points at the same pixel coordinate point (i,j) are used to form the granularity vector f of that pixel. i,j :

[0154] f i,j =[F1(i,j),F2(i,j),…,F5(i,j)],f i,j ∈R 5 (17)

[0155] S104.2: For the granularity vector f i,j Perform feature activation; let the activated feature granularity vector be e. i,j ∈R 512 The activation feature weight vector is W. i,j ∈R 5×512 :

[0156] e i,j =f i,j ·Wi,j (18)

[0157] Wherein, the activation feature weight vector W i,j It is the parameter vector that is subsequently updated by the backpropagation algorithm;

[0158] S104.3: Calculate the longitudinal cross-granularity cross-attention weight b i,j,i′,j′ ∈R 512×512 b i,j,i′,j′ It is the feature granularity vector e at pixel coordinates (i,j). i,j and the feature granularity vector e at pixel coordinates (i′, j′) i′,j′ The cross-attention weights are shown in Equation (19):

[0159] b i,j,i′,j′ =softmax(ReLU(e i,j ,e i′ , j′ (19)

[0160] S104.4: Calculate the longitudinal cross-graininess cross-attention output:

[0161]

[0162] S104.5: The activated feature granularity vector is restored to maintain the same scale as the input granularity vector; similar to the lateral cross-branch attention output, it also undergoes residual calculation and pooling processing to obtain...

[0163]

[0164] Where S i,j ∈R 512×5 The feature recovery vector, and W i,j Similarly, both are parameter vectors that need to be learned; the last one... and Maintain consistency in feature dimensions; The granular vector data is reorganized and presented as a multi-branch vector. The new multi-branch feature vector is denoted as... Compared to traditional attention mechanisms that typically focus only on local information, cross-granularity attention mechanisms consider feature information at different granularities simultaneously, enhancing the model's feature representation capabilities by fusing information from both global and local perspectives.

[0165] S105 obtains the transformer mechanical fault identification results based on the horizontal attention output and vertical cross-granularity attention output of the multi-branch feature vector.

[0166] See Figure 6The horizontal cross-branch cross-attention output and the vertical cross-granularity cross-attention output are linearly added together to form the three-dimensional cross-attention output M. n :

[0167]

[0168] The 3D cross-attention output M n The data is integrated and passed through a multilayer perceptron to obtain the final recognition result. Let the multilayer perceptron function be MLP(·) and the concatenation function be concat(·). The output expression is shown below:

[0169] Output = MLP(concat(M n ),n=1,2,…,5 (23).

[0170] Through the processing steps S101 to S106, the identification results of transformer acoustic mechanical faults are obtained by calculating the transformer multi-branch feature vector and three-dimensional cross-attention output based on the transformer acoustic data.

[0171] See Figure 7 This invention discloses a transformer acoustic signature fault identification system based on a three-dimensional cross-attention mechanism, comprising:

[0172] The preprocessing module preprocesses the acquired transformer acoustic signature data to obtain frame signals;

[0173] The regularization module extracts feature vectors from the frame signal, obtains several branch feature vectors, and performs regularization and normalization on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors.

[0174] The first acquisition module calculates the lateral cross-branch attention weights between different branch feature vectors after regularization and normalization, and obtains the lateral attention output of the multi-branch feature vectors.

[0175] The second acquisition module calculates the longitudinal cross-granularity attention weight between unit vectors by taking the same feature point of the different branch feature vectors after regularization as the granularity vector, and obtains the longitudinal cross-granularity attention output of the multi-branch feature vector.

[0176] The third acquisition module acquires the transformer mechanical fault identification results based on the horizontal attention output and vertical cross-granularity attention output of the multi-branch feature vector.

[0177] Example:

[0178] Taking transformer acoustic fault diagnosis in the power industry as an example, the present invention achieves the above objectives through the following technical solutions:

[0179] Step 1: Perform preprocessing on the voiceprint signal.

[0180] To evaluate the performance of the proposed method, the sampling rate of the collected transformer acoustic signature data samples was set to 16000Hz and divided into 1-second slices, resulting in the collection of 8130 audio slice samples. Different types of acoustic signature samples were included. Figure 8 As shown. To extract multi-branch feature vectors, the voiceprint signal needs to be preprocessed. Voiceprint preprocessing mainly includes three parts: pre-emphasis, framing, and windowing. Pre-emphasis is to suppress the spectral tilt of the voiceprint signal to achieve a balanced spectrum. Since the voiceprint signal has time-varying characteristics, direct feature extraction will result in unstable features, directly affecting subsequent model training and inference. To suppress time-varying characteristics, the voiceprint signal needs to be framed. To reduce signal abrupt changes and spectral leakage at frame edges, the framed signal needs to be windowed. Common window functions include Hamming windows and Hanning windows. This invention chooses to use a Hamming window. Let M be the Hamming window length, L be the audio signal length, and S be the Hamming window step size. The relationship between the number of frames N after voiceprint signal segmentation and M, L, and S is as follows.

[0181]

[0182] After preprocessing, the voiceprint signal is divided into several frame signals.

[0183] Step 2: Extract the multi-branch feature vector of the frame signal. The multi-branch feature vector of the frame signal consists of five parts: MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening features. The following is the process for extracting different features. Let the frame signal be x(n), n=1,2,…,N.

[0184] Step 2.1: Extract Mel Frequency Cepstral Coefficients (MFCCs). MFCCs are a technique that converts audio signals into a set of coefficients that capture spectral characteristics, and are widely used in speech and audio processing.

[0185] Step 2.1.1: Perform Fast Fourier Transform on the frame signal

[0186] Step 2.1.2: Calculate the energy spectrum of the frequency domain signal after Fourier transform.

[0187] Step 2.1.3: Convert the frequency f to Mel scale.

[0188] Step 2.1.4: Divide the Mel scale axis into K+1 segments (K+2 points in total), and let the Mel scale frequency at each point be f(m);

[0189] Step 2.1.5: Configure a Mel-triangle filter H between every two segments. m (k), a total of K are configured:

[0190]

[0191] Step 2.1.6: Calculate H for each filter m The logarithmic energy s(m) of (k):

[0192]

[0193] Step 2.1.7: Perform Discrete Cosine Transform to obtain the MFCC coefficients:

[0194]

[0195] Step 2.2: Extract Linear Predictive Coding (LPC) coefficients. Predictive coefficients represent the characteristics of the filter and can capture features such as formants and harmonics in the signal. Calculate the autocorrelation function of the frame signal:

[0196]

[0197] The P-order LPC coefficients of the framed signal are calculated iteratively using the Levinson-Durbin recursive algorithm.

[0198] Step 2.3: Calculate vibration entropy, spectral center, and spectral broadening. Vibration entropy reflects the complexity of sound vibration; therefore, it is introduced as a characteristic feature. After calculating the Fourier transform of the framed signal, the formula for calculating vibration entropy from the frequency domain signal is shown below:

[0199]

[0200] The spectrum center is used to describe the energy concentration range of the acoustic signature signal, and the calculation formula is as follows:

[0201]

[0202] Spectral broadening describes the width of the spectral energy distribution of an audio signal, and is calculated using the following formula:

[0203]

[0204] Among them, A kThe amplitude of the frequency domain signal.

[0205] Step 2.4: Normalize the multi-branch vectors. The previous steps have calculated the MFCC coefficient, LPC coefficient, vibration entropy, spectral center, and spectral broadening. For ease of subsequent calculations, the multi-branch feature vectors need to be regularized. Let the lengths of these vectors be D1, D2, D3, D4, and D5, and the number of frames be N. Since the lengths of different branch vectors are different, they need to be normalized to a uniform length of D.

[0206] D = max{D1,D2,D3,D4,D5}

[0207] Vectors shorter than the specified length are padded with zeros. After normalization, the five branch vectors are denoted as follows:

[0208] F1,F2,F3,F4,F5∈R N×D

[0209] Step 3: Calculate the lateral cross-branch attention output. Let F be the feature vectors of different multi-branch branches. n For n = 1, 2, ..., 5, the following is the process for calculating the cross-branch attention output.

[0210] Step 3.1: Calculate the attention value query vector for each. qn Key vector k n Sum vector v n .

[0211] q n =W q •F n

[0212] k n =W k •F n

[0213] v n =W v •F n

[0214] Among them, W q W k W v ∈R 512×N These are the feature parameters that need to be learned in the network.

[0215] Step 3.2: Calculate the lateral cross-branch attention weight 'a' n,n′ ∈R 512×512 ,a n,n′ It is the branch vector F n and F n' Cross-attention weights between them.

[0216]

[0217] Step 3.3: Calculate the cross-branch attention value vector:

[0218]

[0219] Step 3.4: Pooling and residual connections. Cross-branch attention branch output. After passing through a linear output layer, it is transformed into a multi-branch feature vector F. n Same dimensions, then through residual layers and F n Direct computation is performed, and finally, cross-branch attention output is obtained through an average pooling layer.

[0220] Step 3.4.1: Output the cross-branch attention branch. Transform into F n Same dimensions:

[0221]

[0222] Step 3.4.2: After passing through the residual layer and pooling layer, the final lateral cross-branch attention output is obtained.

[0223]

[0224] in, Dimensions and F n Maintaining consistency. After the above operations, the cross-branch attention output is obtained. Traditional attention mechanisms mainly focus on information interaction within a single branch, while cross-branch attention allows information exchange between different branches. This allows for the sharing and integration of different feature representations across multiple branches, thereby improving the model's ability to model complex relationships. Cross-branch attention expands the expressive power of traditional attention mechanisms. By introducing interaction between branches, it allows the model to more flexibly control and adjust the importance and weights of different branches, thus better adapting to complex tasks and data distributions.

[0225] Step 4: Calculate the longitudinal cross-granularity cross-attention output. This is done by applying the feature vectors F from the different branches mentioned above. n Building upon this, the following section presents the process for calculating the vertical cross-granularity cross-attention output.

[0226] Step 4.1: Calculate the granularity vector for different pixel coordinates. For different branch feature vectors F... n Feature points at the same pixel coordinates (i,j) are denoted as follows:

[0227] Fn (i,j)i=1,2,…,N,j=1,2,…,D

[0228] The feature points at the same pixel coordinate point (i,j) are used to form the granularity vector f of that pixel. i,j :

[0229] f i,j =[F1(i,j),F2(i,j),…,F5(i,j)],f i,j ∈R 5

[0230] Step 4.2: Feature Activation. To improve the granularity vector f i,j Information representation capability for granular vector f i,j Perform feature activation. Let the activated feature granularity vector be e. i,j ∈R 512 The activation feature weight vector is W. i,j ∈R 5×512 :

[0231] e i,j =f i,j ·W i,j

[0232] Wherein, the activation feature weight vector W i,j It is the parameter vector that is subsequently updated by the backpropagation algorithm.

[0233] Step 4.3: Calculate the longitudinal cross-granularity cross-attention weight b i,j,i′,j′ ∈R 512×512 b i,j,i′,j′ It is the feature granularity vector e at pixel coordinates (i,j). i,j and the feature granularity vector e at pixel coordinates (i′, j′) i′,j′ The cross-attention weights between the two are calculated using the following formula:

[0234] b i,j,i′,j′ =softmax(ReLU(e i,j ,e i′ , j′ ))

[0235] Step 4.4: Calculate the longitudinal cross-granularity cross-attention output:

[0236]

[0237] Step 4.5: Feature Compression. The activated feature granularity vector is restored, maintaining the same scale as the input granularity vector. Similar to the lateral cross-branch attention output above, it undergoes residual calculation and pooling processing to obtain...

[0238]

[0239] Among them, S i,j ∈R 512×5 The feature recovery vector, and W i,j Similarly, both are parameter vectors that need to be learned. The final one... and Maintain consistency in feature dimensions. The granular vector data is reorganized and presented as a multi-branch vector. The new multi-branch feature vector is denoted as... Compared to traditional attention mechanisms that typically focus only on local information, cross-granularity attention mechanisms consider feature information at different granularities simultaneously, enhancing the model's feature representation capabilities by fusing information from both global and local perspectives.

[0240] Step 5: Integrate the 3D cross-attention outputs to obtain the final recognition result.

[0241] Step 5.1: Feature Addition. The horizontal cross-branch cross-attention output and the vertical cross-granularity cross-attention output are linearly added together to form the three-dimensional cross-attention output M. n :

[0242]

[0243] Finally, the 3D cross-attention output M is... n The data is integrated and passed through a multilayer perceptron to obtain the final recognition result. Let the multilayer perceptron function be MLP(·) and the concatenation function be concat(·). The output expression is shown below:

[0244] Output = MLP(concat(M n n = 1, 2, ..., 5

[0245] Thus, through the processing steps one through five, based on the transformer acoustic fingerprint data, the identification results of transformer acoustic fingerprint mechanical faults were obtained by calculating the transformer multi-branch feature vector and three-dimensional cross-attention output.

[0246] Step 5.2: Result Comparison. Compared with traditional neural network identification methods (CNN, LSTM, GRU), the accuracy of the transformer acoustic print fault identification method based on the three-dimensional cross-attention mechanism on the training and test sets is shown in [link to documentation]. Figure 9 .

[0247] Furthermore, a comparison with traditional methods in terms of accuracy, recall, precision, and F1 score is shown in Table 1:

[0248] Table 1 Performance comparison between the present invention and classical algorithms

[0249]

[0250] As shown in Table 1, this model demonstrates significant advantages over traditional methods in terms of accuracy, recall, precision, and F1 score. This is because the algorithm has a stronger information representation capability, resulting in higher accuracy on the test set. This is because the cross-granularity attention mechanism considers feature information at different granularities simultaneously, enhancing the model's feature representation capability by fusing information from both global and local perspectives.

[0251] An embodiment of the present invention provides a terminal device. This terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the various method embodiments described above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0252] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention.

[0253] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0254] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0255] The memory can be used to store the computer program and / or module. The processor implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.

[0256] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0257] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A transformer voiceprint fault recognition method based on a three-dimensional cross-attention mechanism, characterized in that, include: The collected transformer acoustic signature data is preprocessed to obtain frame signals; Feature vectors are extracted from the frame signal to obtain several branch feature vectors. Regularization and normalization are performed on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors. The transverse cross-branch cross-attention weight between different branch eigenvectors of the canonical normed post-quantization is calculated to obtain the transverse attention output of the multi-branch eigenvector, specifically: let the different multi-branch eigenvectors be ; Step 5.1: Calculate the attention value query vector for different multi-branch feature vectors respectively. Key vector Sum value vector ; in, These are the feature parameters that need to be learned in the network; Step 5.2: Calculate the lateral cross-branch attention weights , It is a branch vector and Cross-attention weights between them; Step 5.3: Calculate the cross-branch attention value vector: Step 5.4: Cross-branch attention branch output After passing through a linear output layer, it is transformed into a multi-branch feature vector. Same dimensions, then through residual layers and Direct computation is performed, and finally, the cross-branch attention output is obtained through an average pooling layer. Step 5.4.1: Output the cross-branch attention branch. Transform into and Same dimensions: Step 5.4.2: After passing through the residual layer and pooling layer, the final lateral cross-branch attention output is obtained. : in, Dimensions and Maintain consistency; For the regularized feature vectors of different branches, using the same feature point as the granularity vector, the vertical cross-granularity attention weights between unit vectors are calculated to obtain the vertical cross-granularity attention output of the multi-branch feature vectors, specifically: Step 6.1: For feature vectors of different branches Same pixel coordinates The feature points are denoted as: This represents the uniform length after normalizing branch vectors of different lengths; Same pixel coordinates The feature points form the granular vector of this pixel. : Step 6.2: For granular vectors Perform feature activation; let the activated feature granularity vector be... The activation feature weight vector is : Among them, the activation feature weight vector It is the parameter vector that is subsequently updated by the backpropagation algorithm; Step 6.3: Calculate the vertical cross-granularity cross attention weights ; Pixel coordinates Feature granularity vector at the location and pixel coordinates Feature granularity vector at the location The cross-attention weights are shown in Equation (19): Step 6.4: Calculate the longitudinal cross-granularity cross-attention output: Step 6.5: Restore the activated feature granularity vector, maintaining the same scale as the input granularity vector; similar to the lateral cross-branch attention output, it also undergoes residual calculation and pooling processing to obtain... in For the feature recovery vector, and Similarly, both are parameter vectors that need to be learned; the last one... and Maintain consistency in feature dimensions; The granular vector data is reorganized and presented as a multi-branch vector. The new multi-branch feature vector is denoted as... ; Based on the horizontal attention output and vertical cross-granularity attention output of multi-branch feature vectors, the mechanical fault identification results of transformers are obtained.

2. The transformer acoustic signature fault identification method based on a three-dimensional cross-attention mechanism according to claim 1, characterized in that, The preprocessing of the acquired transformer acoustic signature signal data to obtain frame signals specifically includes: pre-emphasis, framing, and windowing; the pre-emphasis is used to suppress the spectral tilt of the acoustic signature signal and achieve a balanced spectrum; the framing is used to suppress the time-varying characteristics of the transformer acoustic signature signal data; and the windowing is used to reduce signal abrupt changes and spectral leakage at frame edges.

3. The transformer acoustic signature fault identification method based on a three-dimensional cross-attention mechanism according to claim 2, characterized in that, The extraction of feature vectors from the frame signal to obtain several branch feature vectors specifically involves: the multi-branch feature vectors of the frame signal include MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening features; let the frame signal be... ; Feature extraction is performed on the MFCC coefficients, specifically as follows: Step 2.1: Perform Fast Fourier Transform on the frame signal Step 2.2: Calculate the energy spectrum of the frequency domain signal after Fourier transform. ; Step 2.3: Frequency Convert to Mel scale ; Step 2.4: Divide the Mel scale axis into Segment, let the Mel scale frequency at each point be . ; Step 2.5: Configure a Mel-triangle filter between every two segments. A total of 100 configurations indivual: Step 2.6: Calculate each filter logarithmic energy : Step 2.7: Perform discrete cosine transform to obtain the MFCC coefficients: The LPC coefficients are extracted, specifically by calculating the autocorrelation function of the frame signal. The framed signal is calculated iteratively based on the Levinson-Durbin recursive algorithm. P LPC coefficients of order; Vibration entropy reflects the complexity of sound vibration. After calculating the Fourier transform of the framed signal, the vibration entropy is obtained from the frequency domain signal, as shown in formula (6): The spectral center is used to describe the energy concentration range of the acoustic signal, as shown in formula (7): Spectral broadening is used to describe the width of the spectral energy distribution of an audio signal, as shown in formula (8): in, The amplitude of the frequency domain signal.

4. The transformer acoustic signature fault identification method based on a three-dimensional cross-attention mechanism according to claim 3, characterized in that, The process of regularizing and normalizing the feature vectors of different branches to obtain the regularized and normalized feature vectors of different branches specifically involves: based on the obtained MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening, assuming that the vector lengths of the MFCC coefficients, LPC coefficients, vibration entropy, spectral center, and spectral broadening are respectively... Let the number of frames be ; Standardize the branch vectors of different lengths and unify their lengths to a uniform value. : Vectors that are too short are padded with zeros; after normalization, the five branch vectors are denoted as follows:

5. The transformer acoustic signature fault identification method based on a three-dimensional cross-attention mechanism according to claim 1, characterized in that, The horizontal attention output and vertical cross-granularity attention output based on multi-branch feature vectors are used to obtain the transformer mechanical fault identification results, specifically: The horizontal cross-branch cross-attention output and the vertical cross-granularity cross-attention output are linearly added together to form the three-dimensional cross-attention output. : 3D cross-attention output The data is integrated and passed through a multilayer perceptron to obtain the final recognition result. Let the multilayer perceptron function be... The concatenation function is The output expression is as follows:

6. A transformer acoustic fingerprint fault identification system based on a three-dimensional cross-attention mechanism, used to implement the transformer acoustic fingerprint fault identification method based on a three-dimensional cross-attention mechanism as described in claim 1, characterized in that, include: The preprocessing module preprocesses the acquired transformer acoustic signature data to obtain frame signals; The regularization module extracts feature vectors from the frame signal, obtains several branch feature vectors, and performs regularization and normalization on the different branch feature vectors to obtain the regularized and normalized different branch feature vectors. The first acquisition module calculates the lateral cross-branch attention weights between different branch feature vectors after regularization and normalization, and obtains the lateral attention output of the multi-branch feature vectors. The second acquisition module calculates the longitudinal cross-granularity attention weight between unit vectors by taking the same feature point of the different branch feature vectors after regularization as the granularity vector, and obtains the longitudinal cross-granularity attention output of the multi-branch feature vector. The third acquisition module acquires the transformer mechanical fault identification results based on the horizontal attention output and vertical cross-granularity attention output of the multi-branch feature vector.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5.

Images