Transformer voiceprint recognition method for complex environment

Abstract

The invention discloses a transformer voiceprint recognition method for a complex environment. The method comprises the following steps: S1, collecting transformer voiceprint data; s2, combining a multi-scale learning principle with adaptive expert fusion, and constructing an initial double-layer voiceprint feature fault prediction model; s3, in combination with multi-scale joint loss and expert load balancing loss, constructing a training objective function, and training the initial double-layer voiceprint feature fault prediction model to obtain a double-layer voiceprint feature fault prediction model; and S4, based on the double-layer voiceprint feature fault prediction model, carrying out dynamic reasoning according to real-time computing resources of the equipment, and completing transformer fault judgment. According to the method, multi-scale representation learning and a sparse expert cooperation mechanism are fused, so that the effects of adapting to multiple working conditions, edge device deployment and cross-scale knowledge migration are achieved.

Description

Technical Field

[0001] This invention belongs to the field of voiceprint recognition technology, specifically relating to a method for voiceprint recognition of transformers in complex environments. Background Technology

[0002] As the core hub of power transmission in a power system, the operating status of transformers directly determines the reliability and security of the power grid. Voiceprint signals, acting as the "acoustic fingerprint" of a transformer, can non-invasively and in real-time reflect changes in the internal mechanical structure and electromagnetic characteristics of the equipment. For example, loose core bolts can lead to a 25%-35% increase in the amplitude of the voiceprint in the low-frequency band (50-100Hz), winding deformation can cause harmonic distortion rates exceeding 15% in the 150-250Hz frequency band, and DC bias can cause a single-sided suppression of the 50Hz fundamental frequency component of more than 30%. Therefore, transformer condition monitoring technology based on voiceprint recognition has become a key supporting means for the transformation of power operation and maintenance towards "predictive maintenance."

[0003] However, existing transformer acoustic signature recognition methods face three major technical bottlenecks in practical applications, making it difficult to meet the needs of complex scenarios: First, poor adaptability to multiple operating conditions. Changes in transformer operating load and ambient temperature lead to significant "scale differences" in acoustic signature features—smooth acoustic signature signals under low load and enhanced impulse characteristics under high load. Traditional methods, which use fixed-scale feature extraction (such as Mel spectrum features with a single compression ratio), cannot adapt to the dynamic changes in features under multiple operating conditions. Second, severe performance degradation under high compression ratios. Substation edge monitoring equipment (such as edge gateways and embedded monitoring terminals) has limited computing resources and needs to highly compress acoustic signature signals to reduce computational costs. Existing compression methods (such as average pooling and PCA compression) result in the loss of fine-scale fault features, failing to meet the requirements for accurate identification under high compression ratios. Third, low cross-scale knowledge utilization. Acoustic signature features at different scales (such as coarse-scale features with a compression ratio of 4:1 and fine-scale features with a compression ratio of 32:1) contain complementary information—coarse-scale features reflect the overall operating status of the equipment, while fine-scale features depict the details of local faults. Traditional methods process features at different scales independently without establishing cross-scale knowledge associations, resulting in poor model adaptability in scenarios with dynamic resource changes (such as fluctuations in computing power of edge devices) and an inability to achieve a dynamic balance between accuracy and efficiency.

[0004] In existing technologies, voiceprint recognition methods based on deep residual networks (ResNet) and enhanced channel attention time delay neural networks (EC-TDNN) rely on fixed-scale features and sufficient computational resources, failing to address the issues of adaptability to various operating conditions and resource constraints. While few-sample methods based on meta-learning can handle data scarcity, they do not involve multi-scale feature modeling and cross-scale knowledge transfer. Recognition methods based on a single expert architecture struggle to meet the specialized extraction requirements of features at different scales. Therefore, there is an urgent need for a transformer voiceprint recognition method that can adaptively adapt to multi-scale features, maintain performance under high compression ratios, and is resource-efficient. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a transformer voiceprint recognition method for complex environments, which integrates multi-scale representation learning and sparse expert collaboration mechanism to achieve the effects of adapting to multiple working conditions, edge device deployment, and cross-scale knowledge transfer.

[0006] This invention provides a transformer voiceprint recognition method for complex environments, comprising the following steps:

[0007] S1. Collect transformer acoustic signature data;

[0008] S2. Combining the principles of multi-scale learning with adaptive expert fusion, an initial two-layer voiceprint feature fault prediction model is constructed;

[0009] S3. Combining multi-scale joint loss and expert load balancing loss, a training objective function is constructed to train the initial two-layer voiceprint feature fault prediction model, thus obtaining the two-layer voiceprint feature fault prediction model.

[0010] S4. Based on the dual-layer acoustic signature fault prediction model, dynamic reasoning is performed according to the real-time computing resources of the equipment to complete the transformer fault determination.

[0011] In step S2, the initial dual-layer voiceprint feature fault prediction model includes a multi-scale voiceprint token generation model and a multi-scale adaptive expert fusion feature modeling model. The input of the multi-scale voiceprint token generation model is the original voiceprint data, which generates multi-scale voiceprint tokens that are input into the multi-scale adaptive expert fusion feature modeling model. The multi-scale adaptive expert fusion feature modeling model adaptively extracts and performs cross-scale knowledge association on the input multi-scale voiceprint tokens, and finally outputs the fault probability as the output of the initial dual-layer voiceprint feature fault prediction model.

[0012] The multi-scale voiceprint token generation model includes a noise suppression module and a multi-scale feature extraction module. The noise suppression module first uses adaptive wavelet thresholding for denoising, and then designs a multi-order infinite impulse response notch filter based on adaptive frequency band notch filtering to suppress noise, obtaining denoised voiceprint data, which is then input into the multi-scale feature extraction module. The multi-scale feature extraction module, based on the principle of multi-scale learning, sets the scale level according to the resource constraints of edge devices, extracts the initial multi-scale token sequence, and then performs feature standardization on the initial multi-scale token sequence to obtain the multi-scale voiceprint token sequence.

[0013] The multi-scale adaptive expert fusion feature model is based on a frozen pre-trained language model. The multi-scale adaptive expert fusion module is inserted in parallel into the Transformer layer of the frozen LLM, working in parallel with the original multi-head self-attention and feedforward networks. The final output of the multi-scale adaptive expert fusion module is fused with the output of the original multi-head self-attention and feedforward networks of the LLM. Layer normalization is used to ensure training stability, and the result is used as the final output of the transformer layer. Finally, the transformer layer is processed to obtain the fault prediction probability as the output of the initial two-layer voiceprint feature fault prediction model.

[0014] The adaptive wavelet threshold denoising method specifically involves: using the db6 wavelet basis to perform a 6-level wavelet decomposition on the original transformer acoustic fingerprint data to obtain wavelet coefficients at each scale. , where j is the scale index and k is the time index;

[0015] Soft thresholding is applied to the wavelet coefficients to preserve the transformer acoustic signature characteristics and suppress noise interference. The processed wavelet coefficients are shown below. Express it using the following formula: ;in, It is a symbolic function; The adaptive threshold for the j-th scale;

[0016] Adaptive threshold at scale j Calculate using the following formula: ;in, is the standard deviation of the noise at the j-th scale, which is estimated using the wavelet coefficients of the first scale; N is the length of the acoustic signature signal.

[0017] Processed wavelet coefficients Wavelet reconstruction is performed to obtain a preliminary denoised speaker signal.

[0018] The specific parameter configuration of the multi-order infinite impulse response notch filter is as follows:

[0019] Center frequency Where m is the interference coefficient, which takes the value of 1, 2 or 3, and is for narrowband interference of 50Hz integer multiples generated by cooling fans and cooling pumps;

[0020] The quality factor Q is 12; the notch bandwidth is guaranteed to be no greater than 4Hz to avoid suppressing effective acoustic signature characteristics.

[0021] The stopband attenuation is greater than or equal to 60dB.

[0022] The multi-scale feature extraction module sets five scale levels based on edge device resource constraints, with corresponding compression ratios of 4, 8, 16, 24, and 32, respectively; where the compression ratio is the ratio of the number of voiceprint frames before compression to the number of tokens after compression.

[0023] The initial multi-scale token extraction includes the following steps:

[0024] The voiceprint signal in the denoised voiceprint data is processed by frame segmentation, and 80-dimensional Mel filter bank features are extracted. , where T is the total number of frames; the frame length of the frame segmentation process is 25ms, and the frame shift is 10ms;

[0025] For each scale level r, an adaptive step-size pooling strategy is used to generate the initial voiceprint token sequence for the corresponding scale. It can be expressed using the following formula: ;in, Let be the pooling step size, and ;k is the token index, and j is the feature dimension index, and ; i is the frame number index;

[0026] The feature standardization includes global standardization and local standardization; global standardization applies to the initial multi-scale voiceprint token sequence. Mean and variance normalization is performed to eliminate global feature distribution differences, resulting in a global standard token sequence. It can be expressed using the following formula: ;in, For the initial multi-scale voiceprint token sequence The mean, For the initial multi-scale voiceprint token sequence The standard deviation of is calculated using the following formula: ;in, For token sequence The number of tokens, and ;

[0027] Local standardization for each global standard token sequence The internal 80-dimensional features are locally normalized to obtain a multi-scale voiceprint token sequence. It can be expressed using the following formula: ;in, For each global standard token sequence The feature mean of the k-th token. For each global standard token sequence The feature standard deviation of the k-th token is calculated using the following formula: .

[0028] The multi-scale adaptive expert fusion module includes a shared routing unit, dedicated expert units, and a global expert unit; the number of dedicated experts in the dedicated expert unit... The number of experts in the global expert unit is 23. =1;

[0029] The shared routing unit supports voiceprint token sequences at all scales. Shared weights are used to calculate the voiceprint token sequence for each scale. Matching score between tokens and experts And input it into the dedicated expert unit to match the score. Calculate using the following formula: ;in, Let m be the routing weight of the m-th dedicated expert; The bias for the m-th dedicated expert; This represents the softmax operation on the expert dimension; z is the voiceprint token sequence. The z-th token;

[0030] The dedicated expert unit and the global expert unit adopt a bottleneck structure, specifically linear downsampling-GELU activation-linear upsampling, to achieve feature extraction and dimensionality preservation, expressed by the following formula: ;in, The downsampling weights; This is the downsampling bias; For upsampling weights; For upsampling bias; The activation function for the Gaussian error linear unit;

[0031] The dedicated expert unit performs Top-k sparse activation based on matching scores, selects the k highest-scoring dedicated experts, and calculates corresponding gating values ​​to regulate the output of the dedicated expert unit; the gating values ​​for the dedicated experts... Express it using the following formula: ;

[0032] The output of the multi-scale adaptive expert fusion module The combination of global expert unit output and dedicated expert unit output is represented by the following formula: ;

[0033] The final output of the Transformer layer in the multi-scale adaptive expert fusion feature modeling model is represented by the following formula: ;in, This represents the hidden state of the voiceprint token sequence at scale r for the l-th Transformer layer. The hidden state of the (l-1)th Transformer layer for the speaker token sequence at scale r; FFN is the feedforward network processing; MHSA is the multi-head self-attention processing; This is a layer normalization operation.

[0034] In step S3, the total loss function for model training Express it using the following formula: ;in, For multi-scale joint loss; For expert load balancing losses;

[0035] The model training used the AdamW optimizer with a weight decay coefficient of 0.1 and a cosine annealing learning rate scheduler. The initial learning rate was 1e-3, which linearly decreased to 1e-5 after 10 epochs of training.

[0036] The multi-scale loss For each scale of fault classification loss, a weighted average is calculated based on the scale's importance using the following formula: ; Cross-entropy loss for the model's fault prediction of voiceprint token sequences at scale r; Y represents the fault prediction probability of the model for the voiceprint token sequence at scale r; Y is the voiceprint sample label. Scale weights; cross-entropy loss of the model for fault prediction of voiceprint token sequences at scale r. The following formula is used to measure the difference between the predicted probability and the true label: The model's probability of predicting faults in a voiceprint token sequence at scale r. Calculate using the following formula: ;in, This is the classification weight matrix; For classification bias; This refers to the hidden state of the final transformer layer.

[0037] The expert load balancing loss Express it using the following formula: ;in, Assign a frequency to the token of the m-th dedicated expert; The percentage of the total matching score for the m-th exclusive expert; the token allocation frequency for the m-th exclusive expert. Calculate using the following formula: Where B is the training batch size; The average number of tokens across all scales; The indicator function; the percentage of the total matching score for the m-th specialist. Calculate using the following formula: .

[0038] Step S4 includes the following steps:

[0039] Define resource constraint indicators and select the corresponding voiceprint scale based on the resource constraint indicators;

[0040] A two-layer voiceprint feature fault prediction model is used to infer the fault prediction probability of multi-scale voiceprint token sequences.

[0041] Fault determination is performed by combining the fault prediction probability at the selected scale with the fault prediction probability at adjacent scales.

[0042] Step S4 is as follows:

[0043] Define the resource constraint index C and calculate it using the following formula: Where TOPS represents the remaining computing power to be used; GB represents the remaining memory to be used.

[0044] The corresponding voiceprint scale is dynamically selected based on resource constraint index C, specifically as follows:

[0045] like Then select a voiceprint scale r of 4; if If so, then choose a voiceprint scale r of 16; Then, the voiceprint scale r is selected as 32;

[0046] A two-layer voiceprint feature-based fault prediction model is used to infer fault prediction probabilities from multi-scale voiceprint token sequences. ;

[0047] Fault determination is performed based on the fault prediction probability at a selected scale combined with the fault prediction probabilities at adjacent scales, specifically as follows:

[0048] like If so, the transformer is directly determined to be faulty;

[0049] like If the fault prediction probability of adjacent scales is called, and the fault is determined to be faulty if the fault probability of at least two scales is greater than 0.7, otherwise it is determined to be normal.

[0050] like If it is, then it is directly judged as normal.

[0051] The transformer voiceprint recognition method for complex environments described in this application also includes: reducing inference costs through expert weight quantization and dynamic truncation of silent segments;

[0052] The expert weight quantization is performed by using INT8 quantization to quantize the weights in the shared routing unit, dedicated expert unit, and global expert unit;

[0053] The dynamic truncation of the silent segment is specifically based on the voiceprint signal energy threshold. The energy in the truncated voiceprint signal is less than The silent segment; the voiceprint signal energy threshold Calculate using the following formula: ;in, The average energy of a voiceprint signal labeled as normal. The standard deviation of the energy of a voiceprint signal labeled as normal.

[0054] This invention discloses a transformer voiceprint recognition method for complex environments, which integrates multi-scale representation learning and sparse expert collaboration mechanism to achieve the effects of adapting to multiple working conditions, edge device deployment, and cross-scale knowledge transfer. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0056] This invention provides a transformer voiceprint recognition method for complex environments, the flowchart of which is shown below. Figure 1 As shown, it includes the following steps:

[0057] S1. Collect transformer acoustic signature data;

[0058] S2. Combining the principles of multi-scale learning with adaptive expert fusion, an initial two-layer voiceprint feature fault prediction model is constructed;

[0059] In step S2, the initial dual-layer voiceprint feature fault prediction model includes a multi-scale voiceprint token generation model and a multi-scale adaptive expert fusion feature modeling model. The input of the multi-scale voiceprint token generation model is the original voiceprint data, which generates multi-scale voiceprint tokens that are input into the multi-scale adaptive expert fusion feature modeling model. The multi-scale adaptive expert fusion feature modeling model adaptively extracts and performs cross-scale knowledge association on the input multi-scale voiceprint tokens, and finally outputs the fault probability as the output of the initial dual-layer voiceprint feature fault prediction model.

[0060] The multi-scale voiceprint token generation model includes a noise suppression module and a multi-scale feature extraction module. The noise suppression module first uses adaptive wavelet thresholding for denoising, and then designs a multi-order infinite impulse response notch filter based on adaptive frequency band notch filtering to suppress noise, obtaining denoised voiceprint data, which is then input into the multi-scale feature extraction module. The multi-scale feature extraction module, based on the principle of multi-scale learning, sets the scale level according to the resource constraints of edge devices, extracts the initial multi-scale token sequence, and then performs feature standardization on the initial multi-scale token sequence to obtain the multi-scale voiceprint token sequence.

[0061] The multi-scale adaptive expert fusion feature model is based on a frozen pre-trained language model. The multi-scale adaptive expert fusion module is inserted in parallel into the Transformer layer of the frozen LLM, working in parallel with the original multi-head self-attention and feedforward networks. The final output of the multi-scale adaptive expert fusion module is fused with the output of the original multi-head self-attention and feedforward networks of the LLM. Layer normalization is used to ensure training stability, and the result is used as the final output of the transformer layer. Finally, the transformer layer is processed to obtain the fault prediction probability as the output of the initial two-layer voiceprint feature fault prediction model.

[0062] The adaptive wavelet threshold denoising method specifically involves: using the db6 wavelet basis to perform a 6-level wavelet decomposition on the original transformer acoustic fingerprint data to obtain wavelet coefficients at each scale. , where j is the scale index and k is the time index;

[0063] Soft thresholding is applied to the wavelet coefficients to preserve the transformer acoustic signature characteristics and suppress noise interference. The processed wavelet coefficients are shown below. Express it using the following formula: ;in, It is a symbolic function; The adaptive threshold for the j-th scale;

[0064] Adaptive threshold at scale j Calculate using the following formula: ;in, is the standard deviation of the noise at the j-th scale, which is estimated using the wavelet coefficients of the first scale; N is the length of the acoustic signature signal.

[0065] Processed wavelet coefficients Wavelet reconstruction is performed to obtain a preliminary denoised speaker signal.

[0066] The specific parameter configuration of the multi-order infinite impulse response notch filter is as follows:

[0067] Center frequency Where m is the interference coefficient, which takes the value of 1, 2 or 3, and is for narrowband interference of 50Hz integer multiples generated by cooling fans and cooling pumps;

[0068] The quality factor Q is 12; the notch bandwidth is guaranteed to be no greater than 4Hz to avoid suppressing effective acoustic signature characteristics.

[0069] The stopband attenuation is greater than or equal to 60dB.

[0070] The multi-scale feature extraction module sets five scale levels based on edge device resource constraints, with corresponding compression ratios of 4, 8, 16, 24, and 32, respectively; where the compression ratio is the ratio of the number of voiceprint frames before compression to the number of tokens after compression.

[0071] The initial multi-scale token extraction includes the following steps:

[0072] The voiceprint signal in the denoised voiceprint data is processed by frame segmentation, and 80-dimensional Mel filter bank features are extracted. , where T is the total number of frames; the frame length of the frame segmentation process is 25ms, and the frame shift is 10ms;

[0073] For each scale level r, an adaptive step-size pooling strategy is used to generate the initial voiceprint token sequence for the corresponding scale. It can be expressed using the following formula: ;in, Let be the pooling step size, and ;k is the token index, and j is the feature dimension index, and ; i is the frame number index;

[0074] The feature standardization includes global standardization and local standardization; global standardization applies to the initial multi-scale voiceprint token sequence. Mean and variance normalization is performed to eliminate global feature distribution differences, resulting in a global standard token sequence. It can be expressed using the following formula: ;in, For the initial multi-scale voiceprint token sequence The mean, For the initial multi-scale voiceprint token sequence The standard deviation of is calculated using the following formula: ;in, For token sequence The number of tokens, and ;

[0075] Local standardization for each global standard token sequence The internal 80-dimensional features are locally normalized to obtain a multi-scale voiceprint token sequence. It can be expressed using the following formula: ;in, For each global standard token sequence The feature mean of the k-th token. For each global standard token sequence The feature standard deviation of the k-th token is calculated using the following formula: .

[0076] The multi-scale adaptive expert fusion module includes a shared routing unit, dedicated expert units, and a global expert unit; the number of dedicated experts in the dedicated expert unit... The number of experts in the global expert unit is 23. =1;

[0077] The shared routing unit supports voiceprint token sequences at all scales. Shared weights are used to calculate the voiceprint token sequence for each scale. Matching score between tokens and experts And input it into the dedicated expert unit to match the score. Calculate using the following formula: ;in, Let m be the routing weight of the m-th dedicated expert; The bias for the m-th dedicated expert; This represents the softmax operation on the expert dimension; z is the voiceprint token sequence. The z-th token;

[0078] The dedicated expert unit and the global expert unit adopt a bottleneck structure, specifically linear downsampling-GELU activation-linear upsampling, to achieve feature extraction and dimensionality preservation, expressed by the following formula: ;in, The downsampling weights; This is the downsampling bias; For upsampling weights; For upsampling bias; The activation function for the Gaussian error linear unit;

[0079] The dedicated expert unit performs Top-k sparse activation based on matching scores, selects the k highest-scoring dedicated experts, and calculates corresponding gating values ​​to regulate the output of the dedicated expert unit; the gating values ​​for the dedicated experts... Express it using the following formula: ;

[0080] The output of the multi-scale adaptive expert fusion module The combination of global expert unit output and dedicated expert unit output is represented by the following formula: ;

[0081] The final output of the Transformer layer in the multi-scale adaptive expert fusion feature modeling model is represented by the following formula: ;in, This represents the hidden state of the voiceprint token sequence at scale r for the l-th Transformer layer. The hidden state of the (l-1)th Transformer layer for the speaker token sequence at scale r; FFN is the feedforward network processing; MHSA is the multi-head self-attention processing; This is a layer normalization operation.

[0082] S3. Combining multi-scale joint loss and expert load balancing loss, a training objective function is constructed to train the initial two-layer voiceprint feature fault prediction model, thus obtaining the two-layer voiceprint feature fault prediction model.

[0083] In step S3, the total loss function for model training Express it using the following formula: ;in, For multi-scale joint loss; For expert load balancing losses;

[0084] The model training used the AdamW optimizer with a weight decay coefficient of 0.1 and a cosine annealing learning rate scheduler. The initial learning rate was 1e-3, which linearly decreased to 1e-5 after 10 epochs of training.

[0085] The multi-scale loss For each scale of fault classification loss, a weighted average is calculated based on the scale's importance using the following formula: ; Cross-entropy loss for the model's fault prediction of voiceprint token sequences at scale r; Y represents the fault prediction probability of the model for the voiceprint token sequence at scale r; Y is the voiceprint sample label. Scale weights; cross-entropy loss of the model for fault prediction of voiceprint token sequences at scale r. The following formula is used to measure the difference between the predicted probability and the true label: The model's probability of predicting faults in a voiceprint token sequence at scale r. Calculate using the following formula: ;in, This is the classification weight matrix; For classification bias; This refers to the hidden state of the final transformer layer.

[0086] The expert load balancing loss Express it using the following formula: ;in, Assign a frequency to the token of the m-th dedicated expert; The percentage of the total matching score for the m-th exclusive expert; the token allocation frequency for the m-th exclusive expert. Calculate using the following formula: Where B is the training batch size; The average number of tokens across all scales; The indicator function; the percentage of the total matching score for the m-th specialist. Calculate using the following formula: .

[0087] S4. Based on the dual-layer acoustic signature fault prediction model, dynamic reasoning is performed according to the real-time computing resources of the equipment to complete the transformer fault determination.

[0088] Step S4 includes the following steps:

[0089] Define resource constraint indicators and select the corresponding voiceprint scale based on the resource constraint indicators;

[0090] A two-layer voiceprint feature fault prediction model is used to infer the fault prediction probability of multi-scale voiceprint token sequences.

[0091] Fault determination is performed by combining the fault prediction probability at the selected scale with the fault prediction probability at adjacent scales.

[0092] Step S4 is as follows:

[0093] Define the resource constraint index C and calculate it using the following formula: Where TOPS represents the remaining computing power to be used; GB represents the remaining memory to be used.

[0094] The corresponding voiceprint scale is dynamically selected based on resource constraint index C, specifically as follows:

[0095] like Then select a voiceprint scale r of 4; if If so, then choose a voiceprint scale r of 16; Then, the voiceprint scale r is selected as 32;

[0096] A two-layer voiceprint feature-based fault prediction model is used to infer fault prediction probabilities from multi-scale voiceprint token sequences. ;

[0097] Fault determination is performed based on the fault prediction probability at a selected scale combined with the fault prediction probabilities at adjacent scales, specifically as follows:

[0098] like If so, the transformer is directly determined to be faulty;

[0099] like If the fault prediction probability of adjacent scales is called, and the fault is determined to be faulty if the fault probability of at least two scales is greater than 0.7, otherwise it is determined to be normal.

[0100] like If it is, then it is directly judged as normal.

[0101] The transformer voiceprint recognition method for complex environments described in this application also includes: reducing inference costs through expert weight quantization and dynamic truncation of silent segments;

[0102] The expert weight quantization is performed by using INT8 quantization to quantize the weights in the shared routing unit, dedicated expert unit, and global expert unit;

[0103] The dynamic truncation of the silent segment is specifically based on the voiceprint signal energy threshold. The energy in the truncated voiceprint signal is less than The silent segment; the voiceprint signal energy threshold Calculate using the following formula: ;in, The average energy of a voiceprint signal labeled as normal. The standard deviation of the energy of a voiceprint signal labeled as normal.

[0104] The method of the present invention will be further described below with reference to an embodiment:

[0105] Acoustic data was collected from 10 oil-immersed transformers (capacity 120MVA-240MVA) in 5 substations ranging from 220kV to 500kV, including:

[0106] Normal sample: 500 segments (10s per segment, sampling rate 16kHz), covering load 30%-100%, ambient temperature -10℃-40℃;

[0107] Fault samples: 3 types of typical faults (loose core bolts, winding deformation, DC bias), 30 segments in each type (simulating small sample scenarios in actual operation and maintenance);

[0108] Interference data includes cooling fan noise (50 / 100Hz), footsteps (instantaneous pulse, amplitude 10-20dB), and wind and rain noise (wideband, SNR 5dB-20dB), simulating the complex interference environment of a substation.

[0109] The training environment for the model uses an NVIDIA RTX 3090 GPU (24GB RAM), Python 3.9, and PyTorch 2.0; the inference environment for the model uses an NVIDIA Jetson AGX Orin (8GB RAM, 200 TOPS computing power) and a Raspberry Pi 4B (4GB RAM, 1 TOPS computing power) to simulate the resource constraints of edge devices.

[0110] The evaluation metrics are accuracy, precision, recall, and F1 score, calculated as follows:

[0111] Recognition rate: ;

[0112] Accuracy: ;

[0113] Recall rate: ;

[0114] F1 score: .

[0115] Existing mainstream transformer voiceprint recognition methods were selected as comparative models to verify the superiority of the method of the present invention in multi-condition, high compression ratio, and resource-constrained scenarios. Existing comparative models include:

[0116] Comparison Model 1: Voiceprint recognition method based on deep residual network (ResNet-18) (traditional fixed-scale feature modeling).

[0117] Comparison Model 2: Voiceprint recognition method based on Enhanced Channel Attention Time Delay Neural Network (EC-TDNN) (existing mainstream voiceprint recognition model).

[0118] Comparison Model 3: A few-shot voiceprint recognition method based on meta-learning (Prototypical Network) (a few-shot scenario optimization method);

[0119] The final experimental results are shown in Table 1.

[0120] Table 1 Comparative test results

[0121] As shown in Table 1, the evaluation indicators of the method of the present invention are significantly better than those of the comparative model. The recognition rate is 12.6 percentage points higher than that of the best comparative model (comparative model 3), proving its superiority in multi-condition scenarios. The memory usage of the method of the present invention after quantization is only 0.3GB, which is 60%-86% lower than that of the comparative model. The inference time is 0.6s / s, which is 2-4 times higher than that of the comparative model, adapting to the resource constraints of edge devices. In the high compression ratio (32:1) scenario, the recognition rate of the method of the present invention is still 93.7%, with a performance degradation of only 2.5%, while the recognition rates of comparative models 1-3 are all below 70%, proving its performance stability under high compression ratio.

[0122] To verify the contribution of the core modules of this invention, an ablation experiment was designed, in which each core module was removed in turn, and the changes in model performance were observed.

[0123] The following versions were used as a comparison model in the ablation experiments:

[0124] Ablation version 1: Removed the "Noise Suppression" module (only basic mean filtering and denoising are retained);

[0125] Ablation version 2: Removed the "Multi-scale Feature Extraction" module (using a fixed scale r=16);

[0126] Ablation Version 3: Removed "Exclusive Expert Unit" (only Global Expert Unit is retained);

[0127] Ablation version 4: Removes "multi-scale joint loss" during training (uses single-scale r=16 loss);

[0128] The results of the ablation experiment are shown in Table 2.

[0129] Table 2 Comparison Results of Ablation Experiments

[0130] As shown in Table 2, the noise suppression module improved the recognition rate by 13.7 percentage points by accurately removing multiple types of interference; the multi-scale feature extraction module improved the recognition rate by 10.4 percentage points by adapting to the scale differences of multiple working conditions; the dedicated expert unit improved the recognition rate by 7.9 percentage points by professional feature extraction; and the multi-scale joint loss during training improved the recognition rate by 5.6 percentage points by fusing multi-scale information.

[0131] The collaborative efforts of each core module ultimately achieve highly accurate transformer voiceprint recognition, verifying the rationality and innovation of the technical solution of this invention.

Claims

1. A transformer voiceprint recognition method for complex environments, characterized in that, Includes the following steps: S1. Collect transformer acoustic signature data; S2. Combining the principles of multi-scale learning with adaptive expert fusion, an initial two-layer voiceprint feature fault prediction model is constructed; S3. Combining multi-scale joint loss and expert load balancing loss, a training objective function is constructed to train the initial two-layer voiceprint feature fault prediction model, thus obtaining the two-layer voiceprint feature fault prediction model. S4. Based on the dual-layer acoustic signature fault prediction model, dynamic reasoning is performed according to the real-time computing resources of the equipment to complete the transformer fault determination.

2. The transformer voiceprint recognition method for complex environments according to claim 1, characterized in that, In step S2, the initial dual-layer voiceprint feature fault prediction model includes a multi-scale voiceprint token generation model and a multi-scale adaptive expert fusion feature modeling model; the input of the multi-scale voiceprint token generation model is the original voiceprint data, and the generated multi-scale voiceprint token is input into the multi-scale adaptive expert fusion feature modeling model. The multi-scale adaptive expert fusion feature modeling model adaptively extracts and cross-scale knowledge associations from the input multi-scale voiceprint tokens, and finally outputs the fault probability as the output of the initial two-layer voiceprint feature fault prediction model. The multi-scale voiceprint token generation model includes a noise suppression module and a multi-scale feature extraction module. The noise suppression module first uses adaptive wavelet thresholding for noise reduction, and then designs a multi-order infinite impulse response notch filter based on adaptive frequency band notch filtering to suppress noise, obtaining denoised voiceprint data and inputting it into the multi-scale feature extraction module; the multi-scale feature extraction module is based on the multi-scale learning principle, sets the scale level according to the resource constraints of edge devices, extracts the initial multi-scale token sequence, and then performs feature standardization on the initial multi-scale token sequence to obtain the multi-scale voiceprint token sequence; The multi-scale adaptive expert fusion feature model is based on a frozen pre-trained language model. The multi-scale adaptive expert fusion module is inserted in parallel into the Transformer layer of the frozen LLM, working in parallel with the original multi-head self-attention and feedforward networks. The final output of the multi-scale adaptive expert fusion module is fused with the output of the original multi-head self-attention and feedforward networks of the LLM. Layer normalization is used to ensure training stability, and the result is used as the final output of the transformer layer. Finally, the transformer layer is processed to obtain the fault prediction probability as the output of the initial two-layer voiceprint feature fault prediction model.

3. The transformer voiceprint recognition method for complex environments according to claim 2, characterized in that, In the noise suppression module, the adaptive wavelet threshold denoising specifically involves: using the db6 wavelet basis to perform a 6-level wavelet decomposition on the original transformer acoustic fingerprint data to obtain wavelet coefficients at each scale. , where j is the scale index and k is the time index; Soft thresholding is applied to the wavelet coefficients to preserve the transformer acoustic signature characteristics and suppress noise interference. The processed wavelet coefficients are shown below. Express it using the following formula: ;in, It is a symbolic function; The adaptive threshold for the j-th scale; Adaptive threshold at scale j Calculate using the following formula: ;in, is the standard deviation of the noise at the j-th scale, which is estimated using the wavelet coefficients of the first scale; N is the length of the acoustic signature signal. Processed wavelet coefficients Wavelet reconstruction is performed to obtain a preliminary denoised speaker signal; The specific parameter configuration of the multi-order infinite impulse response notch filter is as follows: Center frequency Where m is the interference coefficient, which takes the value of 1, 2 or 3, and is for narrowband interference of 50Hz integer multiples generated by cooling fans and cooling pumps; The quality factor Q is 12; the notch bandwidth is guaranteed to be no greater than 4Hz to avoid suppressing effective acoustic signature characteristics. The stopband attenuation is greater than or equal to 60dB.

4. The transformer voiceprint recognition method for complex environments according to claim 2, characterized in that, The multi-scale feature extraction module sets five scale levels based on edge device resource constraints, with corresponding compression ratios of 4, 8, 16, 24, and 32, respectively; where the compression ratio is the ratio of the number of voiceprint frames before compression to the number of tokens after compression. The initial multi-scale token extraction includes the following steps: The voiceprint signal in the denoised voiceprint data is processed by frame segmentation, and 80-dimensional Mel filter bank features are extracted. , where T is the total number of frames; the frame length of the frame segmentation process is 25ms, and the frame shift is 10ms; For each scale level r, an adaptive step-size pooling strategy is used to generate the initial voiceprint token sequence for the corresponding scale. It can be expressed using the following formula: ;in, Let be the pooling step size, and ;k is the token index, and j is the feature dimension index, and ; i is the frame number index; The feature standardization includes global standardization and local standardization; global standardization applies to the initial multi-scale voiceprint token sequence. Mean and variance normalization is performed to eliminate global feature distribution differences, resulting in a global standard token sequence. It can be expressed using the following formula: ;in, For the initial multi-scale voiceprint token sequence The mean, For the initial multi-scale voiceprint token sequence The standard deviation of is calculated using the following formula: ;in, For token sequence The number of tokens, and ; Local standardization for each global standard token sequence The internal 80-dimensional features are locally normalized to obtain a multi-scale voiceprint token sequence. It can be expressed using the following formula: ;in, For each global standard token sequence The feature mean of the k-th token. For each global standard token sequence The feature standard deviation of the k-th token is calculated using the following formula: .

5. The transformer voiceprint recognition method for complex environments according to claim 2, characterized in that, The multi-scale adaptive expert fusion module includes a shared routing unit, dedicated expert units, and a global expert unit; the number of dedicated experts in the dedicated expert unit... The number of experts in the global expert unit is 23. =1; The shared routing unit supports voiceprint token sequences at all scales. Shared weights are used to calculate the voiceprint token sequence for each scale. Matching score between tokens and experts And input it into the dedicated expert unit to match the score. Calculate using the following formula: ;in, Let m be the routing weight of the m-th dedicated expert; The bias for the m-th dedicated expert; This represents the softmax operation on the expert dimension; z is the voiceprint token sequence. The z-th token; The dedicated expert unit and the global expert unit adopt a bottleneck structure, specifically linear downsampling-GELU activation-linear upsampling, to achieve feature extraction and dimensionality preservation, expressed by the following formula: ;in, The downsampling weights; This is the downsampling bias; For upsampling weights; For upsampling bias; The activation function for the Gaussian error linear unit; The dedicated expert unit performs Top-k sparse activation based on matching scores, selects the k highest-scoring dedicated experts, and calculates corresponding gating values ​​to regulate the output of the dedicated expert unit; the gating values ​​for the dedicated experts... Express it using the following formula: ; The output of the multi-scale adaptive expert fusion module The combination of global expert unit output and dedicated expert unit output is represented by the following formula: ; The final output of the Transformer layer in the multi-scale adaptive expert fusion feature modeling model is represented by the following formula: ;in, This represents the hidden state of the voiceprint token sequence at scale r for the l-th Transformer layer. The hidden state of the (l-1)th Transformer layer for the speaker token sequence at scale r; FFN is the feedforward network processing; MHSA is the multi-head self-attention processing; This is a layer normalization operation.

6. The transformer voiceprint recognition method for complex environments according to claim 1, characterized in that, In step S3, the total loss function for model training Express it using the following formula: ;in, For multi-scale joint loss; For expert load balancing losses; The model training used the AdamW optimizer with a weight decay coefficient of 0.1 and a cosine annealing learning rate scheduler. The initial learning rate was 1e-3, which linearly decreased to 1e-5 after 10 epochs of training.

7. The transformer voiceprint recognition method for complex environments according to claim 6, characterized in that, The multi-scale loss For each scale of fault classification loss, a weighted average is calculated based on the scale's importance using the following formula: ; Cross-entropy loss for the model's fault prediction of voiceprint token sequences at scale r; Y represents the fault prediction probability of the model for the voiceprint token sequence at scale r; Y is the voiceprint sample label. Scale weights; cross-entropy loss of the model for fault prediction of voiceprint token sequences at scale r. The following formula is used to measure the difference between the predicted probability and the true label: The model's probability of predicting faults in a voiceprint token sequence at scale r. Calculate using the following formula: ;in, This is the classification weight matrix; For classification bias; This refers to the hidden state of the final transformer layer. The expert load balancing loss Express it using the following formula: ;in, Assign a frequency to the token of the m-th dedicated expert; The percentage of the total matching score for the m-th exclusive expert; the token allocation frequency for the m-th exclusive expert. Calculate using the following formula: Where B is the training batch size; The average number of tokens across all scales; The indicator function; the percentage of the total matching score for the m-th specialist. Calculate using the following formula: .

8. The transformer voiceprint recognition method for complex environments according to claim 1, characterized in that, Step S4 includes the following steps: Define resource constraint indicators and select the corresponding voiceprint scale based on the resource constraint indicators; A two-layer voiceprint feature fault prediction model is used to infer the fault prediction probability of multi-scale voiceprint token sequences. Fault determination is performed by combining the fault prediction probability at the selected scale with the fault prediction probability at adjacent scales.

9. The transformer voiceprint recognition method for complex environments according to claim 8, characterized in that, Step S4 is as follows: Define the resource constraint index C and calculate it using the following formula: Where TOPS represents the remaining computing power to be used; GB represents the remaining memory to be used. The corresponding voiceprint scale is dynamically selected based on resource constraint index C, specifically as follows: like Then select a voiceprint scale r of 4; if If so, then choose a voiceprint scale r of 16; Then, the voiceprint scale r is selected as 32; A two-layer voiceprint feature-based fault prediction model is used to infer fault prediction probabilities from multi-scale voiceprint token sequences. ; Fault determination is performed based on the fault prediction probability at a selected scale combined with the fault prediction probabilities at adjacent scales, specifically as follows: like If so, the transformer is directly determined to be faulty; like If the fault prediction probability of adjacent scales is called, and the fault is determined to be faulty if the fault probability of at least two scales is greater than 0.7, otherwise it is determined to be normal. like If so, it is directly judged as normal.

10. The transformer voiceprint recognition method for complex environments according to claim 1, characterized in that, The transformer voiceprint recognition method for complex environments also includes: Reduce inference costs by using expert weight quantification and dynamic truncation of silent segments; The expert weight quantization is performed by using INT8 quantization to quantize the weights in the shared routing unit, dedicated expert unit, and global expert unit; The dynamic truncation of the silent segment is specifically based on the voiceprint signal energy threshold. The energy in the truncated voiceprint signal is less than The silent segment; the voiceprint signal energy threshold Calculate using the following formula: ;in, The average energy of a voiceprint signal labeled as normal. The standard deviation of the energy of a voiceprint signal labeled as normal.

Images