A transformer voiceprint anomaly detection method based on generative adversarial network

Abstract

The present invention relates to a transformer voiceprint anomaly detection method based on a generative adversarial network, comprising the following steps: obtaining voiceprint data and preprocessing it to obtain preprocessed spectral data; constructing an abnormal voiceprint detection model; inputting the preprocessed spectral data into the abnormal voiceprint detection model for training; obtaining voiceprint data to be detected and preprocessing it, inputting the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and outputting a detection result. The present invention can balance the uneven distribution of normal and abnormal voiceprint data, enhance the generalization ability of the model, and enable the model to maintain high detection performance even when processing small amounts of abnormal data. The present invention also enhances the ability to distinguish abnormal voiceprints: it can effectively identify abnormal voiceprints in complex noisy environments, reduces the influence of noise and interference factors, and improves detection accuracy. It can also capture fine-grained features in voiceprint signals and further optimize the extraction of these features, thereby more accurately identifying abnormal voiceprints.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a transformer voiceprint anomaly detection method based on a generative adversarial network. Background Art

[0002] With the rapid development of power systems, the safe operation of transformers has received increasing attention. Voiceprint detection, as a non-invasive monitoring method, can effectively identify the operating status of transformers. However, traditional voiceprint anomaly detection methods often have the following shortcomings: First, limited feature extraction capabilities: When processing voiceprint signals, traditional methods have difficulty effectively capturing the time-frequency characteristics and global dependencies of signals, resulting in poor detection results; Second, data distribution is uneven: In actual applications, normal voiceprint data is abundant, while abnormal voiceprint data is scarce, resulting in insufficient model generalization capabilities; Third, the ability to distinguish abnormal voiceprints is weak: When identifying abnormal voiceprints, traditional methods are easily affected by noise and interference factors, resulting in low detection accuracy; Fourth, insufficient fine-grained feature extraction: Traditional methods have difficulty extracting fine-grained features from voiceprint signals, which affects the recognition effect of abnormal voiceprints.

[0003] To address these issues, researchers have begun focusing on the application of generative adversarial networks (GANs) in voiceprint anomaly detection. While GANs possess powerful generative and feature-learning capabilities, combining them with the characteristics of voiceprint signals to improve the accuracy and robustness of anomaly detection remains a pressing challenge. The Transformer encoder architecture, due to its superior time-frequency feature extraction capabilities, has gradually become a research hotspot. Furthermore, contrastive learning, as an effective feature optimization method, has also shown promise in voiceprint anomaly detection. Therefore, combining GANs, Transformers, and contrastive learning techniques to develop an efficient and robust voiceprint anomaly detection algorithm has significant practical application value. Summary of the Invention

[0004] In order to solve the problems of limited feature extraction capability, weak abnormal voiceprint differentiation capability, and insufficient fine-grained feature extraction in existing voiceprint anomaly detection methods, the purpose of the present invention is to provide a transformer voiceprint anomaly detection method based on a generative adversarial network, which enhances the generalization ability of the model, enhances the differentiation capability of abnormal voiceprints, and optimizes fine-grained feature extraction.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a transformer voiceprint anomaly detection method based on a generative adversarial network, the method comprising the following steps in sequence:

[0006] (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data;

[0007] (2) Constructing an abnormal voiceprint detection model, which uses a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders.

[0008] (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model;

[0009] (4) Obtain the voiceprint data to be detected and preprocess it, input the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and output the detection results.

[0010] In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals of the transformer during operation from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

[0011] In step (1), the pre-processing specifically includes the following steps in order:

[0012] (1a) performing data enhancement to expand the dataset, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition;

[0013] (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, and perform framing on the original time domain signal. Use the Hanning window, set the frame length N and frame shift M;

[0014] (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal to convert the time domain signal into frequency domain representation and calculate the spectrum:

[0015] S(f,t)=STFT(x(t))

[0016] Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents the continuous audio signal;

[0017] (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency;

[0018] (1e) Logarithmic compression: taking the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range;

[0019] (1f) Data normalization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

[0020] The step (3) specifically includes the following steps in order:

[0021] (3a) Classify the preprocessed spectral data using the idea of ​​contrastive learning, treating samples of the same class as positive samples and samples of different classes as negative samples;

[0022] (3b) Negative samples are input into the generator of the generative adversarial network, and the generator generates fake samples;

[0023] (3c) Generative adversarial training: Forged samples and positive samples are fed into the discriminator together, and training is performed by distinguishing between positive and forged samples;

[0024] (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

[0025] The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss;

[0026] The contrastive learning loss The calculation formula is:

[0027]

[0028] Where χ is the sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under p z represents the noise distribution of negative samples input to the generator;

[0029] sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity:

[0030]

[0031] The generator loss as follows:

[0032]

[0033] Where G(z) is the fake sample generated by the generator;

[0034] Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for:

[0035]

[0036] Where λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in

[0037] The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss

[0038] The adversarial loss for:

[0039]

[0040] Where p data Indicates the real data distribution, Represents the true data distribution p data Expected value under

[0041] The classification loss for:

[0042]

[0043] Where y is the multi-classification label of x; D cls (x|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

[0044] It can be seen from the above technical solution that the beneficial effects of the present invention are: First, the present invention solves the problem of uneven data distribution: by adopting the data enhancement strategy, the present invention can balance the uneven distribution of normal and abnormal voiceprint data, enhance the generalization ability of the model, and enable the model to maintain high detection performance when processing a small amount of abnormal data; Second, the present invention enhances the ability to distinguish abnormal voiceprints: combining the advantages of GAN and Transformer, the present invention can effectively identify abnormal voiceprints in complex noise environments, reduce the influence of noise and interference factors, and improve the detection accuracy; Third, the present invention optimizes fine-grained feature extraction: the Transformer architecture can capture fine-grained features in voiceprint signals. At the same time, the present invention further optimizes the extraction of these features through GAN and contrastive learning, thereby more accurately identifying abnormal voiceprints; Fourth, the loss function designed by the present invention improves the model performance: the present invention effectively integrates GAN, Transformer and contrastive learning technologies through a carefully designed loss function, and synergistically improves the model's generation ability, sequence modeling effect and representation learning ability, making the overall detection algorithm more efficient and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the training method for the abnormal voiceprint detection model in the present invention. DETAILED DESCRIPTION

[0046] like Figure 1 As shown, a transformer voiceprint anomaly detection method based on a generative adversarial network includes the following steps in sequence:

[0047] (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data;

[0048] (2) Construct an abnormal voiceprint detection model. The abnormal voiceprint detection model adopts a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders. Both the generator and the discriminator adopt the Transformer structure. The generator generates pseudo voiceprint features by sampling from the noise distribution to expand the data distribution. The discriminator is not only used to distinguish between generated features and real features, but also realizes voiceprint category recognition through multi-classification tasks, thereby improving the detection ability of abnormal voiceprints. The generator generates pseudo voiceprint features by sampling from the noise distribution and optimizes them through adversarial training to make the generated features closer to the real voiceprint distribution, thereby improving the generalization ability of the model. The discriminator realizes the recognition function of voiceprint categories by introducing multi-classification tasks, thereby improving the ability to distinguish abnormal voiceprints.

[0049] (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model;

[0050] (4) Obtain the voiceprint data to be detected and preprocess it, input the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and output the detection results.

[0051] In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals of the transformer during operation from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

[0052] In step (1), the pre-processing specifically includes the following steps in order:

[0053] (1a) performing data enhancement to expand the dataset, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition;

[0054] (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, and perform framing on the original time domain signal. Use the Hanning window, set the frame length N and frame shift M;

[0055] (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal to convert the time domain signal into frequency domain representation and calculate the spectrum:

[0056] S(f,t)=STFT(x(t))

[0057] Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents the continuous audio signal;

[0058] (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency;

[0059] (1e) Logarithmic compression: taking the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range;

[0060] (1f) Data normalization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

[0061] The step (3) specifically includes the following steps in order:

[0062] (3a) The preprocessed spectral data is classified. Using the idea of ​​contrastive learning, samples of the same class are used as positive samples, and samples of different classes are used as negative samples. A contrastive learning mechanism is introduced to construct enhanced positive sample pairs, optimize the consistency of feature space distribution, and improve the ability to extract fine-grained features. The contrastive learning mechanism constructs positive sample pairs by enhancing the time-frequency characteristics of the voiceprint signal, thereby improving the robustness of the generative adversarial network for abnormal voiceprint detection.

[0063] (3b) Negative samples are input into the generator of the generative adversarial network, and the generator generates fake samples;

[0064] (3c) Generative adversarial training: Forged samples and positive samples are fed into the discriminator together, and training is performed by distinguishing between positive and forged samples;

[0065] (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

[0066] The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss;

[0067] The contrastive learning loss The calculation formula is:

[0068]

[0069] Where χ is the sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under p z represents the noise distribution of negative samples input to the generator;

[0070] sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity:

[0071]

[0072] The generator loss as follows:

[0073]

[0074] Where G(z) is the fake sample generated by the generator;

[0075] Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for:

[0076]

[0077] Where λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in

[0078] The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss

[0079] The adversarial loss for:

[0080]

[0081] Where p data Indicates the real data distribution, Represents the true data distribution p data Expected value under

[0082] The classification loss for:

[0083]

[0084] Where y is the multi-classification label of x; D cls (y|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

[0085] The Transformer architecture of this invention captures the time-frequency characteristics and global dependencies of voiceprint signals through a multi-head attention mechanism, thereby enhancing the model's feature extraction capabilities. Combining the generative power of the GAN network, the strong feature extraction capabilities of the Transformer, and the feature optimization advantages of contrastive learning, it achieves efficient detection of voiceprint anomalies.

[0086] In summary, the present invention solves the problem of uneven data distribution: by adopting the data enhancement strategy, the present invention can balance the uneven distribution of normal and abnormal voiceprint data, enhance the generalization ability of the model, and enable the model to maintain a high detection performance when processing a small amount of abnormal data; the present invention enhances the ability to distinguish abnormal voiceprints: combining the advantages of GAN and Transformer, the present invention can effectively identify abnormal voiceprints in complex noise environments, reduce the influence of noise and interference factors, and improve the detection accuracy; the present invention optimizes fine-grained feature extraction: the Transformer architecture can capture fine-grained features in voiceprint signals. At the same time, the present invention further optimizes the extraction of these features through GAN and contrastive learning, thereby more accurately identifying abnormal voiceprints; the loss function designed by the present invention improves the model performance: the present invention effectively integrates GAN, Transformer and contrastive learning technologies through a carefully designed loss function, and synergistically improves the model's generation ability, sequence modeling effect and representation learning ability, making the overall detection algorithm more efficient and robust.

Claims

1. A transformer voiceprint anomaly detection method based on a generative adversarial network, characterized by: The method comprises the following steps in sequence: (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data; (2) Constructing an abnormal voiceprint detection model, which uses a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders. (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model; (4) obtaining the voiceprint data to be detected and preprocessing it, inputting the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and outputting the detection results; The step (3) specifically includes the following steps in order: (3a) Classify the preprocessed spectral data using the idea of ​​contrastive learning, treating samples of the same class as positive samples and samples of different classes as negative samples; (3b) Negative samples are input into the generator of the generative adversarial network, and the generator generates fake samples; (3c) Generative adversarial training: Forged samples and positive samples are fed into the discriminator together, and training is performed by distinguishing between positive and forged samples; (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

2. The transformer voiceprint anomaly detection method based on a generative adversarial network according to claim 1 is characterized by: In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals of the transformer during operation from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

3. The transformer voiceprint anomaly detection method based on a generative adversarial network according to claim 1 is characterized by: In step (1), the pre-processing specifically includes the following steps in order: (1a) performing data enhancement to expand the dataset, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition; (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, and perform framing on the original time domain signal. Use the Hanning window, set the frame length N and frame shift M; (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal to convert the time domain signal into frequency domain representation and calculate the spectrum: S(f,t)=STFT(x(t)) Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents the continuous audio signal; (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency; (1e) Logarithmic compression: taking the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range; (1f) Data normalization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

4. The transformer voiceprint anomaly detection method based on a generative adversarial network according to claim 1 is characterized by: The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss; The contrastive learning loss The calculation formula is: Where, is a sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under p z represents the noise distribution of negative samples input to the generator; sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity: The generator loss as follows: Where G(z) is the fake sample generated by the generator; Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for: Where λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss The adversarial loss for: Where p data Indicates the real data distribution, Represents the true data distribution p data Expected value under The classification loss for: Where y is the multi-classification label of x; D cls (y|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

Images