Transformer voiceprint anomaly detection method based on generative adversarial network

Abstract

The invention relates to a transformer voiceprint anomaly detection method based on a generative adversarial network, and the method comprises the steps: obtaining voiceprint data, and carrying out the preprocessing of the voiceprint data, and obtaining the preprocessed spectrum data; constructing an abnormal voiceprint detection model; inputting the preprocessed spectrum data into an abnormal voiceprint detection model for training; and obtaining voiceprint data to be detected and preprocessing the voiceprint data to be detected, inputting the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and outputting a detection result. According to the method, unbalanced distribution of normal voiceprint data and abnormal voiceprint data can be balanced, the generalization ability of the model is enhanced, and the model can keep high detection performance when processing a small amount of abnormal data; according to the method, the distinguishing capability of the abnormal voiceprint is enhanced, the abnormal voiceprint can be effectively recognized in a complex noise environment, the influence of noise and interference factors is reduced, and the detection accuracy is improved; fine-grained features in the voiceprint signals can be captured, extraction of the features is further optimized, and therefore abnormal voiceprints can be recognized more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a transformer voiceprint anomaly detection method based on a generative adversarial network. Background Art

[0002] With the rapid development of power systems, the safe operation of transformers has received increasing attention. Voiceprint detection, as a non-invasive monitoring method, can effectively identify the operating status of transformers. However, traditional voiceprint anomaly detection methods often have the following shortcomings: First, limited feature extraction capabilities: When processing voiceprint signals, traditional methods find it difficult to effectively capture the time-frequency characteristics and global dependencies of signals, resulting in poor detection results; Second, data distribution is uneven: In actual applications, normal voiceprint data is abundant, while abnormal voiceprint data is less, resulting in insufficient model generalization capabilities; Third, the ability to distinguish abnormal voiceprints is weak: When identifying abnormal voiceprints, traditional methods are easily affected by noise and interference factors, resulting in low detection accuracy; Fourth, insufficient fine-grained feature extraction: Traditional methods find it difficult to extract fine-grained features in voiceprint signals, affecting the recognition effect of abnormal voiceprints.

[0003] In response to the above problems, researchers have begun to focus on the application of generative adversarial networks (GAN) in the field of voiceprint anomaly detection. The GAN network has powerful generation and feature learning capabilities, but how to combine it with the characteristics of voiceprint signals to improve the accuracy and robustness of anomaly detection is still an urgent problem to be solved. On this basis, the Transformer encoder structure has gradually become a research hotspot due to its excellent time-frequency feature extraction capabilities. In addition, contrastive learning, as an effective feature optimization method, also shows potential in the field of voiceprint anomaly detection. Therefore, combining GAN, Transformer and contrastive learning techniques to study an efficient and robust voiceprint anomaly detection algorithm has important practical application value. Summary of the invention

[0004] In order to solve the problems of limited feature extraction capability, weak abnormal voiceprint differentiation capability and insufficient fine-grained feature extraction in existing voiceprint anomaly detection methods, the purpose of the present invention is to provide a transformer voiceprint anomaly detection method based on a generative adversarial network, which enhances the generalization ability of the model, enhances the differentiation capability of abnormal voiceprints, and optimizes fine-grained feature extraction.

[0005] To achieve the above object, the present invention adopts the following technical solution: a transformer voiceprint anomaly detection method based on a generative adversarial network, the method comprising the following steps in order:

[0006] (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data;

[0007] (2) constructing an abnormal voiceprint detection model, which uses a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders.

[0008] (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model;

[0009] (4) Acquire the voiceprint data to be detected and preprocess it, input the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and output the detection result.

[0010] In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals when the transformer is running from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

[0011] In step (1), the preprocessing specifically includes the following steps in order:

[0012] (1a) performing data enhancement to expand the data set, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition;

[0013] (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, perform framing on the original time domain signal, use the Hanning window, set the frame length N and frame shift M;

[0014] (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal, convert the time domain signal into frequency domain representation, and calculate the spectrum diagram:

[0015] S(f,t)=STFT(x(t))

[0016] Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents a continuous audio signal;

[0017] (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency;

[0018] (1e) Logarithmic compression: Take the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range;

[0019] (1f) Data standardization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

[0020] The step (3) specifically comprises the following steps in order:

[0021] (3a) Classify the preprocessed spectral data, using the idea of ​​contrastive learning to take samples of the same class as positive samples and samples of different classes as negative samples;

[0022] (3b) The negative sample is input into the generator of the generative adversarial network, and the generator generates a fake sample;

[0023] (3c) Generate adversarial training: send the forged samples and positive samples together to the discriminator, and train by distinguishing positive samples from forged samples;

[0024] (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

[0025] The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss;

[0026] The contrastive learning loss The calculation formula is:

[0027]

[0028] In the formula, χ is the sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under z represents the noise distribution of negative samples input to the generator;

[0029] sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity:

[0030]

[0031] The generator loss as follows:

[0032]

[0033] Where G(z) is the forged sample generated by the generator;

[0034] Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for:

[0035]

[0036] In the formula, λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in

[0037] The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss

[0038] The adversarial loss for:

[0039]

[0040] In the formula, p data Indicates that in the real data distribution, Represents the true data distribution p data Expected value under

[0041] The classification loss for:

[0042]

[0043] Where y is the multi-classification label of x; D cls (x|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

[0044] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention solves the problem of uneven data distribution: by adopting the data enhancement strategy, the present invention can balance the uneven distribution of normal and abnormal voiceprint data, enhance the generalization ability of the model, and enable the model to maintain a high detection performance when processing a small amount of abnormal data; second, the present invention enhances the ability to distinguish abnormal voiceprints: combining the advantages of GAN and Transformer, the present invention can effectively identify abnormal voiceprints in complex noise environments, reduce the influence of noise and interference factors, and improve the detection accuracy; third, the present invention optimizes the extraction of fine-grained features: the Transformer architecture can capture the fine-grained features in the voiceprint signal. At the same time, the present invention further optimizes the extraction of these features through GAN and contrastive learning, so as to more accurately identify abnormal voiceprints; fourth, the loss function designed by the present invention improves the model performance: the present invention effectively integrates GAN, Transformer and contrastive learning techniques through a carefully designed loss function, and synergistically improves the generation ability, sequence modeling effect and representation learning ability of the model, making the overall detection algorithm more efficient and robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The figure is a flow chart of the training method of the abnormal voiceprint detection model in the present invention. DETAILED DESCRIPTION

[0046] like Figure 1 As shown, a transformer voiceprint anomaly detection method based on a generative adversarial network comprises the following steps in sequence:

[0047] (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data;

[0048] (2) Construct an abnormal voiceprint detection model. The abnormal voiceprint detection model adopts a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders. Both the generator and the discriminator adopt the Transformer structure. The generator generates pseudo voiceprint features by sampling from the noise distribution to expand the data distribution. The discriminator is not only used to distinguish between generated features and real features, but also to realize voiceprint category recognition through multi-classification tasks, thereby improving the detection ability of abnormal voiceprints. The generator generates pseudo voiceprint features by sampling from the noise distribution, and optimizes them through adversarial training to make the generated features closer to the real voiceprint distribution and improve the generalization ability of the model. The discriminator realizes the recognition function of voiceprint categories by introducing multi-classification tasks, thereby improving the ability to distinguish abnormal voiceprints.

[0049] (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model;

[0050] (4) Acquire the voiceprint data to be detected and preprocess it, input the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and output the detection result.

[0051] In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals when the transformer is running from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

[0052] In step (1), the preprocessing specifically includes the following steps in order:

[0053] (1a) performing data enhancement to expand the data set, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition;

[0054] (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, perform framing on the original time domain signal, use the Hanning window, set the frame length N and frame shift M;

[0055] (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal, convert the time domain signal into frequency domain representation, and calculate the spectrum diagram:

[0056] S(f,t)=STFT(x(t))

[0057] Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents a continuous audio signal;

[0058] (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency;

[0059] (1e) Logarithmic compression: Take the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range;

[0060] (1f) Data standardization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

[0061] The step (3) specifically comprises the following steps in order:

[0062] (3a) Classify the spectral data obtained by preprocessing, use the idea of ​​contrastive learning, take samples of the same class as positive samples, and samples of different classes as negative samples; introduce contrastive learning mechanism, optimize the consistency of feature space distribution by constructing enhanced positive sample pairs, and improve the ability to extract fine-grained features. The contrastive learning mechanism constructs positive sample pairs by enhancing the time-frequency features of voiceprint signals, thereby improving the robustness of the generative adversarial network for abnormal voiceprint detection.

[0063] (3b) The negative sample is input into the generator of the generative adversarial network, and the generator generates a fake sample;

[0064] (3c) Generate adversarial training: send the forged samples and positive samples together to the discriminator, and train by distinguishing positive samples from forged samples;

[0065] (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

[0066] The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss;

[0067] The contrastive learning loss The calculation formula is:

[0068]

[0069] In the formula, χ is the sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under z represents the noise distribution of negative samples input to the generator;

[0070] sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity:

[0071]

[0072] The generator loss as follows:

[0073]

[0074] Where G(z) is the forged sample generated by the generator;

[0075] Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for:

[0076]

[0077] In the formula, λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in

[0078] The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss

[0079] The adversarial loss for:

[0080]

[0081] In the formula, p data Indicates that in the real data distribution, Represents the true data distribution p data Expected value under

[0082] The classification loss for:

[0083]

[0084] Where y is the multi-classification label of x; D cls (y|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

[0085] The Transformer structure of the present invention captures the time-frequency characteristics and global dependencies of voiceprint signals through a multi-head attention mechanism, thereby enhancing the feature extraction capability of the model. Combining the generation capability of the GAN network, the strong feature extraction capability of the Transformer, and the feature optimization advantages of contrastive learning, efficient detection of voiceprint anomalies is achieved.

[0086] In summary, the present invention solves the problem of uneven data distribution: by adopting the data enhancement strategy, the present invention can balance the uneven distribution of normal and abnormal voiceprint data, enhance the generalization ability of the model, and enable the model to maintain a high detection performance when processing a small amount of abnormal data; the present invention enhances the ability to distinguish abnormal voiceprints: combining the advantages of GAN and Transformer, the present invention can effectively identify abnormal voiceprints in complex noise environments, reduce the influence of noise and interference factors, and improve the detection accuracy; the present invention optimizes fine-grained feature extraction: the Transformer architecture can capture fine-grained features in voiceprint signals. At the same time, the present invention further optimizes the extraction of these features through GAN and contrastive learning, thereby more accurately identifying abnormal voiceprints; the loss function designed by the present invention improves the model performance: the present invention effectively integrates GAN, Transformer and contrastive learning techniques through a carefully designed loss function, and synergistically improves the model's generation ability, sequence modeling effect and representation learning ability, making the overall detection algorithm more efficient and robust.

Claims

1. A transformer voiceprint anomaly detection method based on a generative adversarial network, characterized in that: The method comprises the following steps in order: (1) Acquire voiceprint data and preprocess it to obtain preprocessed spectrum data; (2) constructing an abnormal voiceprint detection model, which uses a generative adversarial network. The generative adversarial network consists of a generator and a discriminator. The generator consists of multiple transformer modules, and the discriminator consists of multiple transformer encoders. (3) inputting the preprocessed spectrum data into the abnormal voiceprint detection model for training to obtain a trained abnormal voiceprint detection model; (4) Acquire the voiceprint data to be detected and preprocess it, input the preprocessed voiceprint data to be detected into the trained abnormal voiceprint detection model, and output the detection result.

2. The transformer voiceprint anomaly detection method based on generative adversarial network according to claim 1 is characterized by: In step (1), a multi-channel voiceprint acquisition device is used to collect sound signals when the transformer is running from the environment. The multi-channel voiceprint acquisition device includes multiple microphone arrays, which can simultaneously capture sound wave signals in multiple directions and different frequency ranges, thereby improving the quality and accuracy of the voiceprint data.

3. The transformer voiceprint anomaly detection method based on generative adversarial network according to claim 1 is characterized by: In step (1), the preprocessing specifically includes the following steps in order: (1a) performing data enhancement to expand the data set, wherein the data enhancement includes time-frequency masking, random time cropping, and noise superposition; (1b) Time domain signal sampling and framing: Read the original time domain signal from the voiceprint recording, set the sampling rate, perform framing on the original time domain signal, use the Hanning window, set the frame length N and frame shift M; (1c) Perform STFT short-time Fourier transform: Perform STFT on the framed signal, convert the time domain signal into frequency domain representation, and calculate the spectrum diagram: S(f,t)=STFT(x(t)) Where S(f,t) is the energy distribution of the signal at frequency f and time t; x(t) represents a continuous audio signal; (1d) Convert to Mel spectrum: Use Mel filter bank to divide the spectrum into frequency bands to simulate the human auditory system's perception of frequency; (1e) Logarithmic compression: Take the logarithm of the amplitude of the Mel spectrum to enhance the contrast of the dynamic range; (1f) Data standardization: The Mel spectra of all samples are mean-variance normalized to stabilize the feature distribution.

4. The transformer voiceprint anomaly detection method based on generative adversarial network according to claim 1 is characterized by: The step (3) specifically comprises the following steps in order: (3a) Classify the preprocessed spectral data, using the idea of ​​contrastive learning to take samples of the same class as positive samples and samples of different classes as negative samples; (3b) The negative sample is input into the generator of the generative adversarial network, and the generator generates a fake sample; (3c) Generate adversarial training: send the forged samples and positive samples together to the discriminator, and train by distinguishing positive samples from forged samples; (3d) Perform multi-classification labeling on the voiceprint data obtained in step (1), and input the labeled voiceprint data into the discriminator for multi-classification training.

5. The transformer voiceprint anomaly detection method based on generative adversarial network according to claim 1 is characterized by: The loss function of the abnormal voiceprint detection model includes contrastive learning loss, generator loss and discriminator loss; The contrastive learning loss The calculation formula is: In the formula, x is the sample pool for contrastive learning, including positive samples and negative samples; f(x) represents the feature representation of sample x, x + is a positive sample; τ is a temperature hyperparameter used to control the smoothness of the distribution; z gen is the latent representation of the fake samples generated by the generator; Indicates that in p z The expected value under z represents the noise distribution of negative samples input to the generator; sim(a,b) represents the similarity measure between sample a and sample b, using cosine similarity: The generator loss as follows: Where G(z) is the forged sample generated by the generator; Contrastive learning loss and the generator loss Combined, we get the total loss function of the generator for: In the formula, λ contrast is a hyperparameter used to adjust the contrastive learning loss The total loss in the generator The weight in The discriminator is trained twice, using adversarial loss and classification loss To optimize the network parameters, that is, the same neural network is trained twice; the discriminator loss includes the adversarial loss and classification loss The adversarial loss for: In the formula, p data Indicates that in the real data distribution, Represents the true data distribution p data Expected value under The classification loss for: Where y is the multi-classification label of x; D cls (y|x) represents the predicted probability of the discriminator for the multi-class label y given a sample x; D is the discriminator and G is the generator.

Images