A method for constructing speaker recognition model in noisy environment
By introducing denoising modules and ECAPA-TDNN networks into the voiceprint recognition model, dynamically adjusting the parameters and window function lengths, the problem of low recognition accuracy in noise environments is solved, and higher speaker recognition accuracy and voice characteristics are achieved Extraction effect.
Patent Information
- Application Number
- CN202210734918.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-21
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing voiceprint recognition technology shows poor robustness in noise environments and cannot effectively meet the speaker's recognition needs in real scenarios. The recognition accuracy needs to be improved.
Added a denoising module to the speaker recognition model, remove noise through short-time Fourier transform and spectral subtraction technology, dynamically adjust the over-subtraction factor and the lower spectrum limit parameters, and train in combination with the ECAPA-TDNN network, and adaptively adjust the window function length to Improve recognition accuracy.
The accuracy of speaker recognition in noisy environments has been significantly improved. The test results show that the error rate is increased from 7% to 2.30%, and the combined effect of speech feature extraction and neural network is optimized.
Smart Images

Figure CN115116446B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of voiceprint recognition, and in particular to a method for constructing a speaker recognition model in a noisy environment. Background Art
[0002] As an important part of biometric recognition, voiceprint recognition is widely used in security, medical, financial and smart home fields. At present, speaker recognition technology has achieved satisfactory results in a quiet laboratory environment and with sufficient voice data. However, the actual application environment is unpredictable, resulting in poor robustness of the speaker recognition system. In real applications, affected by different noises in the environment, the system performance is significantly reduced compared to a pure environment, which seriously hinders the application and development of speaker recognition technology.
[0003] At present, most speech denoising solutions are based on deep neural networks, which are large in size and require high computational effort, making them inconvenient to be used in specific tasks such as speech recognition and voiceprint recognition. Therefore, current voiceprint recognition algorithms cannot meet the needs of speaker recognition with noise in real scenarios, and their recognition accuracy needs to be improved. Summary of the invention
[0004] The purpose of the present invention is to provide a method for constructing a speaker recognition model in a noisy environment.
[0005] The technical solution for achieving the purpose of the present invention is as follows:
[0006] A method for constructing a speaker recognition model in a noisy environment, wherein the speaker recognition model includes a denoising module, a speech feature extraction module and a neural network;
[0007] The denoising module is:
[0008] 1.1 Divide the input speech into frames and perform short-time Fourier transform to extract the linear spectrum P of the speech under the window function length T y (ω);
[0009] 1.2 Noise estimation, get the linear spectrum P of the noise e (ω);
[0010] 1.3 The linear spectrum of speech P y (ω) and the linear spectrum of the noise P e (ω) are converted into the energy spectrum P of speech respectively Y (ω) and the energy spectrum of the noise P E (ω), and then perform spectral subtraction to obtain the energy spectrum P of the clean speech X (ω), that is:
[0011] P Y (ω)=|P y(ω)| 2 ,
[0012] P E (ω)=|P e (ω)| 2 ,
[0013]
[0014]
[0015] Among them, α is the over-reduction factor, β is the spectrum lower limit parameter, γ is the spectrum coefficient, γ = 1;
[0016] 1.4 Smoothing the Energy Spectrum P of Clean Speech X (ω);
[0017] 1.5 Use inverse Fourier transform to restore clean speech;
[0018] Training the speaker recognition model: inputting the speech training set into the denoising module to obtain clean speech, extracting speech features using the speech feature extraction module, and then putting the speech features into the neural network for training to obtain a speaker feature vector; the speech features extracted by the speech feature extraction module are mel spectrograms; the neural network is ECAPA-TDNN; during the training process, the over-subtraction factor α and the spectrum lower limit parameter β of the denoising module are updated by the gradient descent method, and back propagation is used to update the gradient together with other parameters of the neural network;
[0019] The trained speaker recognition model is used to obtain the speaker's feature vector, and the cosine similarity between the speaker's feature vector and the feature vector already in the database is calculated to obtain the cosine distance. The speaker is determined based on the cosine distance and the threshold.
[0020] A further technical solution is that the window function length T is adaptively adjusted during the training process of the speaker recognition model: the selectable lengths are 256, 512, 1024 and 2048, the default is 256, and when the number of training rounds exceeds 40, the length of the window function is adjusted when the lowest equal error rate does not change for more than 10 rounds, and the length of the window function is selected in turn from four values.
[0021] The present invention adds a denoising module to the front end of the speaker recognition network and uses an improved spectral subtraction method to denoise noisy speech. For the two parameters of the spectral subtraction method (oversubtraction factor α and spectral lower limit parameter β), the gradient descent method is used to dynamically adjust the values of the two parameters by the neural network, retaining the speech information that is more in line with the speaker recognition task, thereby improving the accuracy of speaker recognition in a noisy environment. The noisy speech signal selects a window function with a variable window length for short-time Fourier transform, which can prevent the spectrum from leaking the speaker information required for the speaker recognition task. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a structural diagram of a specific embodiment.
[0023] Figure 2 Schematic diagram of Mel filter bank.
[0024] Figure 3 This is a curve diagram of equal error rate transformation when the speech denoising module is not added and when the speech denoising module is added.
[0025] Figure 4 This is the ECAPA-TDNN network structure diagram. DETAILED DESCRIPTION
[0026] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0027] Specific implementation examples Figure 1 As shown, the following steps are included:
[0028] 1. Load the original audio and read the original voice data. In this embodiment, the sampling frequency of the voice is 16000 Hz.
[0029] 2. Send the read voice data to the denoising module for voice denoising. The specific steps are as follows:
[0030] A1: Perform short-time Fourier transform on the input speech, extract the speech signal and divide it into frames, and obtain the linear spectrum characteristics of short-time Fourier transform (STFT) under variable window length. The formula can be expressed as:
[0031] y(t)=x(t)·ω(t),0≤t≤T-1(1)
[0032]
[0033] In the formula, x(t) is the signal at time t, and ω(t-τ) is the window function. Corresponding to the discrete scenario, STFT(ω, τ) is a two-dimensional matrix, and each column represents the result sequence after windowing the signal at different positions and performing Fourier transform on the obtained segments.
[0034] The window function length T is adaptively adjusted by the speaker recognition network during training, and the speech signal is processed to obtain the linear spectrogram P of the speech signal. y (ω).
[0035] A2: Take the period of time when there is no speech but only noise for noise estimation. In this embodiment, take the first N frames for noise estimation, 0<N≤30. Here, N=5 is taken by default, and the linear spectrum P of the noise is obtained. e (ω). The noise intensity in these N frames is averaged as the estimated noise.
[0036] A3: Square the modulus of the linear spectrum, convert the linear spectrum into an energy spectrum, perform spectral subtraction, and use the energy spectrum P of the speech Y (ω) Energy spectrum P minus the noise E (ω), and the energy spectrum P of the clean speech is obtained X (ω), its principle can be expressed as:
[0037] P Y (ω)=|P y (ω)| 2 (3)
[0038] P E (ω)=|P e (ω)| 2 (4)
[0039] P X (ω)=P Y (ω)-P E (ω) (5)
[0040] For the two parameters of the spectrum subtraction method, α (oversubtraction factor) and β (spectral lower limit parameter), a neural network is used to dynamically adjust the values of the two parameters to retain speech information that is more suitable for the speaker recognition task. γ is the spectrum coefficient. When γ is equal to 1, it is equivalent to subtracting two energy spectra. When it is equal to 0.5, it is equivalent to first subtracting two amplitude spectra and then squaring the result to obtain the energy spectrum. Here, the value of γ is 1.
[0041]
[0042]
[0043] A4: Smoothing the spectrum after spectrum subtraction;
[0044] Compute the maximum noise residual:
[0045]
[0046] P e (ω) is the initial noise estimate, which is obtained by taking the average of the noise intensity in the first N frames, P t (ω) is the noise spectrum of the tth frame in N frames, max P(ω) is the calculated maximum noise residual, and the part smaller than the maximum residual is replaced by the minimum value of the adjacent frame:
[0047]
[0048] k is the time interval. When the noise spectrum is smaller than the maximum residual, the minimum value from the adjacent t+k frame to tk frame is selected to replace it, making the speech transition smoother.
[0049] A5: Use inverse Fourier transform to restore the signal.
[0050] 3. Extract the mel spectrogram Mel(X, Y) from the denoised speech. The relationship between the mel scale and frequency is as follows:
[0051]
[0052] F mel is the perceived frequency in Mel, and f is the actual frequency in Hz. Figure 2 shown.
[0053] 4. Put the spectrogram features into the ECAPA-TDNN network training to obtain the speaker feature vector The two parameters of the spectral subtraction method are updated by calculating the gradient through the gradient descent method, and the back propagation is updated together with other parameters of the neural network. The parameters are updated using the gradient descent optimization method to minimize the loss function. The denoising module uses the updated over-subtraction factor α and the spectrum lower limit parameter β to continue denoising. For the length T of the window function, the optional lengths are 256, 512, 1024 and 2048, and the default is 256. When the number of training rounds exceeds 40 rounds, if the lowest equal error rate does not change for more than ten rounds, adjust the length of the window function. The length of the window function is selected in turn from the four values. Among them, the structure of the ECAPA-TDNN network is as follows: Figure 4 shown.
[0054] 5. When applying, use the trained speaker recognition model to obtain the speaker's feature vector and convert the speaker feature vector Compared with the feature vectors already in the database The cosine similarity calculation is performed as shown below:
[0055]
[0056] Among them, d is the cosine distance, and the speaker is determined based on the size of d and the threshold.
[0057] Simulation experiment:
[0058] The speaker dataset used in this method is the Free ST Chinese Mandarin Corpus Chinese dataset, and the noise dataset is the Noise92 noise dataset. The Free ST Chinese Mandarin Corpus Chinese dataset itself is a clean speech dataset. The factory noise in the Noise92 noise dataset is selected and combined with the Free ST ChineseMandarin Corpus dataset to synthesize a noisy dataset with a signal-to-noise ratio of 5dB. The Free ST Chinese MandarinCorpus dataset contains a total of 855 people, each with 120 voices. 90% of the 855 people are used as training sets and 10% as test sets. The test results after using this method show that the error rate reached 2.30%, which is 7% higher than that without adding the denoising module. The error rate conversion curve is shown as follows Figure 3 As shown. For other speech features, such as MFCC, PLP, etc., and other neural networks, such as ResNet, LSTM, etc., the combination of speech features and networks in this embodiment produces the best experimental results.
Claims
1. A method for constructing a speaker recognition model in a noisy environment, characterized in that: The speaker recognition model includes a denoising module, a speech feature extraction module and a neural network; The denoising module is: 1.1 Divide the input speech into frames and perform short-time Fourier transform to extract the linear spectrum P of the speech under the window function length T y (ω); 1.2 Noise estimation, get the linear spectrum P of the noise e (ω); 1.3 The linear spectrum P of speech y (ω) and the linear spectrum of the noise P e (ω) are converted into the energy spectrum P of speech respectively Y (ω) and the energy spectrum of the noise P E (ω), and then perform spectral subtraction to obtain the energy spectrum P of the clean speech X (ω), that is: P Y (ω)=|P y (oh)| 2 , P E (ω)=|P e (oh)| 2 , Among them, α is the over-reduction factor, β is the spectrum lower limit parameter, γ is the spectrum coefficient, γ = 1; 1.4 Smoothing the Energy Spectrum P of Clean Speech X (ω); 1.5 Use inverse Fourier transform to restore clean speech; Training the speaker recognition model: inputting the speech training set into the denoising module to obtain clean speech, extracting speech features using the speech feature extraction module, and then putting the speech into the neural network for training to obtain a speaker feature vector; the speech features extracted by the speech feature extraction module are mel spectrograms; the neural network is ECAPA-TDNN; During the training process, the over-subtraction factor α and the spectrum lower limit parameter β of the denoising module are updated by the gradient descent method, and the back propagation is used to update the gradient together with other parameters of the neural network; The trained speaker recognition model is used to obtain the speaker's feature vector, and the cosine similarity between the speaker's feature vector and the feature vector already in the database is calculated to obtain the cosine distance. The speaker is determined based on the cosine distance and the threshold.
2. The method for constructing a speaker recognition model in a noisy environment as claimed in claim 1, characterized in that: The window function length T is adaptively adjusted during the training process of the speaker recognition model: the selectable lengths are 256, 512, 1024 and 2048, with a default of 256. When the number of training rounds exceeds 40, the length of the window function is adjusted when the lowest equal error rate does not change for more than 10 rounds. The length of the window function is selected in turn from four values.
