A language recognition method based on Gaussian weighted GFCC features

Abstract

The application relates to a language recognition method based on Gaussian weighting GFCC features and belongs to the technical field of voice recognition. First, GFCC features are extracted from original voice; second, the required Gaussian weight coefficient is calculated according to the dimension of the extracted GFCC features; the calculated Gaussian weight coefficient is multiplied with the corresponding dimension of the GFCC features to obtain weighted features; the improved GFCC features are input into a GMM model for training; and finally, the voice to be recognized is input into a language recognition system for recognition verification. The method provided by the application not only is effective under the condition of pure voice, but also improves the recognition rate under the high-frequency channel noise background environment when the signal-to-noise ratio is 15dB, 10dB or 5dB.

Description

Technical Field

[0001] This invention relates to a language recognition method based on Gaussian weighted GFCC features, belonging to the field of speech recognition technology. Background Technology

[0002] Language identification is the process of analyzing a segment of speech input into a computer to determine its language. The development of automatic language identification has, to some extent, solved the problem of large-scale cross-language speech recognition, enabling people to make a preliminary judgment about languages ​​not native to them. With the development of globalization and the information age, automatic language identification is becoming increasingly important in international communication, especially in information retrieval and security fields, and even in the military field. As international exchanges become more frequent and cooperation increases, the sudden outbreak of the pandemic has forced many international conferences to be held online, making language barriers unavoidable. While current machine translation performance has achieved good recognition results, the lack of language identification assistance at the front end only increases human labor; therefore, language identification still has significant research value. Currently, commonly used features in language recognition include Mel-Frequency Cepstral Coefficients (MFCC), Perceptral Linear Prediction Coefficients (PLPC), and Gammatone Filter Cepstral Coefficients (GFCC). However, relying solely on these features is far from sufficient to achieve better recognition rates, and improvements to these features are necessary. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a language identification method based on Gaussian weighted GFCC features, so as to solve the above-mentioned problem.

[0004] The technical solution of this invention is: a language recognition method based on Gaussian weighted GFCC features. The method involves performing amplitude normalization, pre-emphasis, framing, and windowing on the speech signal at the front end to obtain a framed speech signal. A Fast Fourier Transform (FFT) is then performed on the framed signal, and the modulus is taken to obtain a modulo-adjusted signal. This signal is then passed through a Gammatone filter bank, and the logarithm is taken to obtain a coefficient matrix. This coefficient matrix is ​​then subjected to a Discrete Cosine Transform (DCT) to decorrelate, yielding the GFCC features. Based on the dimension of the extracted features, Gaussian weight coefficients for the corresponding dimension are calculated. These weight coefficients are multiplied by the features to obtain improved features. The improved features are input into a training model for training, resulting in a Gaussian mixture model for the language. Finally, the GFCC features to be recognized are extracted, Gaussian weighted, and input into the language recognition system to obtain the recognition result.

[0005] The specific steps are as follows:

[0006] Step 1: Preprocess the speech signal, including amplitude normalization, pre-emphasis, framing, windowing, and other steps.

[0007] Amplitude normalization eliminates the influence of volume on the recognition result. Let the speech signal be x(n), then the formula for amplitude normalization is as follows:

[0008]

[0009] In the formula, x′(n) is the signal after amplitude normalization, and max(x(n)) is the maximum value of the speech signal.

[0010] Pre-emphasis is used to compensate for the high-frequency components lost during training and recognition. The expression for the pre-emphasis filter is:

[0011] H(z) = 1 - az -1 (2)

[0012] In the formula, a is a constant between 0.90 and 0.99. The pre-emphasized signal is x″(n).

[0013] The purpose of framing is to cut a non-stationary speech signal into short, potentially stationary segments, allowing for processing using methods for stationary signals. To ensure a smooth transition between frames, there will be some overlap between adjacent frames. Extensive experiments show that a frame length of 10ms to 30ms is optimal, and the overlap is typically 1 / 3 to 1 / 2 of the frame length. The i-th frame after framing is denoted as x″. i (m), frame length is W, frame shift is L, and number of frames is F.

[0014] The purpose of adding a window is to reduce spectral leakage. The main types of windows are Hamming windows and Heining windows. The expression for a Hamming window is:

[0015]

[0016] The expression for Haining window is:

[0017]

[0018] In equations (3) and (4), N represents the length of the window function; generally, W = N. The i-th frame signal, after being windowed, yields signal x″′. i (n) is as follows:

[0019] x″′ i (m)=w(m)×x″ i(m), 1≤m≤N (5)

[0020] Step 2: Perform a Fast Fourier Transform on the preprocessed speech signal.

[0021] Using the Fast Fourier Transform to convert the time-domain signal to the frequency domain is more conducive to reading information from the speech signal. i The formula for transforming (n) to the frequency domain is as follows:

[0022] X i (k) = FFT[x″′ i (m)]1≤k≤W,1≤i≤F,1≤m≤W (6)

[0023] In the formula, X i (k) is the signal after the Fast Fourier Transform.

[0024] Step 3: Calculate the magnitude of the speech signal after Fast Fourier Transform (FFT), for X i (k) The signal after modulo taking each data point is |X i (k)|.

[0025] Step 4: After taking the modulus of the speech signal, input it into a Gammatone filter for filtering. Calculate the energy of each frame's spectral line passing through the Gammatone filter using the following formula:

[0026]

[0027] In the formula, S(i,m) is the energy of the spectral line after passing through the Gammatone filter, and G... m (k) represents the frequency domain response of the m-th filter.

[0028] Step 5: Perform a logarithmic operation on the filtered speech signal, and take the logarithm of the filtered energy S(i,m) as ln[S(i,m)].

[0029] Step 6: Perform discrete cosine processing on the logarithmized filtered signal.

[0030] Because adjacent Gammatone filters overlap, the energy passing through the filters exhibits strong correlation, which is detrimental to the back-end model. Therefore, discrete cosine transform is needed to remove the correlation and obtain the final GFCC coefficients.

[0031] gfcc(i,m)=DCT[ln(S(i,m))] (8)

[0032] Step 7: Apply Gaussian weighting coefficients to the obtained feature coefficients.

[0033] For language recognition, numerous experiments have shown that low-frequency information is easily affected by noise, and the higher-order GFCC coefficients have small values ​​and low contribution; the most important information is mainly in the mid-frequency range. Therefore, higher weights are assigned to the more important mid-frequency information, while the low-frequency and high-frequency components are weakened. Let the dimension of the extracted GFCC features be l, then the corresponding Gaussian weight coefficient calculation formula is:

[0034]

[0035] In the formula, C is a constant. To prevent the low-frequency and high-frequency components from completely decaying to zero, C is generally taken as 0.5. A is the increase of the Gaussian weighting coefficient, which controls the magnitude of the coefficient. σ = 1, 2, ..., 10.

[0036] Step 8: Input the cepstral coefficient features of the Gaussian-weighted Gammatone filter into the training model for training, thereby obtaining the Gaussian mixture model.

[0037] Step 9: The speech input to be recognized is scored and judged by the recognition system to obtain the final recognition result.

[0038] The beneficial effects of this invention are: the method proposed in this invention is not only effective under pure speech conditions, but also improves the recognition rate in high-frequency channel noise background environments with signal-to-noise ratios of 15dB, 10dB, and 5dB. Attached Figure Description

[0039] Figure 1 This is the language recognition theory training recognition image of the present invention;

[0040] Figure 2 This is a partial waveform diagram of high-frequency channel noise in this invention;

[0041] Figure 3 The original speech waveform diagram and the partial waveform diagrams of the present invention under three signal-to-noise ratios of 15dB, 10dB and 5dB with high-frequency channel noise.

[0042] Figure 4 This is a waveform diagram of a frame of signal after the clean speech preprocessing of this invention;

[0043] Figure 5 This is a flowchart of the GFCC feature extraction process of the present invention;

[0044] Figure 6 This is a Gaussian weight coefficient diagram of the GFCC feature of this invention when μ=7, σ=[1,2,...,10] is 14-dimensional;

[0045] Figure 7 This is a language recognition diagram of the server and client in the application scenario of this invention;

[0046] Figure 8 This is a diagram showing the recognition results of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0048] Language identification, as an important branch of speech recognition, is more challenging than speaker identification, mainly due to the weak information presented by language in speech, the non-fixed speaker, and the variable content of the speech. Therefore, extracting more effective, robust, and discriminative features is of significant research value. Based on this research background, this invention proposes an improved Gammatone Filter Cepstral Coefficients (GFCC) feature. First, GFCC features are extracted from the original speech. Second, the required Gaussian weight coefficients are calculated based on the dimension of the extracted GFCC features. The calculated Gaussian weight coefficients are multiplied by the corresponding dimension of the GFCC features to obtain the weighted features. The improved GFCC features are then input into a GMM model for training. Finally, the speech to be recognized is input into the language identification system for verification.

[0049] The specific steps are as follows:

[0050] Step 1: Test audio data acquisition;

[0051] The corpora used in this invention are all from China Radio International, and mainly include five languages: Chinese, Tibetan, Uyghur, English, and Kazakh. Each corpus lasts for 10 seconds, with a sampling frequency of 8 kHz and a sampling precision of 16 bits per channel.

[0052] Step 2: Generating noisy speech;

[0053] See Figure 2 and Figure 3 , Figure 2 This is a waveform diagram of high-frequency channel noise. Figure 3 The original waveform and the speech waveforms at signal-to-noise ratios of 15dB, 10dB, and 5dB show that as the signal-to-noise ratio decreases, more of the speech waveform is submerged by noise, and some speech information is masked.

[0054] Step 3: Preprocess the speech signal, including amplitude normalization, pre-emphasis, framing, and windowing.

[0055] Step 3.1: Amplitude normalization.

[0056] The obtained speech signal x(n) is normalized in amplitude to obtain:

[0057]

[0058] Step 3.2: Pre-weighting.

[0059] Pre-emphasis is used to compensate for the high-frequency components lost during training and recognition. The expression for the pre-emphasis filter is:

[0060] H(z) = 1 - az -1 (2)

[0061] In the formula, a is a constant between 0.90 and 0.99. The pre-emphasized signal is x″(n).

[0062] Step 3.3: Frame division.

[0063] The amplitude-normalized signal x′(n) is framed. In this invention, the frame length is W = 256 and the frame shift is L = 128.

[0064] Step 3.4: Add windows.

[0065] The window function used in this invention is the Hamming window. Multiplying each frame signal after framing by the Hamming window yields the windowed signal.

[0066] x″′ i (m)=w(m)×x″ i (m), 1≤m≤N (3)

[0067] See Figure 4 , Figure 4 This is a waveform diagram of a frame of signal after preprocessing the clean speech.

[0068] See Figure 5 , Figure 5 This describes the process for extracting GFCC feature parameters.

[0069] Step 4: Perform a Fast Fourier Transform on the preprocessed speech signal.

[0070] Step 5: Calculate the magnitude of the speech signal after Fast Fourier Transform.

[0071] Step 6: Extract the audio signal and then filter it using a Gammatone filter.

[0072] Step 7: Perform logarithmic operations on the filtered speech signal.

[0073] Step 8: Perform discrete cosine processing on the logarithmized filtered signal.

[0074] Step 9: Obtain GFCC feature parameters. The feature extracted in this invention is a static 14-dimensional GFCC feature, including the C0 energy coefficient term.

[0075] Step 10: Calculate the Gaussian weight coefficients r1, r2, ..., r based on the obtained feature dimensions. 14 After obtaining the weight coefficients, the extracted 14-dimensional GFCC features are multiplied by the weight coefficients to obtain new weighted features. The weighted GFCC feature coefficients of the i-th frame yield new feature parameters:

[0076]

[0077] See Figure 1 This invention employs a Gaussian mixture model language recognition system to train the corresponding language model. The experimental training samples consist of 300 samples for each language. When testing the recognition rate in noisy environments, the 300 speech samples are distributed according to the gradient SNR = [noise-free, 25dB, 20dB, 15dB, 10dB, 5dB], with 50 samples for each gradient. This better simulates real-world noisy environments.

[0078] Performance test examples of the method of this invention under a large number of speech sets:

[0079] The test sample set contains 171 speech samples for each language, totaling 855 speech samples across the five languages. The recognition rates were tested under four scenarios: no noise added, signal-to-noise ratio (SNR) of 15dB, SNR of 10dB, and SNR of 5dB, with and without feature weighting.

[0080] Following the specific steps, the recognition results are shown in Table 1.

[0081]

[0082] Table 1: Average Recognition Rate under Gaussian Weighted GFCC Features

[0083] The experimental results demonstrate the performance of this invention. Compared to the unweighted experiment, the recognition rate after feature weighting can be improved by more than 3%.

[0084] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A language identification method based on Gaussian weighted GFCC features, characterized in that: Step 1: Preprocess the speech signal, including amplitude normalization, pre-emphasis, framing, and windowing; Step 2: Perform Fast Fourier Transform on the preprocessed speech signal; Step 3: Calculate the magnitude of the speech signal after Fast Fourier Transform; Step 4: After taking the modulus of the voice signal, input it into the Gammatone filter for filtering; Step 5: Perform logarithmic operations on the filtered speech signal; Step 6: Perform discrete cosine processing on the logarithmic filtered signal; Step 7: Apply Gaussian weighting coefficients to the obtained feature coefficients; Step 8: Input the cepstral coefficient features of the Gaussian-weighted Gammatone filter into the training model for training, thereby obtaining the Gaussian mixture model; Step 9: The speech input to be recognized is scored and judged by the recognition system to obtain the final recognition result; Step 7 specifically refers to: Let the dimension of the extracted GFCC features be... The corresponding Gaussian weighting coefficients are: （9）； In the formula, A constant The increase of the Gaussian weighting coefficient , .

2. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that, Step 1 specifically refers to: Amplitude normalization: Let the speech signal be... Then the amplitude is normalized to: （1）； In the formula, The signal after amplitude normalization. This represents the maximum value of the speech signal. Preemphasis: The expression for the preemphasis filter is: （2）； In the formula, It is a constant between 0.90 and 0.99, and the pre-emphasized signal is... ; Framing: The length of one frame of signal is... The length of the overlapping portion is taken as the frame length. The first frame after the split Frame signal is Frame length is Frame shift is The number of frames is ; Adding windows: Hamming windows and Haining windows are used for adding windows. The expression for a Hamming window is: （3）； The expression for Haining window is: （4）； In equations (3) and (4), Represents the length of the window function. ; No. The signal obtained after windowing the frame signal for: （5）。 3. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that, Step 2 specifically includes: The Fast Fourier Transform is used to convert the signal from the time domain to the frequency domain. The formula for converting to the frequency domain is: （6）； In the formula, It is the signal after Fast Fourier Transform.

4. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that: The magnitude of the speech signal after the Fast Fourier Transform is .

5. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that, The filtering in Step 4 specifically involves: Calculate the energy of each spectral line as it passes through the Gammatone filter. （7）； In the formula, This represents the energy of the spectral line after passing through the Gammatone filter. For the first The frequency domain response of each filter.

6. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that, The logarithmic operation in Step 5 specifically involves: performing a logarithmic operation on the filtered energy... Take the logarithm as .

7. The language identification method based on Gaussian weighted GFCC features according to claim 1, characterized in that, Step 6 specifically refers to: The correlation is removed by discrete cosine transform to obtain the final GFCC coefficients; （8）。

Images