Improved ELM speech enhancement method and device for noise reduction

Abstract

The application discloses an improved ELM speech enhancement method and device for noise reduction, and belongs to the field of speech signal processing; specifically, first, a noisy speech signal and a clean speech signal are acquired to construct training samples; meanwhile, an improved ELM speech enhancement network is constructed, the training samples are used for training, and a network weight matrix is saved; then, a new noisy speech signal is acquired, input into the trained enhancement network to output a noise-reduced enhanced speech signal; the device comprises an audio acquisition module, an offline training module, an online running module and an audio playing module; the audio acquisition module acquires speech signal samples used for training and inputs the offline training module to train the improved ELM speech enhancement network and save the network weight matrix; a new noisy speech signal is acquired and input into the online running module to directly output a noise-reduced enhanced speech signal; and the audio playing module is used for playing. The application reduces training cost and has lower processing time delay.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing, and specifically relates to an improved ELM (extreme learning machine) speech enhancement method and apparatus for noise reduction. Background Technology

[0002] Classical speech enhancement methods typically make strong assumptions and are weak in handling nonlinear noise; their performance often depends on the accuracy of environmental noise estimation. In recent years, leveraging the powerful learning capabilities of neural networks, deep learning-based speech enhancement methods have demonstrated excellent noise reduction performance. However, these methods suffer from long training times, complex parameter tuning, and difficulty in deployment in embedded devices.

[0003] Compared to deep neural networks, ELM is a single-hidden-layer network that does not require backpropagation to update weight parameters, and the hidden layer weights and biases can be randomly generated. Therefore, ELM networks have advantages such as fast training speed and low processing latency.

[0004] To fully utilize the advantages of ELM networks, they are introduced into the field of speech enhancement. Furthermore, to further improve the ELM network's ability to suppress nonlinear noise, a statistical prior layer is introduced, forming an improved ELM speech enhancement network. The statistical prior layer can suppress some noise and also provides direction for optimizing the improved ELM speech enhancement network. Summary of the Invention

[0005] This invention provides an improved ELM speech enhancement method and apparatus for noise reduction. Compared with classic speech enhancement methods, this method can achieve better speech enhancement results. Compared with deep speech enhancement methods based on amplitude spectrum mapping, it can reduce training overhead and process time latency, and is easier to deploy on low-power devices.

[0006] The improved ELM speech enhancement method for noise reduction includes the following steps:

[0007] Step 1: Obtain noisy speech signals from open-source datasets With clean speech signals Construct a training sample set;

[0008] The specific process of constructing the training sample set is as follows:

[0009] First, using the acquired noisy speech signal With clean speech signals Calculate noise signal , represented as:

[0010] ;

[0011] in, , , Indicates the sample point index of the speech signal; This represents the number of sample points in the speech signal.

[0012] Then, the noisy speech signals were respectively Clean voice signal With noise signal Transform to the time-frequency domain and calculate the amplitude spectrum of the noisy speech signal. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal ;

[0013] in, and These represent the number of frames and the number of frequency points after the speech signal is transformed into the time-frequency domain, respectively.

[0014] Finally, the amplitude spectrum of the noisy speech signal was analyzed. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal Forming a training sample set .

[0015] Step 2: Construct an improved ELM speech enhancement network;

[0016] The improved ELM speech enhancement network architecture consists of one input layer, one statistical prior layer, one hidden layer, and one output layer; wherein the number of nodes in the input layer is... The number of hidden layer nodes is The number of output layer nodes is ;

[0017] The statistical prior layer is initialized with a weight matrix of all zeros. The hidden value is the mean. The variance is Gaussian distribution generates weight matrix and bias vector and the bias vector Expanded to bias matrix ;

[0018] Step 3: Train the improved ELM speech enhancement network based on the training sample set and save the network weight matrix;

[0019] The specific training process is as follows:

[0020] Step 301: Calculate the amplitude spectrum of the training sample set. and The inputs are fed into the improved ELM speech enhancement network, and the weight matrix of the statistical prior layer is calculated. , represented as:

[0021] ;

[0022] in, , , The weight matrix represents the first... Okay, number Column elements; The amplitude spectrum of a clean speech signal is represented by the first... Okay, number Column elements; The amplitude spectrum of the noise signal is represented by the first... Okay, number Column elements;

[0023] Step 302: Based on the weight matrix Calculate the output of the statistical prior layer , represented as:

[0024] ;

[0025] The amplitude spectrum of the noisy speech signal is represented by the first... Okay, number Column elements;

[0026] Step 303: Calculate the output of the statistical prior layer. The input is fed into the hidden layer, resulting in the output of the improved ELM speech enhancement network hidden layer. , represented as:

[0027] ;

[0028] in, This represents the sigmoid activation function. ;

[0029] Step 304: Based on the output of the hidden layer amplitude spectrum of clean speech signal Calculate the weight matrix of the output layer. ;

[0030] Represented as:

[0031] ;

[0032] in, Expressing the request Moore–Penrose pseudo-inverse, ;

[0033] Step 305: Finally, the weight matrix... and Weight matrix and This is used as the network weight matrix and saved.

[0034] Step 4: Acquire new noisy speech signals, input them into the trained improved ELM speech enhancement network for testing, and obtain denoised enhanced speech signals. ;

[0035] The specific process is as follows:

[0036] Step 401: Obtain the new noisy speech signal and transform it to the time-frequency domain to obtain the corresponding amplitude spectrum. Simultaneously, the phase of the noisy speech signal is acquired. ;

[0037] in, This indicates the number of frames after transforming the new noisy speech signal into the time-frequency domain;

[0038] Step 402: Determine the amplitude spectrum Frame count Is it equal to the weight matrix? dimensional coefficients If yes, proceed to step 403; otherwise, proceed to step 404.

[0039] Step 403: Based on the amplitude spectrum Calculate the enhanced amplitude spectrum for testing Proceed to step 406;

[0040] Enhanced amplitude spectrum Represented as:

[0041] ;

[0042] in, Represents the element-wise product of matrices;

[0043] Step 404: The weight matrix... Updated to , represented as:

[0044] ;

[0045] in, , Represents the weight matrix No. List all elements;

[0046] Step 405: Based on the amplitude spectrum With the updated weight matrix Calculate the enhanced amplitude spectrum for testing Proceed to step 406;

[0047] Represented as:

[0048] ;

[0049] in, The frame index represents the amplitude spectrum of the noisy speech signal used in the test; Indicates amplitude spectrum No. List all elements;

[0050] Step 406, based on phase Enhanced amplitude spectrum for testing The enhanced speech signal is obtained through short-time inverse Fourier transform. ;

[0051] This indicates the number of sample points for the enhanced speech signal.

[0052] The improved ELM speech enhancement device for noise reduction includes an audio acquisition module, an improved ELM speech enhancement network offline training module, an improved ELM speech enhancement network online operation module, and an audio playback module.

[0053] The audio acquisition module is used to acquire speech signal sample sets for training and testing of the improved ELM network. The training sample set is input into the offline training module of the improved ELM speech enhancement network to train the improved ELM speech enhancement network and save the weight matrix after training. After testing with the test sample set, a new noisy speech signal is obtained and input into the online running module of the improved ELM speech enhancement network, which directly outputs the denoised enhanced speech signal and plays it through the audio playback module.

[0054] The advantages of this invention are:

[0055] 1) An improved ELM speech enhancement method for noise reduction. Compared with the classic spectral subtraction method, the present invention can effectively suppress nonlinear noise and obtain better speech noise reduction effect. Compared with the mapping-based deep speech enhancement method, the present invention adopts a speech enhancement method based on the improved ELM network, which can effectively reduce training overhead and reduce processing latency.

[0056] 2) An improved ELM voice enhancement device for noise reduction, which starts from the perspective of real-time noise reduction of embedded devices, combines the advantages of prior information and ELM network, and uses an improved ELM voice enhancement network to suppress environmental noise. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating an improved ELM speech enhancement method for noise reduction according to the present invention.

[0058] Figure 2 This is a schematic diagram of the structure of the improved ELM speech enhancement network of the present invention;

[0059] Figure 3 This is a schematic diagram of offline training of the improved ELM speech enhancement network of the present invention;

[0060] Figure 4 This is a schematic diagram of the online operation of the improved ELM speech enhancement network of the present invention;

[0061] Figure 5 This is a schematic diagram of the structure of the improved ELM network voice enhancement device of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0063] Compared with classic speech enhancement methods, such as Wiener filtering and MMSE estimation, this invention can achieve better speech enhancement results; compared with deep speech enhancement methods based on amplitude spectrum mapping, it can reduce training overhead and have lower processing time latency, and is easier to deploy on low-power devices.

[0064] The improved ELM speech enhancement method for noise reduction, such as Figure 1 As shown, it includes the following steps:

[0065] Step 1: Obtain noisy speech signals from open-source datasets With clean speech signals Construct a training sample set;

[0066] The noisy and clean speech signals are provided by open-source datasets, either artificially synthesized or collected by devices.

[0067] The specific process of constructing the training sample set is as follows:

[0068] First, using the acquired noisy speech signal With clean speech signals Calculate noise signal , represented as:

[0069] ;

[0070] in, , , Indicates the sample point index of the speech signal; This represents the number of sample points in the speech signal.

[0071] Then, the noisy speech signals were respectively Clean voice signal With noise signal Transform to the time-frequency domain and calculate the amplitude spectrum of the noisy speech signal. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal ; and These represent the number of frames and the number of frequency points after the speech signal is transformed to the time-frequency domain, respectively. The amplitude spectrum is represented as:

[0072] ;

[0073] in, , , , These represent the Fourier transform length, window length, window shift, and window function, respectively. This indicates the short-time Fourier transform operation; This represents the element-wise modulo operation on a complex matrix; , ; This indicates rounding the element down;

[0074] The Fourier transform length, window length, window shift, and window function are preset based on engineering experience.

[0075] Finally, the amplitude spectrum of the noisy speech signal was analyzed. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal Forming a training sample set .

[0076] Step 2: Construct an improved ELM speech enhancement network;

[0077] like Figure 2 As shown, the improved ELM speech enhancement network architecture consists of one input layer, one statistical prior layer, one hidden layer, and one output layer; wherein, the number of nodes in the input layer is... The number of hidden layer nodes is The number of output layer nodes is ;

[0078] The statistical prior layer is a custom layer used to obtain the weight matrix of the statistical prior layer. The weight matrix is ​​initialized to all zeros. The hidden value is the mean. The variance is Gaussian distribution generates weight matrix and bias vector and the bias vector Expanded to bias matrix Among them, the number of hidden layer nodes , and Pre-set based on engineering experience .

[0079] Step 3: Train the improved ELM speech enhancement network based on the training sample set and save the network weight matrix;

[0080] like Figure 3 As shown, the specific training process is as follows:

[0081] Step 301: Calculate the amplitude spectrum of the training sample set. and The inputs are fed into the improved ELM speech enhancement network, and the weight matrix of the statistical prior layer is calculated. , represented as:

[0082] ;

[0083] in, , , The weight matrix represents the first... Okay, number Column elements; The amplitude spectrum of a clean speech signal is represented by the first... Okay, number Column elements; The amplitude spectrum of the noise signal is represented by the first... Okay, number Column elements;

[0084] Step 302: Based on the weight matrix Calculate the output of the statistical prior layer , represented as:

[0085] ;

[0086] The amplitude spectrum of the noisy speech signal is represented by the first... Okay, number Column elements; Indicates the nth element in the output matrix Okay, number Column elements;

[0087] Step 303: Calculate the output of the statistical prior layer. The input is fed into the hidden layer, resulting in the output of the improved ELM speech enhancement network hidden layer. , represented as:

[0088] ;

[0089] in, This represents the sigmoid activation function. ;

[0090] Step 304: Based on the output of the hidden layer amplitude spectrum of clean speech signal Calculate the weight matrix of the output layer. ;

[0091] Represented as:

[0092] ;

[0093] in, Expressing the request Moore–Penrose pseudo-inverse, ;

[0094] Step 305: Finally, the weight matrix... and Weight matrix and This is used as the network weight matrix and saved.

[0095] Step 4: Acquire new noisy speech signals, input them into the trained improved ELM speech enhancement network for testing, and obtain denoised enhanced speech signals. ;

[0096] like Figure 4 As shown, the specific process is as follows:

[0097] Step 401: Acquire the new noisy speech signal and transform it to the time-frequency domain to obtain the corresponding amplitude spectrum. Simultaneously, the phase of the noisy speech signal is acquired. ;

[0098] in, This indicates the number of frames after transforming the new noisy speech signal into the time-frequency domain;

[0099] Step 402: Determine the amplitude spectrum Frame count Is it equal to the weight matrix? dimensional coefficients If yes, proceed to step 403; otherwise, proceed to step 404.

[0100] Step 403: Based on the amplitude spectrum Calculate the enhanced amplitude spectrum for testing Proceed to step 406;

[0101] Enhanced amplitude spectrum Represented as:

[0102] ;

[0103] in, Represents the element-wise product of matrices;

[0104] Step 404: The weight matrix... Updated to , represented as:

[0105] ;

[0106] in, , Represents the weight matrix No. List all elements;

[0107] Step 405: Based on the amplitude spectrum With the updated weight matrix Calculate the enhanced amplitude spectrum for testing Proceed to step 406;

[0108] Represented as:

[0109] ;

[0110] in, The frame index represents the amplitude spectrum of the noisy speech signal used in the test; Indicates amplitude spectrum No. List all elements;

[0111] Step 406, based on phase Enhanced amplitude spectrum for testing The enhanced speech signal is obtained through short-time inverse Fourier transform. ; This indicates the number of sample points for the enhanced speech signal.

[0112] The improved ELM speech enhancement device for noise reduction, such as Figure 5As shown, it includes an audio acquisition module, an improved ELM speech enhancement network offline training module, an improved ELM speech enhancement network online running module, and an audio playback module;

[0113] The audio acquisition module is used to acquire speech signal sample sets for training and testing of the improved ELM network. The training sample set is input into the offline training module of the improved ELM speech enhancement network to train the improved ELM speech enhancement network and save the weight matrix after training. After testing with the test sample set, a new noisy speech signal is obtained and input into the online running module of the improved ELM speech enhancement network, which directly outputs the denoised enhanced speech signal and plays it through the audio playback module.

[0114] The improved ELM speech enhancement network offline training module is used to provide the weight matrix to the improved ELM speech enhancement network online running module, and stops working after training is completed.

[0115] Example:

[0116] S1: An example of constructing a training sample set is as follows:

[0117] When the number of sample points takes values The acquired noisy speech signal Clean voice signal Fourier transform length , Window moved Window function If it is a Hamming window, then the noise signal for:

[0118] ;

[0119] The number of frequency points and the number of frames after the speech signal is transformed to the time-frequency domain are as follows: , The amplitude spectrum of the noisy speech signal after short-time Fourier transform. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal They are represented as follows:

[0120] ,

[0121] ,

[0122] ;

[0123] S2: Construct an improved ELM speech enhancement network;

[0124] The bias vector Expanded to bias matrix Here are some examples:

[0125] Assumptions: Short-time Fourier transform length Number of hidden layer nodes Then the bias matrix for:

[0126] ;

[0127] in, ;

[0128] S3: Offline training: Based on the training sample set and the constructed improved ELM speech enhancement network, train the improved ELM speech enhancement network and save the model weight matrix;

[0129] like Figure 3 As shown, the specific steps include:

[0130] S301, Based on the improved ELM speech enhancement network and Calculate the weight matrix of the statistical prior layer. ;

[0131] S302, Based on the weight matrix of the statistical prior layer Calculate the output of the statistical prior layer ;

[0132] S303, Statistically calculate the output of the prior layer. The input is fed into the hidden layer of the improved ELM speech enhancement network, and the output of the improved ELM speech enhancement network hidden layer is obtained. ;

[0133] S304, Based on the output of the hidden layer of the improved ELM speech enhancement network amplitude spectrum of clean speech signal The weight matrix of the output layer is obtained. ;

[0134] S305, Save the weight matrix of the improved ELM speech enhancement network. , , and .

[0135] S4: Online Operation: Based on the saved weight matrix of the improved ELM speech enhancement network, test the amplitude spectrum of the noisy speech signal. With phase To obtain an enhanced speech signal ;

[0136] in, This indicates the number of frames after the noisy test speech signal is transformed into the time-frequency domain; This indicates the number of sample points for the enhanced speech signal.

[0137] like Figure 4 As shown, the specific process includes: first, determining the amplitude spectrum of the noisy speech signal used for testing. Frame count Is it equal to the weight matrix of the statistical prior layer of the improved ELM speech enhancement network? dimensional coefficients ;

[0138] When they are equal, the amplitude spectrum of the input test noisy speech signal is used. Calculate the enhanced amplitude spectrum for testing ;

[0139] When not waiting, the weight matrix of the statistical prior layer of the improved ELM speech enhancement network is used. Updated to Based on the input amplitude spectrum , and the updated weight matrix Calculate the enhanced amplitude spectrum for testing

[0140] Finally, based on the phase of the noisy speech signal used in the test... Enhanced amplitude spectrum for testing The enhanced speech signal is obtained through short-time inverse Fourier transform. .

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An improved ELM speech enhancement method for noise reduction, characterized in that, The specific steps are as follows: Step 1: Obtain noisy speech signals from open-source datasets With clean speech signals Construct a training sample set; Step 2: Construct an improved ELM speech enhancement network; The improved ELM speech enhancement network architecture consists of one input layer, one statistical prior layer, one hidden layer, and one output layer; wherein the number of nodes in the input layer is... The number of hidden layer nodes is The number of output layer nodes is ; The statistical prior layer is initialized with a weight matrix of all zeros. The hidden value is the mean. The variance is Gaussian distribution generates weight matrix and bias vector and the bias vector Expanded to bias matrix ; in, and These represent the number of frames and the number of frequency points after the speech signal is transformed into the time-frequency domain, respectively. Step 3: Train the improved ELM speech enhancement network based on the training sample set and save the network weight matrix; The specific training process is as follows: Step 301: Calculate the amplitude spectrum of the training sample set. and The inputs are fed into the improved ELM speech enhancement network, and the weight matrix of the statistical prior layer is calculated. , represented as: ； in, , , The weight matrix represents the first... Okay, number Column elements; The amplitude spectrum of a clean speech signal is represented by the first... Okay, number Column elements; The amplitude spectrum of the noise signal is represented by the first... Okay, number Column elements; Step 302: Based on the weight matrix Calculate the output of the statistical prior layer , represented as: ； The amplitude spectrum of the noisy speech signal is represented by the first... Okay, number Column elements; Step 303: Calculate the output of the statistical prior layer. The input is fed into the hidden layer, resulting in the output of the improved ELM speech enhancement network hidden layer. , is represented as: ； in, This represents the sigmoid activation function. ; Step 304: Based on the output of the hidden layer Amplitude spectrum of clean speech signal Calculate the weight matrix of the output layer. ; Represented as: ； in, Expressing the request Moore–Penrose pseudo-inverse, ; Step 305: Finally, the weight matrix... and Weight matrix and This is stored as a network weight matrix. Step 4: Acquire new noisy speech signals, input them into the trained improved ELM speech enhancement network for testing, and obtain denoised enhanced speech signals. .

2. The method as described in claim 1, characterized in that, The process of training the sample set in step one is as follows: First, using the acquired noisy speech signal With clean speech signals Calculate noise signal , represented as: ； in, , , Indicates the sample point index of the speech signal; Indicates the number of sample points in the speech signal; Then, the noisy speech signals were respectively Clean voice signal With noise signal Transform to the time-frequency domain and calculate the amplitude spectrum of the noisy speech signal. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal ; Finally, the amplitude spectrum of the noisy speech signal was analyzed. Amplitude spectrum of noise signal amplitude spectrum of clean speech signal Forming a training sample set .

3. The method as described in claim 1, characterized in that, The specific process of step four is as follows: Step 401: Obtain the new noisy speech signal and transform it to the time-frequency domain to obtain the corresponding amplitude spectrum. Simultaneously, the phase of the noisy speech signal is acquired. ; in, This indicates the number of frames after transforming the new noisy speech signal into the time-frequency domain; Step 402: Determine the amplitude spectrum Frame count Is it equal to the weight matrix? dimensional coefficients If yes, proceed to step 403; otherwise, proceed to step 404. Step 403: Based on the amplitude spectrum Calculate the enhanced amplitude spectrum for testing Proceed to step 406; Enhanced amplitude spectrum Represented as: ； in, Represents the element-wise product of matrices; Step 404: The weight matrix... Updated to , represented as: ； in, , Represents the weight matrix No. List all elements; Step 405: Based on the amplitude spectrum With the updated weight matrix Calculate the enhanced amplitude spectrum for testing Proceed to step 406; Represented as: ； in, The frame index represents the amplitude spectrum of the noisy speech signal used for testing; Indicates amplitude spectrum No. List all elements; Step 406, based on phase Enhanced amplitude spectrum for testing The enhanced speech signal is obtained through short-time inverse Fourier transform. ; This indicates the number of sample points for the enhanced speech signal.

4. An improved ELM speech enhancement device for noise reduction using the method of claim 1, characterized in that, It includes an audio acquisition module, an improved ELM speech enhancement network offline training module, an improved ELM speech enhancement network online running module, and an audio playback module; The audio acquisition module is used to acquire the speech signal sample set of the improved ELM network used for training and testing, and input the training sample set into the offline training module of the improved ELM speech enhancement network to train the improved ELM speech enhancement network and save the weight matrix after training. After testing with the test sample set, a new noisy speech signal is input into the improved ELM speech enhancement network online operating module, which directly outputs the noise-reduced enhanced speech signal; and plays it through the audio playback module.

Images