Low-computing-power narrowband call noise removal method and device

Abstract

The invention discloses a low-computing-power narrowband call noise removal method and device. The noise removal method comprises the steps of obtaining framing data; determining a data type corresponding to the framing data through the target model, wherein the data type comprises a voice frame and a noise frame; determining a noise variance according to the noise frame; constructing a Kalman equation according to the voice frame; and according to the noise variance and the Kalman equation, noise of the voice frame is removed, and a noise-reduced voice frame is obtained. Compared with the prior art, the method has the advantages that the data type of the framing data is determined, that is, whether the framing data contains voices or not only needs to be labeled, pure voices do not need to be extracted from voices containing noise for labeling, and manufacturing of label data can be simplified; meanwhile, a lightweight target model and a Kalman filtering method are adopted, the method can be suitable for low-computing-power narrow-band communication, noise is filtered in real time, the application range is wide, self-adaption is high, and then real-time noise reduction can be conducted on voice when the method operates on a low-performance dsp.

Description

Technical Field

[0001] The present invention relates to the technical field of speech noise reduction, and in particular to a method and device for removing narrowband call noise with low computing power. Background Art

[0002] Satellite calls are low-bandwidth scenarios. When collecting voice signals in this scenario, due to the long communication link and long distance, the limited bandwidth requires a lot of signal processing work, which inevitably causes unwanted sounds, the so-called noise. These noises mixed into the voice we want to collect will reduce the signal-to-noise ratio, and in severe cases will affect the intelligibility and recognizability of the voice. We all hope to remove the noise in the signal as much as possible, leaving only the voice signal. At this time, it is necessary to use a noise reduction algorithm to process the signal to improve the quality of the voice.

[0003] However, most of the existing technologies use deep learning methods such as the rnnoise method and the Conv-TasNet method to reduce speech noise. The rnnoise method needs to extract pure speech from the noisy speech for annotation, which makes it difficult to produce label data. In addition, this method uses the rnn model, which makes the calculation amount large, restricting its use on DSP chips with smaller computing power, thereby affecting the real-time performance of call speech noise reduction. When the Conv-TasNet method classifies noise and speech, the more types of noise, the more masks generated, and more masks will lead to an increase in calculation amount, thereby affecting the real-time performance of call speech noise reduction.

[0004] Therefore, the existing technology has a problem that the amount of computation required for voice noise reduction is large, resulting in the inability to meet the real-time requirements of call voice noise reduction. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a low-computing power narrowband call noise removal method and device, which can solve the problem in the prior art that the large amount of calculation leads to the inability to meet the real-time requirements of call voice noise reduction.

[0006] In order to solve the above problems, the present invention is implemented according to the following scheme: A low-computing-power narrowband call noise removal method is provided, comprising: Get frame data; Determine the data type corresponding to the framed data by using a target model, wherein the data type includes a speech frame and a noise frame; Determining a noise variance according to the noise frame; Constructing a Kalman equation according to the speech frame; According to the noise variance and the Kalman equation, the noise of the speech frame is removed to obtain a noise-reduced speech frame.

[0007] Compared with the prior art, the beneficial effects of a low-computing power narrowband call noise removal method provided by the present invention are as follows: determining the data type of the framed data, that is, only needing to mark the framed data whether it contains speech, there is no need to extract pure speech from the noisy speech for marking, which can simplify the production of label data; at the same time, a lightweight target model and Kalman filtering method are adopted, which can be applicable to low-computing power narrowband communications, and the noise can be filtered out in real time, and has a wide range of applications and strong adaptability, thereby achieving real-time noise reduction of speech even when running on a low-performance DSP, solving the problem in the prior art that the large amount of calculation makes it impossible to meet the real-time noise reduction of call speech.

[0008] Optionally, the acquiring of frame data includes: Obtaining the original speech signal; The original speech signal is subjected to frame processing, windowing processing and Fourier transform to obtain the frame data.

[0009] Optionally, determining the data type corresponding to the frame data by using a target model includes: Determining the target probability that the framed data is the speech frame by the target model; When the target probability is greater than or equal to a preset threshold, the frame data is the speech frame; When the target probability is less than the preset threshold, the frame data is the noise frame.

[0010] Optionally, determining the target probability that the framed data is the speech frame by using the target model includes: Performing a convolution operation on the frame data to obtain multi-channel speech features; Performing a convolution operation on the multi-channel speech feature to obtain a first speech feature and a second speech feature; Superimposing and reducing the dimension of the first speech feature and the second speech feature to obtain a multi-dimensional speech feature; Performing a convolution operation on the multi-dimensional speech feature to obtain a target speech feature; The cross entropy of the target speech feature is taken to obtain the target probability.

[0011] Optionally, determining the noise variance according to the noise frame includes: Estimating the noise spectrum of the noise frame by recursive averaging and minimum value tracking; Performing inverse Fourier transform on the noise spectrum to obtain a time domain signal of the noise frame; The noise variance is determined according to the time domain signal.

[0012] Optionally, constructing a Kalman equation according to the speech frame includes: Determining LPC coefficients and gains of the speech frame; The Kalman equation is constructed according to the LPC coefficient and the gain.

[0013] Optionally, determining the LPC coefficient and gain of the speech frame includes: Determining the LPC coefficients by a lattice method; Determining an estimated value of the speech frame according to the LPC coefficient; Determining an actual value of the speech frame; The gain is determined according to the estimated value and the actual value.

[0014] Optionally, the speech frame includes actual speech signals at multiple moments; and removing the noise of the speech frame according to the noise variance and the Kalman equation to obtain a noise-reduced speech frame includes: Predicting the speech frame by using the Kalman equation to obtain predicted speech signals at multiple moments of the speech frame; determining a target Kalman gain according to the noise variance; According to the predicted speech signal and the target Kalman gain, the actual speech signal is denoised to obtain a denoised speech signal; Determining a covariance matrix according to the target Kalman gain; Repeat the above steps until all the multiple actual speech signals of the speech frame are denoised, and the multiple denoised speech signals constitute the denoised speech frame.

[0015] Optionally, the target model is a TCN model.

[0016] A computer device is also provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the noise removal method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the noise removal method of the present invention; Figure 2 A schematic diagram of a binary classification model for training speech frames and noise frames of a target model in the present invention; Figure 3 This is a structural diagram of replacing the temporal convolution of the binary classification model in the present invention with the LSTM model. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0019] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0020] See also Figure 1 As shown, the present invention provides a low-computing-power narrowband call noise removal method, comprising: S1: Get frame data, including: First, the original speech signal is obtained; then, the original speech signal is subjected to frame processing, window processing and Fourier transform to obtain frame data.

[0021] When processing speech signals, 20ms-30ms is generally selected as a frame. In the present invention, 20ms is used as a frame of data, that is, the frame data is 20ms. In the process of processing the original speech signal, it is difficult to learn the speech features of the original speech signal that is too short, and the original speech signal that is too long cannot meet the real-time requirements. Therefore, considering the compromise between the two, 10 frames are selected as the unit of the input signal. In the present invention, one frame of data is 20ms, that is, a signal with a duration of 0.2s (200ms) is selected as the original speech signal.

[0022] In the present invention, it is also necessary to determine the number of frames corresponding to the sliding window and the step size. In this embodiment, 10 frames are used as the sliding window and 5 frames are used as the step size. Therefore, when the window processes the current frame data, it will include the current frame and the 5 frames before and after it, for a total of 10 frames; and each time a sliding window is processed, the window will move forward 5 frames and then process it again, so that two adjacent processing windows will overlap by 5 frames.

[0023] S2: Determine the data type corresponding to the frame data through the target model. The data type includes speech frames and noise frames, including: First, a convolution operation is performed on the frame data to obtain a multi-channel speech feature; a convolution operation is performed on the multi-channel speech feature again to obtain a first speech feature and a second speech feature, which are speech features of two channels respectively; then, the first speech feature and the second speech feature are superimposed and dimensionally reduced to obtain a multi-dimensional speech feature, and the first speech feature and the second speech feature of the two groups of channels are merged into a speech feature of one channel through the superposition operation, so the dimension of the multi-dimensional speech feature is the average dimension of the channel dimension corresponding to the first speech feature and the channel dimension corresponding to the second speech feature; a convolution operation is performed on the multi-dimensional speech feature again to obtain a target speech feature, and at this time the target speech feature is a two-dimensional feature, that is, it is changed from a multi-dimensional feature to a two-dimensional feature; finally, the cross entropy of the target speech feature is taken to obtain the target probability.

[0024] After determining the target probability corresponding to the frame data, the target probability is compared with the preset threshold to determine the data type of the frame data. Since accurately distinguishing between noise frames and speech frames is crucial for subsequent processing, the preset threshold in the present invention is 0.8; when the target probability is greater than or equal to the preset threshold, the frame data is a speech frame; when the target probability is less than the preset threshold, the frame data is a noise frame.

[0025] In the present invention, noise is removed from narrowband call voice data. Usually, the sampling rate of the voice signal in a narrowband call is 8 kHz, so each frame of data has a total of 160 data sequence data.

[0026] When performing the convolution operation on the frame data, a one-dimensional convolution operation is performed with a convolution kernel number of 1024, a convolution kernel size of 4, and a convolution step size of 2. Therefore, the obtained multi-channel speech features are specifically speech features of 1024 channels.

[0027] See also Figure 2 As shown in the figure, it is a schematic diagram of the binary classification model of speech frames and noise frames for training the target model. The k value in the figure is taken as 2. The binary classification model includes three temporal convolution groups, and each temporal convolution group includes three convolutions. The convolution parameters padding and dilation increase from bottom to top by an exponential multiple of 2; the convolution is used to perform convolution operations on multi-channel speech features, and each convolution is composed of separable convolution and void convolution, specifically: a 1-dimensional convolution with a convolution kernel number of 3, a convolution kernel size of 1, and a convolution step of 1 is used to reduce channel data; a group convolution with a convolution kernel number of 3, a convolution kernel size of 3, a padding parameter of 2 to the k-1th power, a dilation parameter of 2 to the k-1th power, and a group convolution with a parameter of 3 groups is used for convolution operations, and a temporal convolution group includes three convolutions; a 1-dimensional convolution with a convolution kernel number of 1024, a convolution kernel size of 1, and a convolution step of 1 is used to restore to the number of input channels.

[0028] The multi-channel speech features (speech features of 1024 channels) obtained above are subjected to the above convolution to obtain speech features of two channels, namely, the first speech feature and the second speech feature.

[0029] When performing convolution operations on multi-dimensional speech features, a convolution kernel size of 7, a convolution step of 10, and a dilation of 10 are used.

[0030] The target model is a lightweight TCN model. In order to compare the TCN model with the LSTM model (long short-term memory model), Figure 2 The temporal convolution in is replaced by the LSTM model. For the specific structure, see Figure 3 shown.

[0031] The comparison results of parameters and computing power between the tcn model and the lstn model are as follows:

[0032] From the comparison results, it can be seen that the TCN model using separable convolution and dilated convolution is much smaller than the corresponding LSTM model in terms of model parameters and computing power.

[0033] The present invention determines the data type of the frame data, that is, only needs to mark the frame data whether it contains speech, and does not need to extract pure speech from the noisy speech for marking, thereby simplifying the preparation of label data; at the same time, a lightweight TCN model and Kalman filtering method are adopted, and the overall calculation amount is small, which can be applicable to low-computing power narrowband communication, and the noise can be filtered out in real time, and has a wide range of applications and strong adaptability, thereby achieving real-time noise reduction of speech even when running on a low-performance DSP, solving the problem in the prior art that the large amount of calculation makes it impossible to meet the real-time noise reduction of call speech.

[0034] In addition, the deep learning model TCN is used to replace the manually extracted feature classification, which improves the classification and generalization effects of speech frames and noise frames. The TCN model is used instead of the RNN model, which not only retains the sequence characteristics of the speech signal, but also speeds up the processing of the original speech signal due to the small amount of calculation.

[0035] S3: Determine the noise variance according to the noise frame, including: The noise spectrum of the noise frame is estimated through MCRA which combines recursive averaging and minimum tracking. The noise spectrum is a frequency domain signal. Then the noise spectrum is inverse Fourier transformed to transform the frequency domain signal into a time domain signal to obtain the time domain signal of the noise frame. Finally, the noise variance is determined according to the time domain signal, that is, the noise variance is obtained by taking the mean and variance of the time domain signal.

[0036] S4: Construct the Kalman equation based on the speech frame, including: First, the LPC coefficients are determined by the lattice method. The lattice method is based on the lattice filter structure in digital signal processing and solves the LPC coefficients in a recursive manner.

[0037] Then, according to the LPC coefficient, the estimated value of the speech frame is determined. The calculation formula of the estimated value is as follows:

[0038] express The estimated value of the speech signal at time , represents the estimated value of the speech signal at time i, Indicates the corresponding LPC coefficient.

[0039] Finally, the actual value of the speech frame is determined, and the gain is determined based on the estimated value and the actual value; the speech frame includes speech signals at multiple moments, and the actual value is the average value of multiple speech signals in the speech frame. The actual value is divided by the estimated value, and the result is the gain.

[0040] After the LPC coefficient and gain of the speech frame are determined, the Kalman equation is constructed according to the LPC coefficient and gain, and its expression is as follows:

[0041]

[0042]

[0043]

[0044]

[0045]

[0046] in, is the system state at time k, that is, the speech signal value of the speech frame at time k; The state vector constructed from the P speech signal values ​​before k time, is the state transfer matrix composed of LPC coefficients; The input matrix is ​​composed of gains; is the observation matrix; is the input vector; for noise; is the actual speech signal at time k; is the LPC coefficient; For gain, is the order of the linear differential equation adopted.

[0047] S5: According to the noise variance and Kalman equation, the noise of the speech frame is removed to obtain a noise-reduced speech frame, including: The speech frame is predicted by the Kalman equation constructed as above to obtain the predicted speech signals at multiple moments of the speech frame, and the noise variance obtained as above is used to adjust the measurement covariance matrix of the Kalman filter. Specifically, the noise variance is used as the measurement covariance, and the variance values ​​of the noise frame at multiple moments are used as the variance values ​​at multiple moments during measurement, so as to adjust the measurement covariance matrix.

[0048] The noise of the speech frame is removed, and the calculation formula for obtaining the noise-reduced speech frame is as follows:

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] in, is the state transfer matrix, is the observation matrix, is the identity matrix, is the covariance matrix of the system state at time k, is the noise covariance matrix, is the measurement covariance matrix, is the actual speech signal at time k, is the Kalman gain matrix, To predict the speech signal vector, is the predicted speech signal at time k, is the noise reduction speech signal vector, is the denoised speech signal at time k.

[0055] According to the above formula, the noise variance is used as the measurement covariance matrix R to obtain the target Kalman gain , according to the target Kalman gain at the first moment , the predicted speech signal at the first moment and the actual speech signal at the first moment , we can get the denoised speech signal after denoising at the first moment , and finally update the covariance matrix according to the target Kalman gain , and then repeat the above steps until all the actual speech signals at multiple moments in the speech frame are denoised, and the denoised speech signals at multiple moments constitute a denoised speech signal vector, that is, the denoised speech signal vector finally obtained is the denoised speech frame after the noise of the speech frame is removed.

[0056] The estimated noise spectrum characteristics (noise variance) are used to adjust the measurement covariance matrix in real time, so that the Kalman filter that ultimately reduces the noise of the real-time speech signal has the ability to adapt to the noise, thereby effectively removing the non-stationary noise.

[0057] The present invention also provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the above-mentioned noise removal method.

[0058] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0059] The memory can be used to store the computer program or module, and the processor realizes various functions of the noise removal method by running or executing the computer program or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0060] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A low-computing-power narrowband call noise removal method, characterized in that: include: Get frame data; Determine the data type corresponding to the framed data by using a target model, wherein the data type includes a speech frame and a noise frame; Determining a noise variance according to the noise frame; Constructing a Kalman equation according to the speech frame; According to the noise variance and the Kalman equation, the noise of the speech frame is removed to obtain a noise-reduced speech frame.

2. A low-computing-power narrowband call noise removal method according to claim 1, characterized in that: The step of obtaining the frame data includes: Obtaining the original speech signal; The original speech signal is subjected to frame processing, windowing processing and Fourier transform to obtain the frame data.

3. The low-computing-power narrowband call noise removal method according to claim 1, characterized in that: The determining the data type corresponding to the frame data by using the target model includes: Determining the target probability that the framed data is the speech frame by the target model; When the target probability is greater than or equal to a preset threshold, the frame data is the speech frame; When the target probability is less than the preset threshold, the frame data is the noise frame.

4. The low-computing-power narrowband call noise removal method according to claim 1, characterized in that: The determining, by the target model, a target probability that the framed data is the speech frame comprises: Performing a convolution operation on the frame data to obtain multi-channel speech features; Performing a convolution operation on the multi-channel speech feature to obtain a first speech feature and a second speech feature; Performing superposition and dimensionality reduction operations on the first speech feature and the second speech feature to obtain a multi-dimensional speech feature; Performing a convolution operation on the multi-dimensional speech feature to obtain a target speech feature; The cross entropy of the target speech feature is taken to obtain the target probability.

5. A low-computing-power narrowband call noise removal method according to claim 4, characterized in that: The step of determining the noise variance according to the noise frame comprises: Estimating the noise spectrum of the noise frame by recursive averaging and minimum value tracking; Performing inverse Fourier transform on the noise spectrum to obtain a time domain signal of the noise frame; The noise variance is determined according to the time domain signal.

6. A low-computing-power narrowband call noise removal method according to claim 4, characterized in that: The step of constructing a Kalman equation according to the speech frame comprises: Determining LPC coefficients and gains of the speech frame; The Kalman equation is constructed according to the LPC coefficient and the gain.

7. A low-computing-power narrowband call noise removal method according to claim 6, characterized in that: The determining of the LPC coefficient and gain of the speech frame comprises: Determining the LPC coefficients by a lattice method; Determining an estimated value of the speech frame according to the LPC coefficient; Determining an actual value of the speech frame; The gain is determined according to the estimated value and the actual value.

8. The low-computing-power narrowband call noise removal method according to claim 7, characterized in that: The speech frame includes actual speech signals at multiple moments; The step of removing the noise of the speech frame according to the noise variance and the Kalman equation to obtain a noise-reduced speech frame comprises: Predicting the speech frame by using the Kalman equation to obtain predicted speech signals at multiple moments of the speech frame; determining a target Kalman gain according to the noise variance; According to the predicted speech signal and the target Kalman gain, the actual speech signal is denoised to obtain a denoised speech signal; Determining a covariance matrix according to the target Kalman gain; Repeat the above steps until all the multiple actual speech signals of the speech frame are denoised, and the multiple denoised speech signals constitute the denoised speech frame.

9. A low-computing-power narrowband call noise removal method according to claim 8, characterized in that: The target model is a TCN model.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the noise removal method as described in any one of claims 1 to 9.

Images