Bluetooth receiving end noise reduction method and device based on non-negative matrix factorization, medium and equipment
By combining non-negative matrix decomposition and deep learning on the Bluetooth receiver, using MDCT spectral coefficients and pre-trained neural networks, the insufficient sound quality and high complexity of speech noise reduction in Bluetooth audio technology are solved, and low power consumption and high efficiency are achieved. Voice noise reduction effect.
Patent Information
- Application Number
- CN202411398531.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-01-07
AI Technical Summary
The existing Bluetooth audio technology has problems such as insufficient sound quality, high memory footprint, high computing complexity and high patent fees in terms of voice noise reduction, especially in embedded systems, which are difficult to deploy deep learning models.
Using a method based on non-negative matrix decomposition and deep learning, subband energy calculation is performed through MDCT spectral coefficients on the Bluetooth receiver side, non-negative matrix decomposition is performed to generate noise reduction gain, and the pre-trained deep neural network is used to correct the gain to achieve speech reduction Noise avoids the problems of large matrix size and complex network model of traditional methods.
While ensuring sound quality, the matrix size and computing volume are reduced, the network structure of the deep learning model is simplified, and it is suitable for classic Bluetooth and low-power Bluetooth, reducing storage and computing needs.
Smart Images
Figure CN119274572A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of Bluetooth audio technology, and in particular, relates to a Bluetooth receiver speech noise reduction method, device, storage medium and equipment based on non-negative matrix decomposition and deep learning. Background Art
[0002] The current mainstream Bluetooth audio encoders include: SBC: mandatory requirement of A2DP protocol, the most widely used, and must be supported by all Bluetooth audio devices, but the sound quality is average; AAC-LC: good sound quality and widely used, many mainstream mobile phones support it, but compared with SBC, it occupies a large amount of memory and has high computational complexity. Many Bluetooth devices are based on embedded platforms with limited battery capacity, poor processor computing power and limited memory, and its patent fees are high; aptX series: good sound quality, but high bit rate, aptX requires a bit rate of 384kbps, while aptX -HD has a bit rate of 576kbps, and it is a unique technology of Qualcomm and is relatively closed; LDAC: The sound quality is good, but the bit rate is also very high, which are 330kbps, 660kbps and 990kbps respectively. Since the wireless environment of Bluetooth devices is particularly complex, it is difficult to stably support such a high bit rate, and it is a unique technology of Sony and is also very closed; LHDC: The sound quality is good, but the bit rate is also very high, typically including 400kbps, 600kbps and 900kbps. Such a high bit rate puts high demands on the baseband / RF design of Bluetooth. Based on the above reasons, the Bluetooth International Alliance Bluetooth Sig and many manufacturers launched LC3, which is mainly for low-power Bluetooth and can also be used for classic Bluetooth. It has the advantages of lower latency, higher sound quality and coding gain, and no patent fees in the Bluetooth field, and has attracted the attention of many manufacturers.
[0003] In many Bluetooth applications, such as Bluetooth calls, Bluetooth microphones, and recordings, noise reduction is required.
[0004] Nonnegative Matrix Factorization (NMF) makes all matrix components non-negative and achieves nonlinear dimensionality reduction. NMF has gradually become one of the most popular multidimensional data processing tools in research fields such as signal processing, biomedical engineering, pattern recognition, computer vision and image engineering.
[0005] Non-negative matrix factorization has certain applications in speech noise reduction, but its performance is average at low signal-to-noise ratios and can cause distortion.
[0006] Deep learning has received widespread attention in speech noise reduction. Currently, many mature algorithms have been successfully deployed on servers and PCs with good results. However, due to the complexity of the network model and the large memory space occupied by the weights, it is difficult to deploy in embedded systems. It usually requires the assistance of NPU. Many Bluetooth-related devices are based on embedded systems, and few of them are equipped with NPU. Summary of the invention
[0007] In response to the above-mentioned technical problems existing in the prior art, the present application provides a Bluetooth receiving end speech noise reduction method, device, storage medium and equipment based on non-negative matrix decomposition and deep learning. At the Bluetooth receiving end, based on the MDCT spectral coefficients, the noise reduction gain is generated by performing non-negative matrix decomposition on the sub-band, and then the noise reduction gain is corrected based on deep learning, thereby achieving speech noise reduction. It can not only ensure the sound quality, but also reduce the size of the matrix, avoiding the shortcomings of non-negative matrix decomposition, and avoiding the use of overly complex network models for deep learning.
[0008] In order to achieve the above-mentioned purpose, the first technical solution adopted in the present application is: to provide a Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning, including: at the Bluetooth receiver, input a mono noisy speech code stream, and perform decoding to transform domain noise shaping to obtain MDCT spectrum coefficients; calculate the subband energy according to the MDCT spectrum coefficients, and construct a subband energy observation matrix; perform non-negative matrix decomposition on the subband energy observation matrix, and calculate the speech subband energy value and the noise subband energy value; calculate the first subband noise reduction gain according to the speech subband energy value and the noise subband energy value; input the first subband noise reduction gain into a pre-trained deep neural network, output the second subband noise reduction gain, and perform interpolation on the second subband noise reduction gain to obtain all frequency bin gains; obtain the noise reduction spectrum coefficients according to the frequency bin gains and the MDCT spectrum coefficients; and continue to perform the decoding process on the noise reduction spectrum coefficients to output the noise reduction speech PCM data.
[0009] The second technical solution adopted in the present application is: to provide a Bluetooth receiver speech noise reduction device based on non-negative matrix decomposition and deep learning, including: a module for inputting a mono noisy speech code stream at the Bluetooth receiver, and performing decoding to transform domain noise shaping to obtain MDCT spectrum coefficients; a module for calculating subband energy according to MDCT spectrum coefficients, and constructing a subband energy observation matrix; a module for performing non-negative matrix decomposition on the subband energy observation matrix, and calculating speech subband energy values and noise subband energy values; a module for calculating the first subband noise reduction gain according to the speech subband energy values and noise subband energy values; a module for inputting the first subband noise reduction gain into a pre-trained deep neural network, outputting the second subband noise reduction gain, and performing interpolation on the second subband noise reduction gain to obtain all frequency bin gains; a module for obtaining noise reduction spectrum coefficients according to the frequency bin gains and MDCT spectrum coefficients; a module for continuing to perform the decoding process on the noise reduction spectrum coefficients and outputting noise reduction speech PCM data.
[0010] The third technical solution adopted in the present application is: providing a computer-readable storage medium storing computer instructions, wherein the computer instructions are operated to execute the Bluetooth receiving end speech noise reduction method based on non-negative matrix decomposition and deep learning in solution one.
[0011] The fourth technical solution adopted in this application is: to provide a computer device, which includes a processor and a memory, the memory stores computer instructions, wherein the processor operates the computer instructions to execute the Bluetooth receiving end speech noise reduction method based on non-negative matrix decomposition and deep learning in solution one.
[0012] The beneficial effects that can be achieved by the technical solution of the present application are: the technical solution of the present application can be applied to both classic Bluetooth (BR, EDR) and low-power Bluetooth (LE, LE Audio). At the Bluetooth receiving end, based on the MDCT spectrum coefficient, the noise reduction gain is generated by performing non-negative matrix decomposition on the subband, and then the noise reduction gain is corrected based on deep learning, thereby realizing speech noise reduction, ensuring the sound quality, avoiding the shortcomings of non-negative matrix decomposition, and avoiding the use of overly complex network models for deep learning. Based on the processing of discrete cosine transform, the phase spectrum is not required during the inverse transform. Compared with the traditional Fourier transform that requires the phase spectrum, it avoids the damage to the reconstructed speech due to the inability to obtain the phase spectrum of pure speech and only using the phase spectrum of noisy speech; performing non-negative matrix decomposition on the subband, compared with the traditional amplitude spectrum, the size of the matrix is greatly reduced, and the amount of calculation and storage are correspondingly reduced; the deep learning model is mainly used to correct the noise reduction gain of the non-negative matrix decomposition. Compared with the traditional end-to-end deep learning noise reduction model based on the amplitude spectrum, the network structure is simpler, and the amount of calculation and storage are correspondingly reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is a flowchart of a specific implementation method of the Bluetooth receiver voice noise reduction method based on non-negative matrix decomposition and deep learning of the present application;
[0014] Figure 2 It is a schematic diagram of a specific example of speech noise reduction at a Bluetooth receiver based on non-negative matrix decomposition and deep learning in the present application;
[0015] Figure 3 It is a schematic diagram of a specific implementation of the Bluetooth receiving end speech noise reduction device based on non-negative matrix decomposition and deep learning of the present application. DETAILED DESCRIPTION
[0016] The preferred embodiments of the present application are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present application can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present application.
[0017] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0018] The specific implementation of the present application is described in detail below using the 48 kHz sampling rate as an example. The principles of other sampling rates are similar to those of the 48 kHz sampling rate and will not be repeated here.
[0019] Figure 1 It is a flowchart of a specific implementation method of the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning in this application.
[0020] exist Figure 1 In a specific embodiment shown, the Bluetooth receiving end speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S101, at the Bluetooth receiving end, a mono noisy speech bit stream is input, and decoding is performed to transform domain noise shaping to obtain MDCT spectrum coefficients.
[0021] In this specific implementation, a monophonic noisy speech bitstream is inputted at the Bluetooth receiving end, decoded to transform domain noise shaping according to a standard process, and MDCT spectrum coefficients are outputted.
[0022] In a specific example of the present application, taking 48kHz sampling rate and 10ms frame length as an example, the MDCT spectrum coefficient is:
[0023] Among them, N F =480
[0024] exist Figure 1 In a specific implementation shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S102, calculating the subband energy according to the MDCT spectrum coefficients and constructing a subband energy observation matrix.
[0025] In a specific embodiment of the present application, subband energy is calculated based on MDCT spectrum coefficients, including: dividing the MDCT spectrum coefficients into subbands to obtain a frequency bin index table corresponding to each subband; and obtaining subband energy based on the MDCT spectrum coefficients and the frequency bin index table.
[0026] Specifically, the MDCT spectrum coefficients are first divided into subbands, and the above MDCT spectrum coefficients are divided into 64 subbands, and the subband sequence number is recorded as b=0, 1, 2, ..., 63. This application takes the division of the MDCT spectrum coefficients into 64 subbands as an example. In practical applications, the subband size can be flexibly configured according to the needs to take into account the requirements of noise reduction performance and computational complexity, and then the frequency bin index table corresponding to each subband is obtained, as follows:
[0027] Index
[64] =[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,24,26,28,30,32,34,36,39,42,45,48,51,55,59,63, 67,71,76,81,86,92,98,105,112,119,127,135,144,154,164,175,186,198,211,225,240,256,273,291,310,330,352,375,400]
[0028] Then, the subband energy is calculated based on the MDCT spectrum coefficients and the frequency bin index table:
[0029]
[0030] After obtaining the subband energy, construct the subband energy observation matrix V noisy .
[0031] exist Figure 1 In a specific embodiment shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S103, performing non-negative matrix decomposition on the sub-band energy observation matrix, and calculating the speech sub-band energy value and the noise sub-band energy value.
[0032] In a specific embodiment of the present application, non-negative matrix decomposition is performed on the subband energy observation matrix, and the speech subband energy value and the noise subband energy value are calculated, including: fixing the speech basic matrix and the noise basic matrix respectively to perform non-negative matrix decomposition on the subband energy observation matrix to obtain a speech activation matrix and a noise activation matrix; multiplying the speech basic matrix with the speech activation matrix to obtain the speech subband energy value; and multiplying the noise basic matrix with the noise activation matrix to obtain the noise subband energy value.
[0033] Specifically, the fixed speech basic matrix W speech , perform non-negative matrix decomposition on the subband energy observation matrix to obtain the speech activation matrix H′ speech .
[0034] Then, fix the noise basis matrix W noise , perform non-negative matrix factorization on the subband energy observation matrix to obtain the noise activation matrix H′ noise .
[0035] Multiply the speech basis matrix by the speech activation matrix to calculate the speech subband energy value:
[0036] V′ speech (b) = W speech H′ speech
[0037] Multiply the noise basis matrix by the noise activation matrix to calculate the noise subband energy value:
[0038] N′ noise (b) = W noise H′ noise
[0039] In a specific embodiment of the present application, the training process of the speech basic matrix and the noise basic matrix includes: selecting clean speech and noise as training sets; performing frame segmentation, windowing and low-delay improved discrete cosine transform on the clean speech and noise respectively to obtain the clean speech spectral coefficients and the noise spectral coefficients; calculating the corresponding subband energies according to the clean speech spectral coefficients and the noise spectral coefficients respectively, and constructing the clean speech subband energy observation matrix and the noise subband energy observation matrix accordingly; and performing non-negative matrix decomposition on the clean speech subband energy observation matrix and the noise subband energy observation matrix respectively to obtain the speech basic matrix and the noise basic matrix.
[0040] Specifically, the speech basic matrix W is speech and the noise basis matrix W noise The training process is briefly described as follows:
[0041] (1) Selecting clean speech and noise as training sets (the following takes the training process of clean speech as an example, the training process of noise is similar and will not be repeated);
[0042] (2) performing framing, windowing and low-delay improved discrete cosine transform on the clean speech to obtain spectral coefficients;
[0043] (3) Calculate the subband energy and construct the subband energy observation matrix V;
[0044] (4) Perform non-negative matrix decomposition on the subband energy observation matrix V to obtain the basic matrix W (i.e., the speech basic matrix W speech ) and the activation matrix H.
[0045] For a non-negative matrix V, non-negative matrix decomposition is to find the optimal solution so that the product of W and H obtained by decomposition is an approximation of V. The steps of non-negative matrix decomposition are briefly described as follows:
[0046] (1) Construct the observation matrix V KL , there are K rows and L columns, each column is the amplitude spectrum of a frame, and there are L columns in total.
[0047] (2) Initialize the basic matrix W KM .
[0048] (3) Initialize the activation matrix H ML In a typical scenario, the above parameters can be K=480, M=240, and L=3~5.
[0049] (4) Iterative solution based on KL divergence criterion is briefly described as follows:
[0050]
[0051] in represents element-wise multiplication, T represents the transpose operation of the matrix, and I represents the identity matrix.
[0052] During the iteration process, the KL divergence of V and WH is calculated. When it is less than the preset threshold, the process stops. In order to avoid long calculation time caused by abnormal situations, the number of iterations can also be preset. When the preset number is reached, the process stops, thereby obtaining the basic matrix and the activation matrix.
[0053] exist Figure 1In a specific embodiment shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S104, calculating the first sub-band noise reduction gain according to the speech sub-band energy value and the noise sub-band energy value.
[0054] In this specific implementation, the first sub-band noise reduction gain, that is, the NMF sub-band noise reduction gain, is calculated:
[0055]
[0056] exist Figure 1 In a specific embodiment shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S105, inputting the first subband noise reduction gain into a pre-trained deep neural network, outputting the second subband noise reduction gain, and performing interpolation on the second subband noise reduction gain to obtain all frequency bin gains.
[0057] In this specific implementation, the first sub-band noise reduction gain is input into a pre-trained deep neural network, and the pre-trained deep neural network outputs a second sub-band noise reduction gain That is, the NN subband noise reduction gain, and then the second subband noise reduction gain is interpolated to obtain all frequency bin gains.
[0058] The following is a brief description of the training process of the pre-trained deep neural network:
[0059] First, obtain a certain amount of pure speech and noise, and then use the above speech basic matrix W speech and the noise basis matrix W noise The training process obtains the speech basic matrix W speech and the noise basis matrix W noise .
[0060] Then, the clean speech and the noise are mixed to obtain the noisy speech, and then feature extraction is performed on the clean speech and the noisy speech. Specifically, the clean speech and the noisy speech are first subjected to low-delay improved discrete cosine transform to output the spectral coefficients corresponding to each other, and then the sub-band energy corresponding to the clean speech and the noisy speech is calculated (the specific calculation method is the same as above, which is omitted here and will not be repeated), wherein, the clean speech: x clean (k), pure speech spectrum coefficient: X clean (k), Energy of pure speech subband: subband,clean (b); Noisy speech x noisy (k), noisy speech spectrum coefficient X noisy (k), noisy speech with energy subband,noisy (b).
[0061] After obtaining the clean speech subband energy and the noisy speech subband energy through calculation, calculate the ideal subband gain:
[0062]
[0063] The noisy speech is then input into the standard LC3 encoder to generate a bitstream, which is then input into the Bluetooth receiver to perform partial decoding to transform domain noise shaping, and then calculate the NMF subband gain based on non-negative matrix decomposition. (The specific calculation method is the same as above, omitted here and will not be repeated).
[0064] Then the NMF subband gain obtained based on non-negative matrix decomposition is input into the deep neural network, and the deep neural network is trained with the ideal subband gain as the target, and the NN subband gain is output. When the loss level reaches the expected level, the deep neural network is frozen to obtain the pre-trained deep neural network.
[0065] Among them, this application does not restrict the selection of deep neural network. Considering the context-correlation characteristics of speech frames, recurrent neural network (RNN) is preferred.
[0066] When doing backpropagation, the loss function used is defined as:
[0067]
[0068] In a specific embodiment of the present application, interpolation is performed on the second sub-band noise reduction gain to obtain all frequency bin gains, including: performing a predetermined operation on the second sub-band noise reduction gain and a waveform symmetrical about the sub-band center frequency bin to obtain all frequency bin gains.
[0069] Specifically, all frequency bin gains are:
[0070]
[0071] Among them, Weight b (k) is a waveform symmetrical about the sub-band center frequency bin, which satisfies the following conditions:
[0072]
[0073] exist Figure 1 In a specific implementation shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S106, obtaining noise reduction spectrum coefficients according to frequency bin gains and MDCT spectrum coefficients.
[0074] In a specific embodiment of the present application, the noise reduction spectrum coefficients are obtained according to the frequency bin gains and the MDCT spectrum coefficients, including: multiplying the frequency bin gains and the MDCT spectrum coefficients to obtain the noise reduction spectrum coefficients.
[0075] Specifically, the frequency bin gain is applied to the MDCT spectrum coefficients to obtain the noise reduction spectrum coefficients:
[0076]
[0077] exist Figure 1 In a specific implementation shown, the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning of the present application includes process S107, continuing to perform a decoding process on the noise reduction spectrum coefficients, and outputting noise reduction speech PCM data.
[0078] In this specific implementation, after the noise reduction spectrum coefficients are obtained, the subsequent decoding process is continued to be completed, and finally the noise-reduced speech PCM data is output.
[0079] In a specific embodiment of the present application, the decoding process is continued on the denoised spectral coefficients, including: continuing to perform the decoding process of low-delay improved inverse discrete cosine transform, overlap-addition and long-term post-filter on the denoised spectral coefficients.
[0080] Figure 2 It is a schematic diagram of a specific example of speech noise reduction at a Bluetooth receiver based on non-negative matrix decomposition and deep learning in this application.
[0081] like Figure 2 A specific example shown, the speech denoising process of the Bluetooth receiver based on non-negative matrix decomposition and deep learning of the present application is: first, at the Bluetooth receiver, input the noisy code stream, perform code stream parsing, arithmetic and residual decoding, noise filling, global gain, time domain noise shaping decoding to transform domain noise shaping decoding in sequence according to the standard decoding process, output MDCT spectrum coefficients, then calculate the subband energy and form a subband energy observation matrix, perform non-negative matrix decomposition to obtain the activation matrix of speech and noise, calculate the subband energy of speech and noise, then calculate the subband gain, and input the subband gain into the trained deep neural network, output the new subband gain, then interpolate the new subband gain to obtain all frequency bin gains, then apply the frequency bin gain to the MDCT spectrum coefficients, thereby obtaining the denoised spectrum coefficients, and then continue to perform the low-delay improved discrete cosine inverse transform, overlap-add and long-term post-filter decoding process on the denoised spectrum coefficients, and finally output the denoised speech data (PCM). This example combines non-negative matrix decomposition and deep learning at the Bluetooth receiver to achieve speech denoising, and its advantages are:
[0082] (1) Generate noise reduction gain based on non-negative matrix decomposition and then correct the gain based on deep learning, which avoids the shortcomings of non-negative matrix decomposition and avoids using overly complex network models for deep learning;
[0083] (2) Performing non-negative matrix factorization on the subbands, compared with the traditional amplitude spectrum, the size of the matrix is greatly reduced, and the amount of calculation and storage are also reduced accordingly;
[0084] (3) The deep learning model is mainly used to correct the denoising gain of non-negative matrix decomposition. Compared with the traditional end-to-end deep learning denoising model based on amplitude spectrum, the network structure is simpler, and the amount of computation and storage are reduced accordingly;
[0085] (4) Inverse Fourier transform requires a phase spectrum, but the phase spectrum of pure speech cannot be obtained, and only the phase spectrum of noisy speech can be used. This application is based on discrete cosine transform processing, and the phase spectrum is not required during inverse transform, thus avoiding the damage of the noisy speech phase spectrum to the reconstructed speech.
[0086] In the Bluetooth receiver speech denoising method based on non-negative matrix decomposition and deep learning of the present application, at the Bluetooth receiver, based on the MDCT spectrum coefficient, the noise reduction gain is generated by performing non-negative matrix decomposition on the subband, and then the noise reduction gain is corrected based on deep learning, so as to achieve speech denoising, ensure the sound quality, avoid the shortcomings of non-negative matrix decomposition, and avoid using too complex network models for deep learning. Based on the processing of discrete cosine transform, the phase spectrum is not required during the inverse transform, compared with the traditional Fourier transform that requires the phase spectrum, it avoids the damage to the reconstructed speech due to the inability to obtain the phase spectrum of pure speech and only using the phase spectrum of noisy speech; performing non-negative matrix decomposition on the subband, compared with the traditional amplitude spectrum, the size of the matrix is greatly reduced, and the amount of calculation and storage are correspondingly reduced; the deep learning model is mainly used to correct the noise reduction gain of the non-negative matrix decomposition, and compared with the traditional end-to-end deep learning noise reduction model based on the amplitude spectrum, the network structure is simpler, and the amount of calculation and storage are correspondingly reduced.
[0087] Figure 3 It is a schematic diagram of a specific implementation of the Bluetooth receiving end speech noise reduction device based on non-negative matrix decomposition and deep learning of the present application.
[0088] exist Figure 3In a specific embodiment shown, the Bluetooth receiving end speech noise reduction device based on non-negative matrix decomposition and deep learning of the present application includes: a module 301 for inputting a mono noisy speech code stream at the Bluetooth receiving end, and performing decoding to transform domain noise shaping to obtain MDCT spectrum coefficients; a module 302 for calculating subband energy according to MDCT spectrum coefficients and constructing a subband energy observation matrix; a module 303 for performing non-negative matrix decomposition on the subband energy observation matrix, and calculating the speech subband energy value and the noise subband energy value; a module 304 for calculating the first subband noise reduction gain according to the speech subband energy value and the noise subband energy value; a module 305 for inputting the first subband noise reduction gain into a pre-trained deep neural network, outputting the second subband noise reduction gain, and performing interpolation on the second subband noise reduction gain to obtain all frequency bin gains; a module 306 for obtaining noise reduction spectrum coefficients according to the frequency bin gains and MDCT spectrum coefficients; and a module 307 for continuing to perform the decoding process on the noise reduction spectrum coefficients and outputting noise reduction speech PCM data.
[0089] In a specific embodiment of the present application, subband energy is calculated based on MDCT spectrum coefficients, including: dividing the MDCT spectrum coefficients into subbands to obtain a frequency bin index table corresponding to each subband; and obtaining subband energy based on the MDCT spectrum coefficients and the frequency bin index table.
[0090] In a specific embodiment of the present application, non-negative matrix decomposition is performed on the subband energy observation matrix, and the speech subband energy value and the noise subband energy value are calculated, including: fixing the speech basic matrix and the noise basic matrix respectively to perform non-negative matrix decomposition on the subband energy observation matrix to obtain a speech activation matrix and a noise activation matrix; multiplying the speech basic matrix with the speech activation matrix to obtain the speech subband energy value; and multiplying the noise basic matrix with the noise activation matrix to obtain the noise subband energy value.
[0091] In a specific embodiment of the present application, the training process of the speech basic matrix and the noise basic matrix includes: selecting clean speech and noise as training sets; performing frame segmentation, windowing and low-delay improved discrete cosine transform on the clean speech and noise respectively to obtain the clean speech spectral coefficients and the noise spectral coefficients; calculating the corresponding subband energies according to the clean speech spectral coefficients and the noise spectral coefficients respectively, and constructing the clean speech subband energy observation matrix and the noise subband energy observation matrix accordingly; and performing non-negative matrix decomposition on the clean speech subband energy observation matrix and the noise subband energy observation matrix respectively to obtain the speech basic matrix and the noise basic matrix.
[0092] In a specific embodiment of the present application, interpolation is performed on the second sub-band noise reduction gain to obtain all frequency bin gains, including: performing a predetermined operation on the second sub-band noise reduction gain and a waveform symmetrical about the sub-band center frequency bin to obtain all frequency bin gains.
[0093] In a specific embodiment of the present application, the noise reduction spectrum coefficients are obtained according to the frequency bin gains and the MDCT spectrum coefficients, including: multiplying the frequency bin gains and the MDCT spectrum coefficients to obtain the noise reduction spectrum coefficients.
[0094] In a specific embodiment of the present application, the decoding process is continued on the denoised spectral coefficients, including: continuing to perform the decoding process of low-delay improved inverse discrete cosine transform, overlap-addition and long-term post-filter on the denoised spectral coefficients.
[0095] In the Bluetooth receiver speech noise reduction device based on non-negative matrix decomposition and deep learning of the present application, at the Bluetooth receiver, based on the MDCT spectrum coefficient, the noise reduction gain is generated by performing non-negative matrix decomposition on the subband, and then the noise reduction gain is corrected based on deep learning, so as to achieve speech noise reduction, ensure the sound quality, avoid the shortcomings of non-negative matrix decomposition, and avoid the use of overly complex network models for deep learning. Based on the processing of discrete cosine transform, the phase spectrum is not required during the inverse transform. Compared with the traditional Fourier transform that requires the phase spectrum, it avoids the damage to the reconstructed speech due to the inability to obtain the phase spectrum of pure speech and only using the phase spectrum of noisy speech; performing non-negative matrix decomposition on the subband, compared with the traditional amplitude spectrum, the size of the matrix is greatly reduced, and the amount of calculation and storage are correspondingly reduced; the deep learning model is mainly used to correct the noise reduction gain of the non-negative matrix decomposition. Compared with the traditional end-to-end deep learning noise reduction model based on the amplitude spectrum, the network structure is simpler, and the amount of calculation and storage are correspondingly reduced.
[0096] In a specific embodiment of the present application, a computer-readable storage medium stores computer instructions, which are operated to execute the Bluetooth receiver voice noise reduction method based on non-negative matrix decomposition and deep learning described in any embodiment. The storage medium can be directly in hardware, in a software module executed by a processor, or in a combination of the two.
[0097] The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium.
[0098] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In an alternative, the storage medium may be integrated with the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.
[0099] In a specific embodiment of the present application, a computer device includes a processor and a memory, wherein the memory stores computer instructions, wherein the processor operates the computer instructions to execute the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning described in any embodiment.
[0100] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0102] The above description is only an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structural transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning, characterized in that: include: At the Bluetooth receiver, a mono noisy speech bitstream is input and decoded to transform domain noise shaping to obtain MDCT spectrum coefficients; Calculating subband energy according to the MDCT spectrum coefficients and constructing a subband energy observation matrix; Performing non-negative matrix decomposition on the sub-band energy observation matrix, and calculating speech sub-band energy values and noise sub-band energy values; Calculating a first sub-band noise reduction gain according to the speech sub-band energy value and the noise sub-band energy value; Inputting the first sub-band noise reduction gain into a pre-trained deep neural network, outputting a second sub-band noise reduction gain, and performing interpolation on the second sub-band noise reduction gain to obtain all frequency bin gains; Obtaining noise reduction spectrum coefficients according to the frequency bin gain and the MDCT spectrum coefficients; as well as The decoding process is continued on the noise reduction spectrum coefficients to output noise reduction speech PCM data.
2. The method for reducing the noise of a Bluetooth receiver voice based on non-negative matrix decomposition and deep learning according to claim 1, characterized in that: The calculating sub-band energy according to the MDCT spectrum coefficients comprises: Dividing the MDCT spectrum coefficients into sub-bands to obtain a frequency bin index table corresponding to each of the sub-bands; and The subband energy is obtained according to the MDCT spectrum coefficients and the frequency bin index table.
3. The method for reducing the noise of a Bluetooth receiving terminal voice based on non-negative matrix decomposition and deep learning according to claim 1, characterized in that: The performing non-negative matrix decomposition on the sub-band energy observation matrix and calculating the speech sub-band energy value and the noise sub-band energy value comprises: The speech basic matrix and the noise basic matrix are fixed respectively, and non-negative matrix decomposition is performed on the sub-band energy observation matrix to obtain a speech activation matrix and a noise activation matrix. Multiplying the speech basic matrix by the speech activation matrix to obtain the speech sub-band energy value; and The noise basic matrix is multiplied by the noise activation matrix to obtain the noise subband energy value.
4. The method for reducing noise at a Bluetooth receiving end based on non-negative matrix decomposition and deep learning according to claim 1, characterized in that: The step of obtaining the noise reduction spectrum coefficient according to the frequency bin gain and the MDCT spectrum coefficient comprises: The frequency bin gain is multiplied by the MDCT spectrum coefficient to obtain the noise reduction spectrum coefficient.
5. The method for reducing noise at a Bluetooth receiving end based on non-negative matrix decomposition and deep learning according to claim 2, characterized in that: The interpolating the second sub-band noise reduction gain to obtain all frequency bin gains includes: The second sub-band noise reduction gain is subjected to a predetermined operation with a waveform symmetrical with respect to the sub-band center frequency bin to obtain all the frequency bin gains.
6. The method for reducing noise at a Bluetooth receiving end based on non-negative matrix decomposition and deep learning according to claim 3, characterized in that: The training process of the speech basic matrix and the noise basic matrix includes: Select pure speech and noise as training sets; Performing framing, windowing and low-delay improved discrete cosine transform on the clean speech and the noise respectively to obtain clean speech spectrum coefficients and noise spectrum coefficients; Calculating corresponding subband energies according to the clean speech spectrum coefficients and the noise spectrum coefficients respectively, and constructing a clean speech subband energy observation matrix and a noise subband energy observation matrix accordingly; and Non-negative matrix decomposition is performed on the clean speech subband energy observation matrix and the noise subband energy observation matrix respectively to obtain the speech basic matrix and the noise basic matrix.
7. The method for reducing the noise of a Bluetooth receiver voice based on non-negative matrix decomposition and deep learning according to claim 1, characterized in that: The step of continuing to perform a decoding process on the noise reduction spectrum coefficients comprises: The decoding process of low-delay inverse modified discrete cosine transform, overlap-add and long-term post-filter is continued to be performed on the denoised spectral coefficients.
8. A Bluetooth receiver speech noise reduction device based on non-negative matrix decomposition and deep learning, characterized in that: include: A module for inputting a mono noisy speech code stream at a Bluetooth receiver, performing decoding to transform domain noise shaping, and obtaining MDCT spectrum coefficients; A module for calculating sub-band energy according to the MDCT spectrum coefficients and constructing a sub-band energy observation matrix; A module for performing non-negative matrix decomposition on the sub-band energy observation matrix and calculating a speech sub-band energy value and a noise sub-band energy value; A module for calculating a first sub-band noise reduction gain according to the speech sub-band energy value and the noise sub-band energy value; A module for inputting the first sub-band noise reduction gain into a pre-trained deep neural network, outputting a second sub-band noise reduction gain, and performing interpolation on the second sub-band noise reduction gain to obtain all frequency bin gains; A module for obtaining noise reduction spectrum coefficients according to the frequency bin gains and the MDCT spectrum coefficients; A module for continuing to perform a decoding process on the noise reduction spectrum coefficients and outputting noise reduction speech PCM data.
9. A computer-readable storage medium storing computer instructions, wherein the computer instructions are operated to execute the Bluetooth receiver speech noise reduction method based on non-negative matrix decomposition and deep learning as described in any one of claims 1-7.
10. A computer device comprising a processor and a memory, wherein the memory stores computer instructions, wherein the processor operates the computer instructions to execute the Bluetooth receiving end speech noise reduction method based on non-negative matrix decomposition and deep learning as described in any one of claims 1-7.