Multidimensional RNN-based speech denoising methods, devices, equipment, and media

Abstract

This invention relates to the field of speech denoising technology and discloses a multi-dimensional RNN-based speech denoising method. It acquires raw speech data with a specified preset storage threshold, removes noisy segments from the raw speech data to obtain denoising speech data, and then performs denoising processing on the denoising speech data based on a preset denoising algorithm to obtain denoised data. Finally, it iteratively trains the RNN algorithm to perform denoising processing on the denoising speech data based on the preset denoising algorithm, obtaining near-true denoised data. This method can reduce a large number of processing steps when denoising similar speech data in subsequent iterations, greatly improving the efficiency and effectiveness of the overall denoising process and enhancing the application effect of the method.

Description

Technical Field

[0001] This invention relates to the field of speech denoising technology, and in particular to speech denoising methods, apparatus, devices and media using multi-dimensional RNNs. Background Technology

[0002] Speech, the material form of language, is the external expression of language and the symbolic system that most directly records human thought processes. It consists of sounds produced by the human vocal organs that possess social significance. Speech data is data about sounds; however, some speech data is prone to noise during recording. To improve the overall quality of speech data, noise reduction processing is necessary.

[0003] Chinese patent disclosure (CN109584895B) discloses a speech denoising method and apparatus. The method includes: sampling a target noisy speech signal according to a sampling frequency in a predetermined format to obtain a sampling point one of the target noisy speech signal; determining a sampling point two of the target denoised speech signal corresponding to the sampling point one of the target noisy speech signal through a speech enhancement model, wherein the speech enhancement model is obtained by machine learning training using multiple sets of training data, and each set of training data includes: the sampling point of the noisy speech signal and the sampling point of the denoised speech signal corresponding to the sampling point of the noisy speech signal; and converting the sampling point two of the target denoised speech signal into the denoised speech of the target speech. This solution addresses the technical problem of low efficiency in speech denoising due to the complexity of current technologies. Current speech denoising methods lack a preliminary processing step, cannot select speech segments, cannot remove invalid parts of the speech data covered by noise, and cannot control the speech length to extract appropriate segments, thus affecting subsequent denoising effects. Furthermore, they do not employ RNN deep learning algorithms, failing to improve the efficiency of subsequent speech denoising processing and resulting in poor application performance. Therefore, a multi-dimensional, improved RNN-based speech denoising method is urgently needed. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problems of existing technologies that cannot select segments of speech, cannot remove invalid parts of speech data covered by noise, cannot control the length of speech to extract appropriate segments, thus affecting the subsequent noise reduction effect, and do not adopt RNN deep learning algorithms, thus failing to improve the efficiency of subsequent speech noise reduction processing.

[0005] The first aspect of this invention provides a multi-dimensional RNN-based speech denoising method, comprising:

[0006] Obtain raw voice data with a specified preset storage threshold;

[0007] Remove the segments containing noise from the original speech data to obtain the noise-free speech data;

[0008] Based on the preset noise reduction algorithm, the noise-reduced speech data is processed to obtain noise-reduced data;

[0009] Based on a pre-set RNN algorithm, the step of denoising the noise-reducing speech data is trained cyclically to obtain near-realistic denoised data.

[0010] Furthermore, in a second implementation of the first aspect of the present invention, S102 includes:

[0011] Enhance the spectrogram of the de-noiseed speech data to obtain enhanced speech data;

[0012] Extract low-level audio features from the enhanced speech data, wherein the low-level audio features include short-time average zero-crossing rate, Mel frequency cepstral coefficients, root mean square value of the spectrum, central moment of the spectrum, monotonicity of the spectrum, bandwidth of the spectrum, and polynomial coefficients of the spectrum.

[0013] The low-level audio features are input into a pre-set training model for training to obtain high-level audio features for audio classification.

[0014] Based on the high-level audio features, the content in the speech data is extracted and the speech data is filtered to obtain noise-removed speech data.

[0015] Furthermore, in a third implementation of the first aspect of the present invention, S102 includes:

[0016] MFCC feature extraction is performed on the speech data to obtain feature data;

[0017] Based on the sparse coding algorithm, the feature data is calculated to obtain sparse coefficients;

[0018] The sparse coefficients are used as features and input into a preset GMM classifier for classification processing to obtain classification features. Based on the classification features, the speech data is classified to obtain de-noiseed speech data.

[0019] Furthermore, in a fourth implementation of the first aspect of the present invention, after S102, the method further includes:

[0020] Power spectrum analysis, slice spectrum analysis, and spectral envelope cepstral analysis were performed on the noise-removed speech data.

[0021] Furthermore, in a fifth implementation of the first aspect of the present invention, S103 includes:

[0022] The noise-removed speech data is filtered and windowed and framed to obtain processed speech data;

[0023] The features extracted from the processed speech data are compared with preset standard samples to obtain comparison values;

[0024] Determine whether the comparison value is greater than a preset difference threshold;

[0025] If so, the processed speech data is subjected to wavelet decomposition and speech reconstruction to obtain noise-reduced data;

[0026] If not, then the de-noiseed speech data will be used directly as the noise reduction data.

[0027] Furthermore, in a sixth implementation of the first aspect of the present invention, S103 includes:

[0028] The time spectrum of the de-noiseed speech data is obtained, and the time spectrum is filtered to obtain a time-domain signal;

[0029] The time-domain signal is sequentially subjected to windowing and framing processing, as well as short-time Fourier transform, to obtain a time-frequency signal.

[0030] Based on a preset masking estimation algorithm, the time-frequency signal is calculated to obtain a denoised real-valued spectrum;

[0031] The denoised real spectrum is normalized to zero mean and unit variance to obtain processed data. The processed data is then input into a pre-set three-layer BLSTM network for training to obtain the recovered spectrum.

[0032] Based on the recovered spectrum, a new speech time domain is reconstructed to obtain noise-reduced data.

[0033] A second aspect of the present invention provides a multi-dimensional RNN speech denoising device, the multi-dimensional RNN speech denoising device comprising:

[0034] The acquisition module is used to acquire raw voice data with a specified preset storage threshold.

[0035] The noise reduction module is used to remove segments containing noise from the original speech data to obtain noise-reduced speech data.

[0036] The noise reduction module is used to perform noise reduction processing on the noise-reduced speech data based on a preset noise reduction algorithm to obtain noise-reduced data;

[0037] The training module is used to iteratively train the step of denoising the noise-reducing speech data based on a preset RNN algorithm to obtain near-realistic denoised data.

[0038] A third aspect of the present invention provides a multi-dimensional RNN speech denoising device, the multi-dimensional RNN speech denoising device comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line;

[0039] The at least one processor invokes the instructions in the memory to cause the device to execute the above-described multi-dimensional RNN speech denoising method.

[0040] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described multidimensional RNN speech denoising method.

[0041] The beneficial effects of this invention are as follows: This method incorporates a preliminary processing step for speech data, which controls the storage size of the speech data, verifies the removal of invalid parts covered by noise from the entire speech data segment, and obtains suitable speech segments. This preprocessing effectively ensures the initial quality of the acquired speech data. To reduce the difficulty of subsequent noise reduction and guarantee its effectiveness, a windowing and framing process for the speech signal is included. This process divides the speech signal into different frames, utilizing these small-segment, small-segment frames for feature extraction and comparative analysis. This eliminates the need for the traditional long, redundant speech segments used for feature extraction and analysis, significantly improving the overall precision and accuracy of the comparative analysis. This lays a good foundation for subsequent wavelet decomposition and speech reconstruction. Furthermore, the method employs an RNN algorithm for deep learning in the speech noise reduction process. When performing noise reduction on similar speech data in subsequent steps, this reduces a large number of processing steps, greatly improving the efficiency and effectiveness of the overall noise reduction process and enhancing the application effect of the method. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the first embodiment of the multi-dimensional RNN speech denoising method in this invention.

[0043] Figure 2 This is a schematic diagram of the second embodiment of the multi-dimensional RNN speech denoising method in this invention.

[0044] Figure 3 This is a schematic diagram of the third embodiment of the multi-dimensional RNN speech denoising method in this invention.

[0045] Figure 4 This is a schematic diagram of the fourth embodiment of the speech denoising method using multi-dimensional RNNs in this invention.

[0046] Figure 5This is a schematic diagram of the fifth embodiment of the speech denoising method using multi-dimensional RNNs in this invention.

[0047] Figure 6 This is a schematic diagram of the sixth embodiment of the speech denoising method using multi-dimensional RNNs in this invention.

[0048] Figure 7 This is a schematic diagram of the first embodiment of the multi-dimensional RNN speech noise reduction device in this invention.

[0049] Figure 8 This is a schematic diagram of an embodiment of a speech noise reduction device using a multi-dimensional RNN, as described in this invention. Detailed Implementation

[0050] This invention provides a multi-dimensional RNN-based speech denoising method, apparatus, device, and medium.

[0051] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1-6 One embodiment of the speech denoising method using multi-dimensional RNNs in this invention includes:

[0053] S101. Obtain raw voice data with a specified preset storage threshold;

[0054] In this embodiment, the original voice data can be acquired by connecting to the Internet for voice download, Bluetooth wireless transmission for voice acquisition, face-to-face fast transfer for voice acquisition, or importing voice data from a USB flash drive, etc.

[0055] The storage threshold is a threshold for setting the storage size of the acquired voice data to avoid excessive data size, which would reduce processing efficiency. Alternatively, a threshold in seconds can be used as the control threshold.

[0056] S102. Remove the segments containing noise from the original speech data to obtain noise-free speech data;

[0057] Furthermore, in the first embodiment of step S102, the following can also be performed:

[0058] S1021. Enhance the spectrogram of the de-noiseed speech data to obtain enhanced speech data;

[0059] S1022. Extract low-level audio features from the enhanced speech data, wherein the low-level audio features include short-time average zero-crossing rate, Mel frequency cepstral coefficients, root mean square value of the spectrum, central moment of the spectrum, monotonic value of the spectrum, bandwidth of the spectrum, and polynomial coefficients of the spectrum.

[0060] S1023. Input the low-level audio features into a pre-set training model for training to obtain high-level audio features for audio classification.

[0061] S1024. Based on the high-level audio features, extract the content from the speech data and filter the speech data to obtain noise-removed speech data.

[0062] In steps S1021-S1024, the enhanced speech data is enhanced twice. After the traditional audio data enhancement, spectrogram data enhancement is performed again to increase the diversity of the data, improve the generalization ability of the feature extraction model, and have a positive impact on the final classification accuracy.

[0063] Low-level audio features include, but are not limited to, short-time average zero-crossing rate, Mel frequency cepstral coefficients, root mean square value of the spectrum, central moment of the spectrum, monotonicity of the spectrum, bandwidth of the spectrum, and polynomial coefficients of the spectrum.

[0064] The trained model can extract high-level deep features from speech content. By extracting high-level deep features, the semantic information of speech can be understood as a whole, taking into account the correlation and differences between speech segments. This allows longer speech segments to be used directly for speech classification and retrieval without segmentation, avoiding the loss of contextual information caused by segmenting long speech data.

[0065] Furthermore, in the second embodiment of step S102, the following can also be performed:

[0066] S102a. Perform MFCC feature extraction on the speech data to obtain feature data;

[0067] S102b: Based on the sparse coding algorithm, the feature data is calculated to obtain sparse coefficients;

[0068] S102c. The sparse coefficients are used as features and input into a preset GMM classifier for classification processing to obtain classification features. Based on the classification features, the speech data is classified to obtain de-noiseed speech data.

[0069] Furthermore, after step S102, the following steps are also included:

[0070] S105. Perform power spectrum analysis, slice spectrum analysis, and spectrum envelope cepstral analysis on the noise-removed speech data.

[0071] S103. Based on the preset noise reduction algorithm, the noise-reduced speech data is processed to obtain noise-reduced data;

[0072] Furthermore, in the first embodiment of step S103, the following can also be performed:

[0073] S1031. The noise-removed speech data is filtered and windowed and framed to obtain processed speech data.

[0074] S1032. The features extracted from the processed speech data are compared with preset standard samples to obtain a comparison value;

[0075] S1033. Determine whether the comparison value is greater than a preset difference threshold;

[0076] S1034. If so, the processed speech data is subjected to wavelet decomposition and speech reconstruction to obtain noise-reduced data.

[0077] S1035. If not, then the de-noiseed speech data is directly used as the noise reduction data.

[0078] In steps S1031-S1035,

[0079] Multiplying y(t) by a certain window function w(t) yields the following windowed speech signal:

[0080] yw(t) = y(t) * w(t);

[0081] Given the expression fN, determine fN-M and gj, j = N-1, ..., NM.

[0082] fN=gN-1+gN-2+......+gN-M+fN-M

[0083] From gj and fN-M, fN is obtained, and the wavelet reconstruction is obtained.

[0084] Let ψJK be an orthogonal basis for the wavelet space. Let y(t) = Vj⁻¹, projecting it onto the spaces Vj and Wj respectively, to obtain the wavelet decomposition of y(t) at scale j:

[0085]

[0086] Where cj,k and dj,k are the residual coefficients and wavelet coefficients in scale j space, respectively.

[0087] The db3 wavelet is a compactly supported orthogonal basis, satisfying the conditions for accurate reconstruction. Its wavelet function and scaling function have an effective support length of 5, and the vanishing moment of the wavelet function is 3. The expression for the db3 wavelet function is as follows:

[0088]

[0089] The db3 wavelet function and scaling function are obtained by iterating once within the tightly supported interval.

[0090] Furthermore, in the first embodiment of step S103, the following can also be performed:

[0091] S103a. Obtain the time spectrum of the de-noiseed speech data and filter the time spectrum to obtain a time-domain signal;

[0092] S103b: The time-domain signal is sequentially subjected to windowing and framing processing and short-time Fourier transform to obtain a time-frequency signal;

[0093] S103c: Based on a preset masking estimation algorithm, the time-frequency signal is calculated to obtain a denoised real-valued spectrum;

[0094] S103d, The denoised real spectrum is normalized to zero mean and unit variance to obtain processed data. The processed data is then input into a preset three-layer BLSTM network for training to obtain the recovered spectrum.

[0095] S103e. Based on the recovered spectrum, a new speech time domain is reconstructed to obtain noise reduction data.

[0096] In steps S103a-S103e, since most neural network-based speech enhancement algorithms target the amplitude spectrum or logarithmic amplitude spectrum of clean speech for training, the enhanced amplitude spectrum is weighted with the phase of the noisy signal to recover the time-domain signal. This method ignores the influence of phase information on the recovery of the time-domain signal. To make the recovered signal closer to the target signal, this invention adds an IFFT layer after the network output layer based on mask estimation. The output mask value is multiplied by the spectrum of the mixed signal to obtain the enhanced spectrum, which is then combined with the phase of the mixed signal and input into the IFFT layer for time-domain reconstruction. The resulting real number is used as the estimate of the time-domain signal, and the loss function is set to the standardized mean square error of the time-domain signal. This allows the entire network to learn more relevant information from the time-domain signal reconstruction, improving the quality and intelligibility of the final output speech.

[0097] The speech features still utilize 11 frames of 246-dimensional composite features with added contextual information. First, a BLSTM network is used to obtain the denoised real spectrum using a real spectrum approximation method. Here, the concept of Real Spectral Mask (RSM) is introduced. The denoised RSM is estimated using BLSTM, and the RSM is multiplied by the real spectrum of the noisy reverberant signal to obtain the denoised real spectrum. Then, the denoised real spectrum is normalized to zero mean and unit variance and used as input to a three-layer BLSTM for de-reverberation. This de-reverberation is then performed by a three-layer BLSTM based on spectral mapping, and after inverse normalization, the overall enhanced real spectrum is obtained. Finally, temporal reconstruction is performed. Since the real spectrum lacks the phase component found in the complex spectrum, the speech signal can be directly recovered from the enhanced real spectrum based on its characteristics. The loss function used remains the temporal signal loss function.

[0098] S104. Based on a preset RNN algorithm, the step of denoising the noise-reducing speech data is trained cyclically to obtain near-realistic denoised data.

[0099] In this embodiment, the method incorporates a preliminary processing step for speech data, which controls the storage size of the speech data, verifies the removal of invalid parts covered by noise from the entire speech data, and obtains suitable speech segments. This preprocessing effectively ensures the initial quality of the acquired speech data. To reduce the difficulty of subsequent noise reduction and ensure the noise reduction effect, a windowing and framing process for the speech signal is included. This process divides the speech signal into different parts and uses these small-segment, small-segment frames for feature extraction and comparative analysis. This eliminates the need for the traditional step of short-circuiting long speech segments for feature extraction and analysis, greatly improving the overall precision and accuracy of the comparative analysis. This lays a good foundation for subsequent wavelet decomposition and speech reconstruction. Furthermore, the method employs an RNN algorithm for deep learning in the speech noise reduction process. When performing noise reduction on similar speech data in subsequent steps, it can reduce a large number of processing steps, greatly improving the efficiency and effectiveness of the overall noise reduction process and enhancing the application effect of the method.

[0100] The above describes the speech denoising method using a multi-dimensional RNN in the embodiments of the present invention. The following describes the speech denoising device using a multi-dimensional RNN in the embodiments of the present invention. Please refer to [link / reference]. Figure 7 One embodiment of the multi-dimensional RNN speech denoising device in this invention includes:

[0101] The acquisition module 201 is used to acquire raw voice data with a specified preset storage threshold.

[0102] The noise reduction module 202 is used to remove segments containing noise from the original speech data to obtain noise-reduced speech data;

[0103] The noise reduction module 203 is used to perform noise reduction processing on the noise-reduced speech data based on a preset noise reduction algorithm to obtain noise-reduced data;

[0104] Training module 204 is used to iteratively train the step of denoising the noise-reducing speech data based on a preset RNN algorithm to obtain near-realistic denoised data.

[0105] In another embodiment of the multi-dimensional RNN speech denoising device of the present invention, the multi-dimensional RNN speech denoising device includes:

[0106] The acquisition module 201 is used to acquire raw voice data with a specified preset storage threshold.

[0107] The noise reduction module 202 is used to remove segments containing noise from the original speech data to obtain noise-reduced speech data;

[0108] The noise reduction module 203 is used to perform noise reduction processing on the noise-reduced speech data based on a preset noise reduction algorithm to obtain noise-reduced data;

[0109] Training module 204 is used to iteratively train the step of denoising the noise-reducing speech data based on a preset RNN algorithm to obtain near-realistic denoised data.

[0110] In the first embodiment of the noise reduction module 202, the following can also be executed:

[0111] Enhance the spectrogram of the de-noiseed speech data to obtain enhanced speech data;

[0112] Extract low-level audio features from the enhanced speech data, wherein the low-level audio features include short-time average zero-crossing rate, Mel frequency cepstral coefficients, root mean square value of the spectrum, central moment of the spectrum, monotonicity of the spectrum, bandwidth of the spectrum, and polynomial coefficients of the spectrum.

[0113] The low-level audio features are input into a pre-set training model for training to obtain high-level audio features for audio classification.

[0114] Based on the high-level audio features, the content in the speech data is extracted and the speech data is filtered to obtain noise-removed speech data.

[0115] In the second embodiment of the noise reduction module 202, the following can also be executed:

[0116] MFCC feature extraction is performed on the speech data to obtain feature data;

[0117] Based on the sparse coding algorithm, the feature data is calculated to obtain sparse coefficients;

[0118] The sparse coefficients are used as features and input into a preset GMM classifier for classification processing to obtain classification features. Based on the classification features, the speech data is classified to obtain de-noiseed speech data.

[0119] The noise reduction module 202 is followed by:

[0120] Power spectrum analysis, slice spectrum analysis, and spectral envelope cepstral analysis were performed on the noise-removed speech data.

[0121] In the first embodiment of the noise reduction module 203, the following can also be performed:

[0122] The noise-removed speech data is filtered and windowed and framed to obtain processed speech data;

[0123] The features extracted from the processed speech data are compared with preset standard samples to obtain comparison values;

[0124] Determine whether the comparison value is greater than a preset difference threshold;

[0125] If so, the processed speech data is subjected to wavelet decomposition and speech reconstruction to obtain noise-reduced data;

[0126] If not, then the de-noiseed speech data will be used directly as the noise reduction data.

[0127] In the second embodiment of the noise reduction module 203, the following can also be performed:

[0128] The time spectrum of the de-noiseed speech data is obtained, and the time spectrum is filtered to obtain a time-domain signal;

[0129] The time-domain signal is sequentially subjected to windowing and framing processing, as well as short-time Fourier transform, to obtain a time-frequency signal.

[0130] Based on a preset masking estimation algorithm, the time-frequency signal is calculated to obtain a denoised real-valued spectrum;

[0131] The denoised real spectrum is normalized to zero mean and unit variance to obtain processed data. The processed data is then input into a pre-set three-layer BLSTM network for training to obtain the recovered spectrum.

[0132] Based on the recovered spectrum, a new speech time domain is reconstructed to obtain noise-reduced data.

[0133] In this embodiment, the method incorporates a preliminary processing step for speech data, which controls the storage size of the speech data, verifies the removal of invalid parts covered by noise from the entire speech data, and obtains suitable speech segments. This preprocessing effectively ensures the initial quality of the acquired speech data. To reduce the difficulty of subsequent noise reduction and ensure the noise reduction effect, a windowing and framing process for the speech signal is included. This process divides the speech signal into different parts and uses these small-segment, small-segment frames for feature extraction and comparative analysis. This eliminates the need for the traditional step of short-circuiting long speech segments for feature extraction and analysis, greatly improving the overall precision and accuracy of the comparative analysis. This lays a good foundation for subsequent wavelet decomposition and speech reconstruction. Furthermore, the method employs an RNN algorithm for deep learning in the speech noise reduction process. When performing noise reduction on similar speech data in subsequent steps, it can reduce a large number of processing steps, greatly improving the efficiency and effectiveness of the overall noise reduction process and enhancing the application effect of the method.

[0134] The above is attached Figure 7 The speech denoising device of the multi-dimensional RNN in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The speech denoising device of the multi-dimensional RNN in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0135] Appendix Figure 8 This is a schematic diagram of the structure of a multi-dimensional RNN speech denoising device 300 provided in an embodiment of the present invention. The multi-dimensional RNN speech denoising device 300 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the multi-dimensional RNN speech denoising device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the multi-dimensional RNN speech denoising device 300.

[0136] The multi-dimensional RNN-based speech noise reduction device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, MacOSX, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 8 The illustrated multidimensional RNN speech denoising device structure does not constitute a limitation on multidimensional RNN speech denoising devices, which may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0137] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the multi-dimensional RNN speech denoising method, apparatus, device and medium.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0139] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-dimensional RNN-based speech denoising method, characterized in that, include: S101: Obtain raw voice data with a specified preset storage threshold; S102: Remove the segments containing noise from the original speech data to obtain noise-free speech data; S103: Based on a preset noise reduction algorithm, the noise-reduced speech data is processed to obtain noise-reduced data; S104: Based on the preset RNN algorithm, perform iterative training on step S103 to obtain realistic and denoised data; Wherein, S102 includes: MFCC feature extraction is performed on the speech data to obtain feature data; Based on the sparse coding algorithm, the feature data is calculated to obtain sparse coefficients; The sparse coefficients are used as features and input into a preset GMM classifier for classification processing to obtain classification features. Based on the classification features, the speech data is classified to obtain de-noiseed speech data. Wherein, S103 includes: The time spectrum of the de-noiseed speech data is obtained, and the time spectrum is filtered to obtain a time-domain signal; The time-domain signal is sequentially subjected to windowing and framing processing, as well as short-time Fourier transform, to obtain a time-frequency signal. Based on a preset masking estimation algorithm, the time-frequency signal is calculated to obtain a denoised real-valued spectrum; The denoised real spectrum is normalized to zero mean and unit variance to obtain processed data. The processed data is then input into a pre-set three-layer BLSTM network for training to obtain the recovered spectrum. Based on the recovered spectrum, a new speech time domain is reconstructed to obtain noise-reduced data.

2. The speech denoising method using multi-dimensional RNNs according to claim 1, characterized in that, S102 includes: Enhance the spectrogram of the de-noiseed speech data to obtain enhanced speech data; Extract low-level audio features from the enhanced speech data, wherein the low-level audio features include short-time average zero-crossing rate, Mel frequency cepstral coefficients, root mean square value of the spectrum, central moment of the spectrum, monotonicity of the spectrum, bandwidth of the spectrum, and polynomial coefficients of the spectrum. The low-level audio features are input into a pre-set training model for training to obtain high-level audio features for audio classification. Based on the high-level audio features, the content in the speech data is extracted and the speech data is filtered to obtain noise-removed speech data.

3. The speech denoising method using multi-dimensional RNNs according to claim 1, characterized in that, Following S102, the following is also included: Power spectrum analysis, slice spectrum analysis, and spectral envelope cepstral analysis were performed on the noise-removed speech data.

4. The speech denoising method using a multi-dimensional RNN according to claim 1, characterized in that, S103 includes: The noise-removed speech data is filtered and windowed and framed to obtain processed speech data; The features extracted from the processed speech data are compared with preset standard samples to obtain comparison values; Determine whether the comparison value is greater than a preset difference threshold; If so, the processed speech data is subjected to wavelet decomposition and speech reconstruction to obtain noise-reduced data; If not, then the de-noiseed speech data will be used directly as the noise reduction data.

5. A multi-dimensional RNN-based speech noise reduction device, characterized in that, include: The acquisition module is used to acquire raw voice data with a specified preset storage threshold. The noise reduction module is used to remove segments containing noise from the original speech data to obtain noise-reduced speech data. The noise reduction module is used to perform noise reduction processing on the noise-reduced speech data based on a preset noise reduction algorithm to obtain noise-reduced data; The training module is used to iteratively train the step of denoising the de-noiseed speech data based on a preset RNN algorithm to obtain near-realistic denoised data. Specifically, the noise reduction module is also used for: The time spectrum of the de-noiseed speech data is obtained, and the time spectrum is filtered to obtain a time-domain signal; The time-domain signal is sequentially subjected to windowing and framing processing, as well as short-time Fourier transform, to obtain a time-frequency signal. Based on a preset masking estimation algorithm, the time-frequency signal is calculated to obtain a denoised real-valued spectrum; The denoised real spectrum is normalized to zero mean and unit variance to obtain processed data. The processed data is then input into a pre-set three-layer BLSTM network for training to obtain the recovered spectrum. Based on the recovered spectrum, a new speech time domain is reconstructed to obtain noise reduction data; Specifically, the noise reduction module is also used for: MFCC feature extraction is performed on the speech data to obtain feature data; Based on the sparse coding algorithm, the feature data is calculated to obtain sparse coefficients; The sparse coefficients are used as features and input into a preset GMM classifier for classification processing to obtain classification features. Based on the classification features, the speech data is classified to obtain de-noiseed speech data.

6. A multi-dimensional RNN-based speech noise reduction device, characterized in that, The multidimensional RNN speech noise reduction device includes: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor invokes the instructions in the memory to cause the multidimensional RNN speech denoising device to perform the multidimensional RNN speech denoising method as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the speech denoising method of the multi-dimensional RNN as described in any one of claims 1-4.

Images