A method, apparatus, device, storage medium, and product for speech noise reduction.

Abstract

This application provides a speech denoising method, apparatus, device, storage medium, and product. The technical solution provided in this application obtains subspace feature information of the speech information to be processed in multiple signal subspaces through a speech denoising model, combines the multiple subspace feature information into denoising feature information, and generates denoised speech information based on the denoising feature information. By decomposing the feature information of the speech information to be processed into multiple subspace feature information from different signal subspaces, the speech denoising model can perform denoising processing on the speech information to be processed from different signal subspaces, reducing the computational load and difficulty of denoising the speech information to be processed, and effectively improving the efficiency of speech denoising.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a speech noise reduction method, apparatus, device, storage medium, and product. Background Technology

[0002] With the development of multimedia technology and the emergence of various conferencing and entertainment software, speech noise reduction is being used more and more widely in scenarios such as voice calls, live audio and video streaming, and multi-person conferences. The speech captured by the microphones of terminal devices usually contains a certain degree of noise. Audio noise reduction algorithms can suppress the noise carried in the speech, thereby improving speech quality.

[0003] Currently, speech denoising is generally based on deep learning speech denoising models. However, traditional speech denoising models have a large computational load, and under the limitation of the computational overhead of terminal devices, they are prone to low denoising efficiency. Summary of the Invention

[0004] This application provides a speech denoising method, apparatus, device, storage medium, and product to solve the problems of large computational load and low denoising efficiency in related technologies, effectively reducing the computational load and improving the speech denoising efficiency.

[0005] In a first aspect, embodiments of this application provide a speech noise reduction method, including:

[0006] Acquire the voice information to be processed;

[0007] The speech information to be processed is input into the trained speech denoising model. The speech denoising model is used to obtain the subspace feature information of the speech information to be processed in multiple signal subspaces. The multiple subspace feature information is combined into denoising feature information, and denoised speech information is generated based on the denoising feature information.

[0008] In a second aspect, embodiments of this application provide a speech noise reduction device, including a data acquisition module and a noise reduction processing module, wherein:

[0009] The data acquisition module is configured to acquire voice information to be processed;

[0010] The noise reduction processing module is configured to input the speech information to be processed into a trained speech noise reduction model, obtain subspace feature information of the speech information to be processed in multiple signal subspaces through the speech noise reduction model, combine the multiple subspace feature information into noise reduction feature information, and generate noise-reduced speech information based on the noise reduction feature information.

[0011] In a third aspect, embodiments of this application provide a voice noise reduction device, including: a memory and one or more processors;

[0012] The memory is used to store one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the speech noise reduction method as described in the first aspect.

[0014] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the speech noise reduction method as described in the first aspect.

[0015] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the speech noise reduction method as described in the first aspect.

[0016] This application embodiment obtains the subspace feature information of the speech information to be processed in multiple signal subspaces through a speech denoising model, combines the multiple subspace feature information into denoising feature information, and generates denoised speech information based on the denoising feature information. By decomposing the feature information of the speech information to be processed into multiple subspace feature information from different signal subspaces, the speech denoising model can perform denoising processing on the speech information to be processed from different signal subspaces, reducing the amount and difficulty of denoising the speech information to be processed, and effectively improving the efficiency of speech denoising. Attached Figure Description

[0017] Figure 1 This is a flowchart of a speech noise reduction method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of a subspace feature information acquisition process provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of a noise-reduced speech information generation process provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of a speech noise reduction model provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a gated loop unit layer provided in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the structure of a speech noise reduction device provided in an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of the structure of a voice noise reduction device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0025] The speech denoising method provided in this application can be applied to various speech denoising scenarios, such as voice calls, live audio and video streaming, and multi-person conferences. It processes the collected speech information by decomposing the feature information of the speech information to be processed into multiple subspace feature information from different signal subspaces. This allows the speech denoising model to process the speech information from different signal subspaces, reducing the computational load and difficulty of denoising the speech information and effectively improving the efficiency of speech denoising. Traditional speech denoising schemes typically process the speech information in a single signal space. However, this method involves a large computational load, and under the computational limitations of terminal devices, it is prone to low denoising efficiency. Therefore, this application provides a speech denoising method according to an embodiment to solve the technical problem of low denoising efficiency in existing speech denoising schemes.

[0026] Figure 1 A flowchart of a speech noise reduction method provided in an embodiment of this application is given. The speech noise reduction method provided in this embodiment of the application can be executed by a speech noise reduction device, which can be implemented by hardware and / or software and integrated into a speech noise reduction device.

[0027] The following description uses a speech denoising device to perform a speech denoising method as an example. (Reference) Figure 1 The speech noise reduction method includes:

[0028] S110: Obtain the voice information to be processed.

[0029] For example, the voice information to be processed provided by this solution can be obtained through real-time recording or through voice selected by the user. For instance, the user can record the voice to be processed using a voice noise reduction device, or select the voice to be processed through network upload or local storage. Optionally, the voice information to be processed can be complete voice data or a voice data stream acquired or uploaded in real time.

[0030] In this context, the voice noise reduction device can be a user-used terminal device or a server. For example, when a terminal device acts as the voice noise reduction device, assuming a user is using it for live streaming, the device collects the user's voice and treats it as voice information to be processed. After noise reduction, the resulting denoised voice information can be uploaded to a server for further processing. When a server acts as the voice noise reduction device, it receives the voice information to be processed from the terminal, performs noise reduction on it, and then sends it back to the corresponding user.

[0031] S120: Input the speech information to be processed into the trained speech denoising model, obtain the subspace feature information of the speech information to be processed in multiple signal subspaces through the speech denoising model, combine the multiple subspace feature information into denoising feature information, and generate denoised speech information based on the denoising feature information.

[0032] The speech denoising device provided in this solution is equipped with a trained deep learning-based speech denoising model. This model can acquire subspace feature information of received speech data in multiple different signal subspaces, combine the feature information from multiple subspaces into denoising feature information, and generate denoised speech information based on the denoising feature information. Specifically, the speech denoising information can focus on the features of the input speech data from different signal subspaces, thereby learning and capturing the subspace features of the speech data in different signal subspaces. In other words, the speech denoising model can decompose the features of the speech data into multiple signal subspaces, allowing the model to focus on information from different aspects of the speech data, effectively improving the stability and denoising effect of speech denoising.

[0033] For example, after acquiring the speech information to be processed, it is input into the speech denoising model. Upon receiving the speech information, the speech denoising model acquires the subspace feature information of the speech information in multiple different signal subspaces. This solution decomposes the features of the speech information to be processed into multiple different signal subspaces through the speech denoising model, allowing the model to focus on the feature information of different signal subspaces. This accurately obtains the subspace feature information corresponding to multiple different signal subspaces, reducing the computational complexity of speech denoising processing, improving speech denoising efficiency, and providing more comprehensive denoising processing for the speech information, effectively improving the stability and effect of speech denoising.

[0034] In one embodiment, the deep learning-based speech denoising model provided by this solution can be trained separately for sample audio data in different signal subspaces during the training process, enabling the speech denoising model to learn the subspace features of the speech data in different signal subspaces. Optionally, the number of signal subspaces provided by this solution can be preset, and the number of signal subspaces can be determined based on the denoising test results of the speech denoising model under different numbers of signal subspaces. Optionally, the denoising test results of the speech denoising model can be determined based on perceptual evaluation of speech quality (PESQ).

[0035] Furthermore, after obtaining feature information from multiple subspaces, the speech denoising model combines these feature information into denoising feature information and generates denoised speech information based on this denoising feature information. Specifically, the speech information to be processed is decomposed into a signal subspace and a noise subspace. Effective feature information is extracted from the speech information by fitting each signal subspace, while noise features corresponding to the noise subspace are filtered out. Each subspace feature information retains the signal features of the speech information to be processed in its corresponding signal subspace, thus achieving effective feature information extraction within that subspace and enabling denoising processing of the speech. After obtaining the subspace feature information corresponding to different signal subspaces, the speech denoising model combines these feature information into denoising feature information. This eliminates the need for the speech denoising model to perform denoising processing on the entire signal space of the speech information to be processed, effectively reducing the computational complexity of speech denoising processing and improving its efficiency.

[0036] In one possible embodiment, such as Figure 2 The provided diagram illustrates a subspace feature information acquisition process. The speech denoising model provided in this solution, when acquiring subspace feature information of the speech information to be processed in multiple signal subspaces, includes:

[0037] S121: Obtain the audio domain information of the speech information to be processed.

[0038] S122: Use fully connected layers and / or convolutional layers to process speech domain information to obtain amplitude feature information.

[0039] S123: Obtain the subspace feature information of amplitude feature information in multiple signal subspaces respectively.

[0040] For example, the speech denoising model provided in this solution, after receiving the speech information to be processed, performs feature extraction on the speech information to be processed to obtain the audio domain information of the speech information to be processed. The audio domain information may include amplitude feature information (mag) and / or phase feature information (phase) corresponding to the speech information to be processed.

[0041] Optionally, the speech information to be processed can be preprocessed before feature extraction to improve the feature extraction effect. Preprocessing of the speech information to be processed can be one or a combination of framing, windowing, short-time Fourier transform, and feature compression.

[0042] Furthermore, the speech denoising model inputs the extracted audio domain information into fully connected layers and / or convolutional layers, which process the audio domain information to obtain amplitude feature information. For example, the fully connected layers and / or convolutional layers process the amplitude feature information in the audio domain information to obtain amplitude feature information. The audio domain information sent to the fully connected layers and / or convolutional layers may include amplitude feature information and phase feature information, or it may only include amplitude feature information.

[0043] Furthermore, for each signal subspace, the speech denoising model acquires the subspace feature information of the amplitude feature information in the corresponding signal subspace, thereby obtaining the subspace feature information of the amplitude feature information in multiple signal subspaces. This scheme acquires the audio domain information of the speech information to be processed, and uses fully connected layers and / or convolutional layers to obtain the amplitude feature information corresponding to the audio domain information. Then, it acquires the subspace feature information of the amplitude feature information in multiple signal subspaces, accurately obtaining the subspace feature information of the speech information to be processed in each signal subspace. This achieves feature information extraction in different signal subspaces, eliminating the need for the speech denoising model to perform denoising processing on the speech information to be processed in the entire signal space. This effectively reduces the computational complexity of speech denoising processing, improves speech denoising efficiency, and extracts subspace feature information for different signal subspaces, resulting in more complete extraction of effective feature information, more accurate restoration of effective speech, and improved speech denoising effect.

[0044] In one possible embodiment, the speech denoising model provided by this solution, when acquiring the subspace feature information of amplitude feature information in multiple signal subspaces respectively, includes: sending amplitude feature information to multiple subspace networks respectively, and using multiple subspace networks to acquire the subspace feature information of amplitude feature information in multiple signal subspaces respectively.

[0045] The speech denoising model provided in this solution is configured with sub-network layers (e.g., RNN (Recurrent Neural Network) layers), and multiple subspace networks are configured within these sub-network layers. Optionally, the subspace networks can be implemented based on operators such as Gated Recurrent Units (GRUs) and / or Long Short-Term Memory (LSTMs). Different subspace networks can acquire amplitude feature information in different signal subspaces.

[0046] For example, after obtaining the amplitude feature information corresponding to the audio domain information, the speech denoising model copies the amplitude feature information into multiple copies (the number of copied amplitude feature information is not less than the number of signal subspaces to be processed), and sends one amplitude feature information to each subspace network. Through each subspace network, the subspace feature information of the amplitude feature information in the corresponding signal subspace is obtained, thereby obtaining the subspace feature information of the amplitude feature information in multiple signal subspaces.

[0047] This scheme uses multiple subspace networks to focus on the features of different signal subspaces in the speech information to be processed. Multiple subspace networks can simultaneously learn and capture the features of different signal subspaces in the speech information to be processed, so as to decompose the features of the speech information to be processed into multiple signal subspaces. The speech denoising model focuses on different aspects of information through different subspace networks, and combines the temporal characteristics of recurrent neural networks (RNNs) to effectively improve the stability and denoising effect of the speech denoising model.

[0048] In one possible embodiment, the speech denoising model provided by this solution, when combining multiple subspace feature information into denoising feature information, includes: performing weighted summation processing on the multiple subspace feature information based on the weight coefficients corresponding to the speech information to be processed in different subspaces to obtain denoising feature information.

[0049] For example, after obtaining the subspace feature information of the speech information to be processed in multiple signal subspaces, the speech denoising model will perform a weighted summation of the determined subspace feature information according to the weight coefficients corresponding to the speech information in different signal subspaces to obtain denoising feature information. This denoising feature information reflects the subspace feature information corresponding to each signal subspace, providing a more comprehensive denoising process and effectively improving the stability and effectiveness of speech denoising. Furthermore, by configuring the weight coefficients corresponding to different signal subspaces for weighted summation of different subspace feature information, the denoising feature information more accurately preserves effective speech features, thus improving the speech denoising effect.

[0050] In one possible embodiment, before the speech denoising model provided by this solution performs weighted summation processing on the feature information of multiple subspaces based on the weight coefficients corresponding to the speech information to be processed in different subspaces to obtain the denoising feature information, it further includes: sending the amplitude feature information corresponding to the speech information to be processed to the weight acquisition network, and using the weight acquisition network to obtain the weight coefficients corresponding to the amplitude feature information in multiple signal subspaces.

[0051] The speech denoising model provided in this solution includes a weight acquisition network, which can be configured within sub-network layers and implemented using convolutional layers (Conv). After obtaining the amplitude feature information corresponding to the audio domain information, the speech denoising model copies the amplitude feature information multiple times (the number of copies is the number of signal subspaces to be processed + 1), and sends one amplitude feature information to each subspace network and one amplitude feature information to the weight acquisition network. The extraction of subspace feature information by the subspace networks and the determination of the weight coefficients for different signal subspaces by the weight acquisition network can be learned through training the speech denoising model.

[0052] For example, after receiving amplitude feature information, the weight acquisition network analyzes and processes the amplitude feature information and outputs the weight coefficients corresponding to each signal subspace. This scheme obtains the weight coefficients corresponding to different signal subspaces through the weight acquisition network, making the denoising feature information more accurate in preserving effective speech features and improving the speech denoising effect.

[0053] It's important to explain that, unlike traditional speech denoising models, which typically employ a single spatial network (e.g., a single gated recurrent unit, GRU), traditional denoising models rely on a single spatial network to denoise noisy speech data. This method requires extensive and complex computations, and the denoising effect is difficult to guarantee. In this scheme, each subspace network in the subnetwork layer only needs to focus on information within its own signal subspace. Therefore, the number of hidden layers in N subspace networks can be approximately 1 / N of that in a single spatial network. Taking GRU as an example, the computational difference between a traditional single spatial network GRU and the multiple subspace networks GRU in this scheme is approximately:

[0054]

[0055] in, The computational cost of the N subspace network GRU provided by this scheme, [(d x +d h )*d h [This refers to the computational cost of a traditional single spatial network GRU, d] x d is the input feature dimension. h For hidden layer dimensions.

[0056] The difference in the number of parameters between a traditional single spatial network GRU and the multiple (N) subspace network GRUs in this scheme is close to the aforementioned difference in computational complexity, with a parameter difference greater than zero. This indicates that the computational complexity of a traditional single spatial network GRU is greater than that of the multiple subspace network GRUs in this scheme. Therefore, the multiple subspace network GRU structure has significant advantages in both computational complexity and number of parameters compared to the traditional single spatial network GRU structure. Compared to the traditional single spatial network GRU structure, the multiple subspace network GRU structure exhibits stronger robustness, stronger speech denoising capabilities, lower computational complexity, and fewer parameters. This makes AI algorithms, including speech denoising, more suitable for deployment on terminal devices with limited computational resources, such as mobile phones. It improves denoising performance while reducing computational and storage requirements, facilitating the deployment of speech denoising on terminals.

[0057] In one possible embodiment, such as Figure 3 As shown in the schematic diagram of a noise reduction speech information generation process, the speech noise reduction model provided in this solution, when generating noise reduction speech information based on noise reduction feature information, includes:

[0058] S124: Enhance the noise reduction feature information to obtain the frequency domain amplitude gain.

[0059] S125: Determine amplitude estimation information based on frequency domain amplitude gain and amplitude feature information of the speech information to be processed.

[0060] S126: Generate denoised speech information based on amplitude estimation information and phase feature information of the speech information to be processed.

[0061] For example, the speech denoising model enhances the generated denoising feature information to obtain a frequency domain amplitude gain, and determines amplitude estimation information based on this frequency domain amplitude gain and the amplitude feature information of the speech information to be processed (i.e., the amplitude feature information corresponding to the speech audio domain information). Optionally, the amplitude estimation information can be obtained by multiplying the frequency domain amplitude gain and the amplitude feature information.

[0062] Furthermore, denoised speech information is generated based on the determined amplitude estimation information and the phase feature information of the speech information to be processed (i.e., the phase feature information corresponding to the audio domain information). Optionally, denoised speech information can be obtained by performing an inverse fast fourier transform (IFFT) on the amplitude estimation information and the phase feature information. In one embodiment, after obtaining the denoised speech information, the denoised speech information is saved, played, or sent to other user terminals or servers. This scheme obtains the frequency domain amplitude gain by enhancing the denoised feature information, determines the amplitude estimation information based on the frequency domain amplitude gain and the amplitude feature information, and then accurately generates denoised speech information based on the amplitude estimation information and the phase feature information of the speech information to be processed, thereby achieving denoising processing of the speech information to be processed.

[0063] In one possible embodiment, the speech denoising model provided by this solution, when enhancing the denoising feature information to obtain frequency domain amplitude gain, includes: processing the denoising feature information using fully connected layers and / or convolutional layers to obtain first feature information; processing the first feature information and the speech domain information of the speech information to be processed using gated recurrent unit layers to obtain second feature information; and processing the second feature information using dilated convolutional layers to obtain frequency domain amplitude gain.

[0064] For example, after obtaining the denoising feature information, the speech denoising model sends the denoising feature information to a fully connected layer and / or a convolutional layer. The fully connected layer and / or convolutional layer process the denoising feature information to obtain the first feature information. Then, the first feature information and the audio domain information of the speech information to be processed (e.g., amplitude feature information in the audio domain information) are added together and sent to a gated recurrent unit (GRU) layer. The sum of the first feature information and the audio domain information is used as the input of the GRU layer, which processes the first feature information and the audio domain information to obtain the second feature information. The second feature information is then sent to a dilated convolutional layer (Dilation Conv), which processes the second feature information to obtain the frequency domain amplitude gain. This scheme processes the noise reduction feature information through fully connected layers and / or convolutional layers to obtain the first feature information, uses gated recurrent unit layers to process the first feature information and speech domain information to obtain the second feature information, and uses dilated convolutional layers to process the second feature information to accurately determine the frequency domain amplitude gain, thereby improving the estimation accuracy of amplitude estimation information and effectively improving the speech noise reduction effect.

[0065] like Figure 4 The provided diagram illustrates the structure of a speech denoising model. The speech denoising model provided by this solution is configured with a feature extraction layer, a first processing layer (fully connected layer FC or convolutional layer Conv), a sub-network layer, a concatenation layer (Concat), a second processing layer (fully connected layer FC or convolutional layer Conv), a gated recurrent unit layer (GRU), and a dilated convolutional layer (Dilation Conv) connected in sequence. The sub-network layer includes N sub-spatial networks (GRU1~N) and a weight acquisition network (Conv).

[0066] In this process, the speech information Input is input into the speech denoising model. The speech denoising model extracts the audio domain information S0 (including amplitude feature information mag and phase feature information phase) of the speech information Input in the feature extraction layer. The audio domain information S0 (e.g., the amplitude feature information mag in the audio domain information S0) is processed by the first processing layer to obtain amplitude feature information S1. The amplitude feature information S1 is copied into N+1 copies, where N copies of the amplitude feature information are used as inputs x1 to N of N subspace networks, and one copy is used as input xN+1 of the weight acquisition network. Each subspace network processes the amplitude feature information and outputs the subspace feature information y1 to N of the corresponding signal subspace. The weight acquisition network determines the weight coefficients α1 to N of each signal subspace based on the amplitude feature information. In the concatenation layer, the subspace feature information is weighted and summed based on the weight coefficients to obtain the denoised feature information S2 (S2 = x1*α1 + x2*α2 + ... + xN*αN). The denoising feature information S2 is processed by a second processing layer to obtain the first feature information S3. The first feature information S3 is added to the speech domain information S0 (e.g., the amplitude feature information mag in the speech domain information S0) and used as the input to the gated recurrent unit layer. After processing by the gated recurrent unit layer, the second feature information S4 is obtained. The second feature information S4 is processed by a dilated convolutional layer to obtain the frequency domain amplitude gain G1. The frequency domain amplitude gain G1 is multiplied by the amplitude feature information mag in the speech domain information S0 to obtain the amplitude estimation information. The amplitude estimation information and the phase feature information phase of the speech domain information S0 are then subjected to an inverse Fourier transform to generate the denoised speech information Out.

[0067] like Figure 5 A schematic diagram of a gated loop unit layer is provided, wherein h t h is the current output of the gated loop unit layer. t-1 x is the output of the gated recurrent unit layer in the previous iteration. t This is the input to the gated loop unit layer. For example... Figure 5 As shown, the hidden state h passed down from the previous time step t-1 and the current input x t Combined with h t-1 and x t The gated recurrent unit layer obtains the output of the current node (subspace feature information) and the hidden state h of the next node. t .

[0068] z t =ρ(W z ·[h t-1 x t ])

[0069] r t =ρ(W r ·[ht-1 x t ])

[0070]

[0071]

[0072] Where σ is the sigmoid function (activation function) with a value range of (0, 1), and the tanh activation function has a value range of (-1, 1). W z To update the matrix, W r Let W be the reset matrix and W be the weight matrix. t To control the updated gating, r t To control the reset gating, First, use the reset gating to handle the hidden state r. t *h t-1 Then concatenate it with the input to obtain [r] t *h t-1 x t ], weight matrix W and [r t *h t-1 x t Multiply by , then use the tanh activation function to restrict the output to the range (-1, 1). (1- t )*h t-1 This indicates a selective "forgetting" of the original hidden state information. This indicates selective "memory" of h~_t containing current information, (1-z t ) and z t This represents a weight for historical and current information, which sums to 1.

[0073] The modeling of the gated loop unit layer can be represented as follows:

[0074] y t =As t +n t

[0075] in, and These represent speech and noise features, respectively. Represents the characteristic driving matrix, [h] in the corresponding gated loop unit layer t-1 x t ], s t Corresponding to h t .

[0076] The speech information to be processed exists in a subspace spanned by N feature vectors:

[0077]

[0078] Where, α i Let y be the i-th weight coefficient. i For the feature information of the i-th subspace, As t This involves a weighted summation of feature information from multiple subspaces based on weight coefficients. The linear portion of each gated recurrent unit layer fits the signal subspace components, while the nonlinear portion fits the nonlinear components of the noise. Based on the structure and modeling of this subspace network, the feature information of the speech information to be processed is decomposed into multiple subspace feature information from different signal subspaces. This allows the speech denoising model to perform denoising processing on the speech information from different signal subspaces, reducing the computational load and difficulty of denoising the speech information and effectively improving the efficiency of speech denoising.

[0079] The above describes a method that uses a speech denoising model to acquire subspace feature information of the speech information to be processed in multiple signal subspaces. This subspace feature information is then combined to form denoising feature information, which is used to generate denoised speech information. By decomposing the feature information of the speech information to be processed into multiple subspace feature information from different signal subspaces, the speech denoising model can perform denoising processing on the speech information from different signal subspaces, reducing the computational load and difficulty of denoising and effectively improving the efficiency of speech denoising. Simultaneously, multiple subspace networks are used to focus on the features of different signal subspaces in the speech information to be processed. These networks can simultaneously learn and capture the features of different signal subspaces in the speech information to be processed, decomposing the features of the speech information to be processed into multiple signal subspaces. The speech denoising model focuses on different aspects of information through different subspace networks, effectively improving the stability and denoising effect of the speech denoising model. Furthermore, by configuring the weight coefficients corresponding to different signal subspaces and performing weighted summation of the feature information from different subspaces, the denoising feature information can more accurately retain effective speech features, further improving the speech denoising effect.

[0080] Figure 6 This is a schematic diagram of the structure of a speech noise reduction device provided in an embodiment of this application. (Reference) Figure 6 The voice noise reduction device includes a data acquisition module 61 and a noise reduction processing module 62.

[0081] The data acquisition module 61 is configured to acquire the speech information to be processed; the noise reduction processing module 62 is configured to input the speech information to be processed into the trained speech noise reduction model, acquire the subspace feature information of the speech information to be processed in multiple signal subspaces through the speech noise reduction model, combine the multiple subspace feature information into noise reduction feature information, and generate noise-reduced speech information based on the noise reduction feature information.

[0082] The above describes a method that uses a speech denoising model to obtain subspace feature information of the speech information to be processed in multiple signal subspaces, combines the multiple subspace feature information into denoising feature information, and generates denoised speech information based on the denoising feature information. By decomposing the feature information of the speech information to be processed into multiple subspace feature information from different signal subspaces, the speech denoising model can perform denoising processing on the speech information to be processed from different signal subspaces, thereby reducing the amount and difficulty of denoising the speech information to be processed and effectively improving the efficiency of speech denoising.

[0083] In one possible embodiment, the speech denoising model is configured as follows when acquiring subspace feature information of the speech information to be processed in multiple signal subspaces:

[0084] Obtain the audio domain information of the speech information to be processed;

[0085] Amplitude feature information is obtained by processing speech domain information using fully connected layers and / or convolutional layers;

[0086] The amplitude characteristic information is obtained in the subspace characteristic information of multiple signal subspaces respectively.

[0087] In one possible embodiment, the speech denoising model is configured as follows when acquiring the subspace feature information of amplitude feature information in multiple signal subspaces:

[0088] Amplitude feature information is sent to multiple subspace networks respectively, and the subspace feature information of amplitude feature information in multiple signal subspaces is obtained by using multiple subspace networks respectively.

[0089] In one possible embodiment, the speech denoising model is configured to combine multiple subspace feature information into denoising feature information as follows:

[0090] Based on the weight coefficients corresponding to the speech information to be processed in different subspaces, the feature information of multiple subspaces is weighted and summed to obtain the noise reduction feature information.

[0091] In one possible embodiment, before the speech denoising model performs a weighted summation of feature information from multiple subspaces based on the weight coefficients corresponding to the speech information to be processed in different subspaces to obtain the denoised feature information, it is further configured as follows:

[0092] The amplitude feature information corresponding to the speech information to be processed is sent to the weight acquisition network, and the weight acquisition network is used to obtain the weight coefficients corresponding to the amplitude feature information in multiple signal subspaces.

[0093] In one possible embodiment, the speech denoising model is configured as follows when generating denoised speech information based on denoising feature information:

[0094] The noise reduction feature information is enhanced to obtain the frequency domain amplitude gain;

[0095] Amplitude estimation information is determined based on the frequency domain amplitude gain and the amplitude feature information of the speech information to be processed.

[0096] Noise-reduced speech information is generated based on amplitude estimation information and phase feature information of the speech information to be processed.

[0097] In one possible embodiment, when the speech denoising model enhances the denoising feature information to obtain the frequency domain amplitude gain, it is configured as follows:

[0098] The first feature information is obtained by processing the denoised feature information using fully connected layers and / or convolutional layers;

[0099] The second feature information is obtained by processing the first feature information and the speech domain information of the speech information to be processed using a gated recurrent unit layer;

[0100] The frequency domain amplitude gain is obtained by processing the second feature information using a dilated convolutional layer.

[0101] It is worth noting that in the above embodiments of the voice noise reduction device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0102] This application also provides a speech noise reduction device, which can integrate the speech noise reduction apparatus provided in this application. Figure 7 This is a schematic diagram of the structure of a speech noise reduction device provided in an embodiment of this application. (Reference) Figure 7 The speech noise reduction device includes: an input device 73, an output device 74, a memory 72, and one or more processors 71; the memory 72 is used to store one or more programs; when one or more programs are executed by one or more processors 71, the one or more processors 71 implement the speech noise reduction method provided in the above embodiments. The speech noise reduction device, apparatus, and computer provided above can be used to execute the speech noise reduction method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0103] This application also provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the speech denoising method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the speech denoising method provided above, but can also perform related operations in the speech denoising method provided in any embodiment of this application. The speech denoising apparatus, device, and storage medium provided in the above embodiments can execute the speech denoising method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the speech denoising method provided in any embodiment of this application.

[0104] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the speech noise reduction method provided in the various embodiments of this application.

Claims

1. A speech noise reduction method, characterized in that, include: Acquire the voice information to be processed; The speech information to be processed is input into the trained speech denoising model. The speech denoising model is used to obtain the subspace feature information of the speech information to be processed in multiple signal subspaces. Based on the weight coefficients of the speech information to be processed in different subspaces, the multiple subspace feature information is weighted and summed to obtain the denoising feature information. The denoised speech information is then generated based on the denoising feature information.

2. The speech noise reduction method according to claim 1, characterized in that, The speech denoising model, when acquiring subspace feature information of the speech information to be processed in multiple signal subspaces, includes: Obtain the audio domain information of the speech information to be processed; The amplitude feature information is obtained by processing the audio domain information using fully connected layers and / or convolutional layers; The amplitude feature information is obtained in the subspace feature information of multiple signal subspaces respectively.

3. The speech noise reduction method according to claim 2, characterized in that, The speech denoising model, when acquiring the subspace feature information of the amplitude feature information in multiple signal subspaces, includes: The amplitude feature information is sent to multiple subspace networks respectively, and the subspace feature information of the amplitude feature information in multiple signal subspaces is obtained by using the multiple subspace networks respectively.

4. The speech noise reduction method according to claim 1, characterized in that, Before the speech denoising model performs weighted summation of multiple subspace feature information based on the weight coefficients corresponding to the speech information to be processed in different subspaces to obtain denoised feature information, it further includes: The amplitude feature information corresponding to the speech information to be processed is sent to the weight acquisition network, and the weight acquisition network is used to obtain the weight coefficients corresponding to the amplitude feature information in multiple signal subspaces.

5. The speech noise reduction method according to claim 1, characterized in that, The speech denoising model, when generating denoised speech information based on the denoising feature information, includes: The noise reduction feature information is enhanced to obtain the frequency domain amplitude gain; Amplitude estimation information is determined based on the frequency domain amplitude gain and the amplitude feature information of the speech information to be processed; Noise-reduced speech information is generated based on the amplitude estimation information and the phase feature information of the speech information to be processed.

6. The speech noise reduction method according to claim 5, characterized in that, The speech denoising model, when enhancing the denoising feature information to obtain the frequency domain amplitude gain, includes: The first feature information is obtained by processing the noise reduction feature information using fully connected layers and / or convolutional layers; The second feature information is obtained by processing the first feature information and the speech domain information of the speech information to be processed using a gated loop unit layer; The frequency domain amplitude gain is obtained by processing the second feature information using a dilated convolutional layer.

7. A voice noise reduction device, characterized in that, It includes a data acquisition module and a noise reduction processing module, wherein: The data acquisition module is configured to acquire voice information to be processed; The noise reduction processing module is configured to input the speech information to be processed into a trained speech noise reduction model, obtain subspace feature information of the speech information to be processed in multiple signal subspaces through the speech noise reduction model, perform weighted summation processing on the multiple subspace feature information based on the weight coefficients corresponding to the speech information to be processed in different subspaces to obtain noise reduction feature information, and generate noise reduction speech information based on the noise reduction feature information.

8. A voice noise reduction device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the speech noise reduction method as described in any one of claims 1-6.

9. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the speech noise reduction method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech noise reduction method according to any one of claims 1-6.

Images