Training method and device of speech noise reduction model and speech noise reduction method and device

Abstract

The present disclosure relates to a training method and device for a speech denoising model and a speech denoising method and device. The training method includes: obtaining a noisy speech signal sample, the noisy speech signal sample is obtained by mixing multiple original speech signals and noise signals, each original speech signal has a distance value, and the distance value represents the distance between the original speech signal and the target microphone position; obtaining a preset first reserved distance value and a target denoised speech signal, the target denoised speech signal is calculated based on the distance value of each original speech signal and the preset first reserved distance value; inputting the amplitude spectrum of the noisy speech signal sample and the preset first reserved distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal; calculating the value of the loss function based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal; and training the speech denoising model based on the calculated value of the loss function.

Description

Technical Field

[0001] The present disclosure relates to the field of audio technology, and more specifically, to a method and device for training a speech noise reduction model and a method and device for speech noise reduction. Background Art

[0002] Audio noise reduction usually refers to the process of removing or weakening the noise part of an audio signal to obtain the desired audio signal. Audio noise reduction in a general sense mainly refers to removing or weakening the noise to obtain the desired signal. At present, the methods of audio noise reduction are mainly divided into traditional noise reduction algorithms and neural network-based noise reduction algorithms. Traditional noise reduction algorithms mainly refer to spectral subtraction, Wiener filtering and other algorithms. They often rely on the additivity of background noise or the statistical characteristics of the desired signal and noise signal. For unexpected noise types such as sudden noise that appear in actual environments, their performance often cannot meet actual needs. Therefore, considering the complexity of the noise destruction process, the noise reduction algorithm based on neural network has developed rapidly, showing obvious advantages in environments with low signal-to-noise ratio and non-stationary noise.

[0003] In conference scenarios, when holding a remote meeting, both parties are often disturbed by various environmental noises, so audio noise reduction becomes an important technology to improve call quality and experience in conference scenarios. However, existing traditional or deep learning noise reduction technologies can usually suppress various non-voice noises, but cannot suppress the interference of unwanted voices coming from a distance. Summary of the invention

[0004] The present disclosure provides a method and device for training a speech noise reduction model, and a method and device for speech noise reduction, so as to at least solve the problems in the above-mentioned related technologies, or may not solve any of the above-mentioned problems.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for training a speech denoising model is provided, comprising: obtaining a noisy speech signal sample, wherein the noisy speech signal sample is obtained by mixing a plurality of original speech signals and a noise signal, each original speech signal having a distance value, the distance value representing the distance between a sound source position of the original speech signal and a target microphone position for collecting the original speech signal; obtaining a preset first reserved distance value and a target denoised speech signal corresponding to the noisy speech signal sample, wherein the preset first reserved distance value represents retaining a speech signal within a range of the preset first reserved distance value from the target microphone position and suppressing speech signals outside the range, the target denoised speech signal The noisy speech signal is obtained by obtaining a retention factor of each original speech signal based on the distance value of each original speech signal and the preset first retention distance value, and weighted summing the multiple original speech signals by taking the retention factor of each original speech signal as the weight of each original speech signal; inputting the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain the estimated amplitude spectrum of the denoised speech signal; calculating the value of the loss function based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal; adjusting the parameters of the speech denoising model based on the calculated value of the loss function to train the speech denoising model.

[0006] Optionally, obtaining the retention factor of each original speech signal based on the distance value of each original speech signal and a preset first retention distance value may include: determining the retention factor of each original speech signal based on a comparison result of the distance value of each original speech signal and the preset first retention distance value.

[0007] Optionally, determining the retention factor of each original speech signal based on the comparison result of the distance value of each original speech signal with the preset first retention distance value may include: determining the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value as 1; determining the retention factor of the original speech signal whose distance value is greater than the preset first retention distance value as less than 1, wherein the larger the distance value, the smaller the retention factor.

[0008] Optionally, determining the retention factor of the original speech signal whose distance value is greater than the preset first retention distance value to be less than 1 may include: determining the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second distance retention value to be a value greater than 0 and less than 1, wherein the larger the distance value, the smaller the retention factor, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; determining the retention factor of the original speech signal whose distance value is above the preset second retention distance value to be 0.

[0009] Optionally, the retention factor of each original speech signal can be expressed as:

[0010]

[0011] Among them, m is the distance value of each original speech signal, m0 is the preset first reserved distance value, m0+a is the preset second reserved distance value, and a is the predetermined distance.

[0012] Optionally, the output of the speech denoising model may be an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample; wherein, inputting the amplitude spectrum of the noisy speech signal sample and the preset first retained distance value into the speech denoising model to obtain the estimated amplitude spectrum of the denoised speech signal may include: inputting the amplitude spectrum of the noisy speech signal sample and the preset first retained distance value into the speech denoising model to obtain the estimated mask ratio; multiplying the estimated mask ratio by the amplitude spectrum of the noisy speech signal sample to obtain the estimated amplitude spectrum of the denoised speech signal.

[0013] According to a second aspect of an embodiment of the present disclosure, a speech denoising method is provided, which is performed by a speech denoising model trained according to a training method of the present disclosure, wherein the speech denoising method comprises: obtaining the speech signal to be denoised at a target microphone for collecting the speech signal to be denoised in an audio and video conference scenario; obtaining the preset first retained distance value; inputting the amplitude spectrum of the speech signal to be denoised and the preset first retained distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal; obtaining an estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal; and inputting the estimated denoised speech signal into the target microphone.

[0014] Optionally, the output of the speech denoising model may be an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the speech signal to be denoised; wherein, inputting the amplitude spectrum of the speech signal to be denoised and the preset first retained distance value into the speech denoising model to obtain the estimated amplitude spectrum of the denoised speech signal may include: inputting the amplitude spectrum of the speech signal to be denoised and the preset first retained distance value into the speech denoising model to obtain the estimated mask ratio; multiplying the estimated mask ratio by the amplitude spectrum of the speech signal to be denoised to obtain the estimated amplitude spectrum of the denoised speech signal.

[0015] Optionally, obtaining the estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal may include: combining the amplitude spectrum of the estimated denoised speech signal and the phase spectrum of the speech signal to be denoised and performing an inverse time-frequency transform to obtain the estimated denoised speech signal.

[0016] Optionally, the speech signal to be denoised may be collected by a sound receiving device at the target microphone.

[0017] According to a third aspect of an embodiment of the present disclosure, a training device for a speech denoising model is provided, comprising: a first acquisition unit, configured to acquire a noisy speech signal sample, wherein the noisy speech signal sample is obtained by mixing a plurality of original speech signals and a noise signal, each original speech signal having a distance value, the distance value representing the distance between a sounding position of the original speech signal and a target microphone position for collecting the original speech signal; a second acquisition unit, configured to acquire a preset first retained distance value and a target denoised speech signal corresponding to the noisy speech signal sample, wherein the preset first retained distance value represents retaining a speech signal within a range of the preset first retained distance value from the target microphone position and suppressing speech signals outside the range, the target denoised speech signal The sound signal is obtained by obtaining a retention factor of each original speech signal based on the distance value of each original speech signal and the preset first retention distance value, and weighted summing the multiple original speech signals by taking the retention factor of each original speech signal as the weight of each original speech signal; the estimation unit is configured to input the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain the estimated amplitude spectrum of the denoised speech signal; the calculation unit is configured to calculate the value of the loss function based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal; the training unit is configured to adjust the parameters of the speech denoising model based on the calculated value of the loss function to train the speech denoising model.

[0018] Optionally, the retention factor of each original speech signal may be determined according to a comparison result between the distance value of each original speech signal and the preset first retention distance value.

[0019] Optionally, the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value can be determined as 1; the retention factor of the original speech signal whose distance value is greater than the preset first retention distance value can be determined as less than 1, wherein the larger the distance value, the smaller the retention factor.

[0020] Optionally, the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second distance retention value can be determined as a value greater than 0 and less than 1, wherein the larger the distance value, the smaller the retention factor, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; the retention factor of the original speech signal whose distance value is above the preset second retention distance value can be determined as 0.

[0021] Optionally, the retention factor of each original speech signal can be expressed as:

[0022]

[0023] Among them, m is the distance value of each original speech signal, m0 is the preset first reserved distance value, m0+a is the preset second reserved distance value, and a is the predetermined distance.

[0024] Optionally, the output of the speech denoising model may be an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample; the estimation unit may be configured to: input the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain the estimated mask ratio; multiply the estimated mask ratio by the amplitude spectrum of the noisy speech signal sample to obtain the estimated amplitude spectrum of the denoised speech signal.

[0025] According to a fourth aspect of an embodiment of the present disclosure, a speech denoising device is provided, wherein the speech denoising device performs operations by means of a speech denoising model trained according to the training method of the present disclosure, wherein the speech denoising device comprises: a first acquisition unit, configured to acquire the speech signal to be denoised at a target microphone for collecting the speech signal to be denoised in an audio and video conference scenario; a second acquisition unit, configured to acquire the preset first retained distance value; an estimation unit, configured to input the amplitude spectrum of the speech signal to be denoised and the preset first retained distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal; a calculation unit, configured to obtain an estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal; and an input unit, configured to input the estimated denoised speech signal into the target microphone.

[0026] Optionally, the output of the speech denoising model may be an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the speech signal to be denoised; wherein the estimation unit may be configured to: input the amplitude spectrum of the speech signal to be denoised and the preset first retained distance value into the speech denoising model to obtain the estimated mask ratio; multiply the estimated mask ratio by the amplitude spectrum of the speech signal to be denoised to obtain the estimated amplitude spectrum of the denoised speech signal.

[0027] Optionally, the signal acquisition unit may be configured to: combine the estimated amplitude spectrum of the denoised speech signal and the phase spectrum of the speech signal to be denoised and perform an inverse time-frequency transform to obtain the estimated denoised speech signal.

[0028] Optionally, the speech signal to be denoised may be collected by a sound receiving device at the target microphone.

[0029] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute a training method for a speech noise reduction model or a speech noise reduction method according to the present disclosure.

[0030] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute a training method for a speech noise reduction model or a speech noise reduction method according to the present disclosure.

[0031] According to a seventh aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising computer instructions, which, when executed by at least one processor, implement the training method of a speech denoising model or the speech denoising method according to the present disclosure.

[0032] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0033] According to the training method and training device of the speech noise reduction model as well as the speech noise reduction method and speech noise reduction device disclosed in the present invention, different characteristics of near and far speech, such as reverberation characteristics and energy characteristics, are utilized, different soft suppression factors are set according to the distance of the speech signal and the specific distance set by the user to generate training labels, and the speech noise reduction model is trained, so that the trained speech noise reduction model can perfectly retain the human voice within a specific distance from the microphone, and suppress the human voice outside the specific distance, and as the distance becomes farther, the suppression effect becomes stronger until it is completely eliminated.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0036] Figure 1 is a schematic diagram illustrating a training process and an inference process of a speech denoising model according to an exemplary embodiment of the present disclosure.

[0037] Figure 2 is a flowchart illustrating a method for training a speech denoising model according to an exemplary embodiment of the present disclosure.

[0038] Figure 3 is a flowchart illustrating a speech noise reduction method according to an exemplary embodiment of the present disclosure.

[0039] Figure 4 is a block diagram illustrating a training apparatus for a speech denoising model according to an exemplary embodiment of the present disclosure.

[0040] Figure 5 is a block diagram illustrating a speech noise reduction apparatus according to an exemplary embodiment of the present disclosure.

[0041] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0044] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0045] Usually, in a conference scenario, it is sufficient to ensure call quality within a certain range, and voice or other audio outside this range can be suppressed or eliminated. However, existing traditional or deep learning noise reduction technologies cannot distinguish the distance between the voice received by the microphone and the microphone in the case of a single microphone. Therefore, they can usually suppress various non-voice noises, but cannot suppress unwanted voices from far away.

[0046] In order to solve the above technical problems, the present invention discloses a distance-based audio noise reduction method in a conference scenario. Based on the original audio noise reduction, this method uses the different characteristics of near and far voices, such as reverberation characteristics and energy characteristics, to set different soft suppression factors according to the distance of the voice signal and the specific distance set by the user to generate training labels, and train the voice noise reduction model, so that the trained voice noise reduction model can perfectly retain the human voice within a specific distance from the microphone, and suppress the human voice outside the specific distance, and as the distance increases, the suppression effect becomes stronger until it is completely eliminated. Below, reference will be made to Figures 1 to 6 The training method and device of the speech denoising model and the speech denoising method and device according to the present disclosure are described in detail.

[0047] Figure 1 is a schematic diagram showing the training process and reasoning process of the speech denoising model according to an exemplary embodiment of the present disclosure. Figure 1 (a) in FIG. 1 shows the training process of the speech denoising model. Figure 1 (b) in FIG. 5 shows the reasoning process of the speech denoising model.

[0048] Reference Figure 1In (a), a speech signal sample can be obtained by mixing multiple original speech signals and noise signals of known distances. Here, the distance refers to the distance between the original speech signal and the target microphone used to collect the original speech signal. The retention factor g of the multiple original speech signals is set according to the distance of the multiple original speech signals and the retention distance m0 set by the user, and the multiple original speech signals are mixed based on the retention factor g to obtain a speech signal label. Here, the retention distance m0 refers to retaining the original speech signal within the m0 distance from the target microphone and suppressing the original speech signal outside the m0 distance from the target microphone. For example, the retention factor g of the original speech signal within the m0 distance from the target microphone is set to 1, and the retention factor g of the original speech signal outside the m0 distance from the target microphone is set to less than 1, and the greater the distance, the smaller the retention factor, until it is 0. The neural network model is trained by speech signal samples, retention distance m0 and speech signal labels to obtain a trained speech denoising model.

[0049] For example, the number of multiple original speech signals can be set to 3. Of course, the present disclosure does not limit the number of original speech signals, and the number of original speech signals can be set as needed. It is generally recommended that the number of original speech signals is less than or equal to 3, and if it is more, it is similar to babble. The original speech signal x1 with a known distance, the original speech signal x2 with a known distance, the original speech signal x3 with a known distance and the noise signal can be mixed to obtain a mixed signal, and the mixed signal is converted from the time domain to the time-frequency domain (for example, short-time Fourier transform (STFT)), and then the amplitude of the frequency domain signal is extracted as the training feature 1. After the original speech signals x1, x2, and x3 with known distances are suppressed g1, g2, and g3 respectively by setting the reserved distance m0, the mixed signal x1*g1+x2*g2+x3*g3 of the three is converted from the time domain to the time-frequency domain (for example, STFT)), and then the amplitude of the frequency domain signal is extracted as a label. In addition, the reserved distance m0 is used as the feature 2 required for the training of the speech denoising model. Feature 1, feature 2 and the label are sent to the speech denoising model for training to obtain the required speech denoising model.

[0050] Reference Figure 1 In (b), the denoised signal is converted from the time domain to the time-frequency domain (such as STFT), and then the amplitude spectrum of the frequency domain signal is sent to the trained speech denoising model together with the retained distance m0 to obtain the denoised signal.

[0051] According to the speech noise reduction model trained by the above training method, the human voice within a specific distance from the microphone can be perfectly preserved, and the human voice outside the specific distance can be suppressed. Moreover, as the distance increases, the suppression effect becomes stronger until it is completely eliminated.

[0052] Figure 2 is a flowchart illustrating a method for training a speech denoising model according to an exemplary embodiment of the present disclosure.

[0053] Reference Figure 2 , in step 201, a noisy speech signal sample can be obtained. Here, the noisy speech signal sample is obtained by mixing multiple original speech signals and noise signals, each original speech signal having a distance value, which represents the distance between the sound position of the original speech signal and the target microphone position used to collect the original speech signal. For example, assuming that there are d original speech signals x1, x2, ..., xd and corresponding known distance values ​​m1, m2, ..., md, the d original speech signals x1, x2, ..., xd and the noise signal can be mixed to obtain a noisy speech signal sample y. Assuming that the original speech signals x1, x2, ..., xd of length T can be x1(t), x2(t), ..., xd(t) in the time domain, respectively, therefore, the noisy speech signal sample y can be y(t) in the time domain, where t represents time, 0<t≤T.

[0054] According to an exemplary embodiment of the present disclosure, the number of original speech signals in the noisy speech signal sample is set to, but not limited to, 3. The present disclosure does not limit the number of original speech signals, and the number of original speech signals can be set as needed.

[0055] According to an exemplary embodiment of the present disclosure, the original voice signal can be collected by a sound receiving device at a target microphone. For example, an audio and video conference scene can be simulated, a sound receiving device is set at a target microphone for receiving voice signals, voice signals are sent at various distances from the target microphone, and these voice signals are collected by the sound receiving device, thereby obtaining multiple original voice signals with various distance values, and recording the distance value of each original voice signal.

[0056] According to an exemplary embodiment of the present disclosure, the noise signal can be obtained by downloading from the Internet, actual recording, etc., and a plurality of original speech signals and the noise signal can be added in the time domain according to a certain signal-to-noise ratio to generate a noisy speech signal sample.

[0057] In step 202, a preset first retention distance value and a target noise reduction speech signal corresponding to the noisy speech signal sample may be obtained. Here, the preset first retention distance value means retaining the speech signal within the preset first retention distance value range from the target microphone position and suppressing the speech signal outside the range. The retention factor of each original speech signal may be obtained based on the distance value of each original speech signal and the preset first retention distance value, and the retention factor of each original speech signal may be used as the weight of each original speech signal to perform weighted summation on the multiple original speech signals to obtain the retention factor. For example, based on the distance values ​​m1, m2, ..., md of each original speech signal and the preset first reserved distance value m0, the retention factors g(m1, m0), g(m2, m0), ..., g(md, m0) can be calculated, and the retention factors g(m1, m0), g(m2, m0), ..., g(md, m0) can be used as weights to perform weighted summation on the d original speech signals x1, x2, ..., xd to obtain the desired target noise reduction speech signal x. For example, the target noise reduction speech signal x can be expressed as:

[0058] x(t)=x1(t)*g(m1,m0)+x2(t)*g(m2,m0)+…+xd(t)*g(mn,m0)

[0059] According to an exemplary embodiment of the present disclosure, each original voice signal can be softly suppressed using a preset first retention distance value, that is, the retention factor of each original voice signal can be determined based on a comparison result of the distance value of each original voice signal with the preset first retention distance value. In principle, it is desirable to retain the original voice signal whose distance value is within the preset first retention distance value, and to suppress the original voice signal whose distance value is outside the preset first retention distance value. Therefore, the retention factor of the original voice signal whose distance value is within the preset first retention distance value can be set to be greater than the original voice signal whose distance value is outside the preset first retention distance value.

[0060] For example, the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value can be determined as 1. The retention factor of the original speech signal whose distance value is greater than the preset first retention distance value can be determined as less than 1, wherein the larger the distance value, the smaller the retention factor. In other words, the original speech signal whose distance value is within the preset first retention distance value is completely retained, while the original speech signal whose distance value is outside the preset first retention distance value is suppressed, and the larger the distance value, the more suppressed.

[0061] For example, the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second retention distance value can be determined as a value greater than 0 and less than 1, wherein the larger the distance value, the smaller the retention factor, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; and the retention factor of the original speech signal whose distance value is above the preset second retention distance value is determined as 0. That is, the original speech signal whose distance value is within the preset first retention distance value is completely retained, while the original speech signal whose distance value is between the preset first retention distance value and the preset second retention distance value is partially suppressed, and the original speech signal whose distance value is above the preset second retention distance value is completely suppressed.

[0062] For example, the preservation factor of the original speech signal is expressed as:

[0063]

[0064] Wherein, m is the distance value of each original speech signal, m0 is the preset first reserved distance value, m0+a is the preset second reserved distance value, and a is the predetermined distance. For example, a can be set to, but not limited to, 1.

[0065] In step 203, the amplitude spectrum of the noisy speech signal sample and the preset first reserved distance value can be input into the speech denoising model to obtain the amplitude spectrum of the estimated denoised speech signal. Here, the speech denoising model can be implemented by any possible neural network model. For example, the noisy speech signal sample can be transformed from the time domain to the time-frequency domain (for example, STFT), and y(t) can be expressed in the time-frequency domain as:

[0066] Y(n,k)=STFT(y(t))

[0067] Wherein, n is the frame sequence, 0<n≤N, N is the total number of frames, k is the center frequency sequence 0<k≤K, K is the total number of frequency points.

[0068] The amplitude Mag of the signal Y(n,k) of the noisy speech signal sample in the time-frequency domain can be taken to obtain the amplitude spectrum MagY(n,k), which can be expressed as:

[0069] MagY(n,k)=abs(Y(n,k))

[0070] Subsequently, the amplitude spectrum MagY(n, k) of the noisy speech signal sample and the preset first reserved distance value m0 are sent to the speech noise reduction model noise reduce() to obtain the estimated amplitude spectrum MagX0(n, k) of the noise-reduced speech signal, which can be expressed as:

[0071] MagX0(n,k)=noise reduce(MagY(n,k),m0)

[0072] According to an exemplary embodiment of the present disclosure, the amplitude spectrum of the noisy speech signal sample can be used as an input of the speech denoising model, and the preset first reserved distance value can be input into the speech denoising model according to any position set by the user in the speech denoising model. For example, the amplitude spectrum of the noisy speech signal sample can be spliced ​​with the preset first reserved distance value and then input into the speech denoising model together. For another example, the preset first reserved distance value can be input into the middle layer or output layer of the speech denoising model.

[0073] According to an exemplary embodiment of the present disclosure, the output of the speech denoising model may be an estimated amplitude spectrum of the denoised speech signal or an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample. In the case where the output of the speech denoising model is an estimated mask ratio, the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value may be input into the speech denoising model to obtain the estimated mask ratio; and the estimated mask ratio is multiplied by the amplitude spectrum of the noisy speech signal sample to obtain the estimated amplitude spectrum of the denoised speech signal.

[0074] In step 204, the value of the loss function may be calculated based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal. For example, if the target denoised speech signal x with a length of T is x(t) in the time domain, where t represents time, 0<t≤T, then after short-time Fourier transform, x(t) may be expressed in the time-frequency domain as:

[0075] X(n,k)=STFT(x(t))

[0076] Wherein, n is the frame sequence index, 0<n≤N, N is the total number of frames, k is the center frequency sequence index, 0<k≤K, K is the total number of frequency points.

[0077] According to an exemplary embodiment of the present disclosure, the mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) between the amplitude spectrum of the estimated denoised speech signal and the amplitude spectrum of the target denoised speech signal can be calculated as the value of the loss function. Of course, the present disclosure does not limit the loss function, and any possible loss function can be used to train the speech denoising model of the present disclosure.

[0078] In step 205, the parameters of the speech denoising model may be adjusted based on the calculated value of the loss function to train the speech denoising model. That is, the parameters of the speech denoising model may be adjusted by back propagation of the loss calculated by the loss function. In addition, during the model training process, batches of noisy speech signal samples may be used to adjust (or update) the parameters of the speech denoising model, and the parameters of the speech denoising model may be iteratively adjusted (or updated) with the goal of minimizing the value of the loss function until the speech denoising model converges.

[0079] Figure 3 is a flow chart showing a method for reducing noise of speech according to an exemplary embodiment of the present disclosure. Here, the method for reducing noise of speech according to an exemplary embodiment of the present disclosure can be described in detail with reference to the above. Figure 2 The speech noise reduction model trained by the training method is executed.

[0080] In step 301, a voice signal to be denoised may be obtained at a target microphone used to collect voice signals to be denoised in a video conferencing scenario. For example, in an audio and video conferencing scenario, the target microphone is used to receive voice signals to be sent to other terminals participating in the audio and video conference via a network. Therefore, before the target microphone receives the voice signal, the voice signal may be denoised first, and then the denoised voice signal may be input into the target microphone, thereby improving the voice signal quality of the audio and video conference. Therefore, the voice signal to be denoised may be collected by a sound receiving device at the target microphone, and then the voice signal to be denoised may be denoised according to the voice denoising method disclosed in the present invention to obtain a denoised voice signal, and the denoised voice signal may be input into the target microphone and sent to other terminals participating in the audio and video conference via a network.

[0081] In step 302, a preset first reserved distance value may be obtained. The preset first reserved distance value may be a preset first reserved distance value used during training.

[0082] In step 303, the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value may be input into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal.

[0083] According to an exemplary embodiment of the present disclosure, the amplitude spectrum of the speech signal to be denoised can be used as an input of the speech denoising model, and the preset first reserved distance value can be input into the speech denoising model according to any position set by the user in the speech denoising model. For example, the amplitude spectrum of the speech signal to be denoised can be spliced ​​with the preset first reserved distance value, and then input into the speech denoising model together. For another example, the preset first reserved distance value can be input into the middle layer or output layer of the speech denoising model.

[0084] According to an exemplary embodiment of the present disclosure, the output of the speech denoising model may be an estimated amplitude spectrum of the denoised speech signal or an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the speech signal to be denoised. In the case where the output of the speech denoising model is an estimated mask ratio, the amplitude spectrum of the speech signal to be denoised and a preset first retention distance value may be input into the speech denoising model to obtain an estimated mask ratio; and the estimated mask ratio may be multiplied by the amplitude spectrum of the speech signal to be denoised to obtain an estimated amplitude spectrum of the denoised speech signal.

[0085] In step 304, an estimated denoised speech signal may be obtained based on the estimated amplitude spectrum of the denoised speech signal.

[0086] According to an exemplary embodiment of the present disclosure, the amplitude spectrum of the estimated noise-reduced speech signal and the phase spectrum of the speech signal to be noise-reduced may be combined and an inverse time-frequency transform (e.g., inverse short-time Fourier transform (ISTFT)) may be performed to obtain the estimated noise-reduced speech signal.

[0087] For example, if the speech signal y' to be noise-reduced with a length of T is y'(t) in the time domain, where t represents time and 0 < t ≤ T, after the short-time Fourier transform, y'(t) can be represented in the time-frequency domain as:

[0088] Y'(n,k) = STFT(y'(t))

[0089] where n is the frame sequence index, 0 < n ≤ N, N is the total number of frames, k is the center frequency sequence index, 0 < k ≤ K, and K is the total number of frequency points.

[0090] Subsequently, the amplitude Mag and phase Pha of the speech signal y' to be noise-reduced are taken respectively:

[0091] MagY'(n,k) = abs(Y'(n,k))

[0092] PhaY'(n,k) = Y'(n,k) / abs(Y'(n,k))

[0093] Subsequently, the amplitude spectrum MagY'(n,k) of the speech signal y' to be noise-reduced and a preset first retention distance value m0 are input into the speech noise reduction model noise reduce(), and the amplitude spectrum MagX'(n,k) of the estimated noise-reduced speech signal is obtained, which can be expressed as:

[0094] MagX'(n,k) = noise reduce(MagY'(n,k),m0)

[0095] Subsequently, the amplitude spectrum MagX'(n,k) of the estimated noise-reduced speech signal and the phase spectrum PhaY'(n,k) of the speech signal y' to be noise-reduced are combined and ISTFT is performed to obtain the estimated noise-reduced speech signal X'(t), which can be expressed as:

[0096] X'(t) = ISTFT(MagX'(n,k)*PhaY'(n,k))

[0097] In step 305, the estimated noise-reduced speech signal may be input into the target microphone for transmission to other terminals participating in the audio-video conference through the network.

[0098] Figure 4 is a block diagram showing a training apparatus of a speech noise reduction model according to an exemplary embodiment of the present disclosure.

[0099] Reference Figure 4 According to an exemplary embodiment of the present disclosure, a training device 400 for a speech denoising model may include a first acquiring unit 401, a second acquiring unit 402, an estimating unit 403, a calculating unit 404 and a training unit 405.

[0100] The first acquisition unit 401 may acquire a noisy speech signal sample. Here, the noisy speech signal sample is obtained by mixing a plurality of original speech signals and a noise signal, each original speech signal having a distance value, which indicates the distance between the sounding position of the original speech signal and the target microphone position for collecting the original speech signal.

[0101] According to an exemplary embodiment of the present disclosure, the number of original speech signals in the noisy speech signal sample is set to, but not limited to, 3. The present disclosure does not limit the number of original speech signals, and the number of original speech signals can be set as needed.

[0102] According to an exemplary embodiment of the present disclosure, the original voice signal can be collected by a sound receiving device at a target microphone. For example, an audio and video conference scene can be simulated, a sound receiving device is set at a target microphone for receiving voice signals, voice signals are sent at various distances from the target microphone, and these voice signals are collected by the sound receiving device, thereby obtaining multiple original voice signals with various distance values, and recording the distance value of each original voice signal.

[0103] According to an exemplary embodiment of the present disclosure, the noise signal can be obtained by downloading from the Internet, actual recording, etc., and a plurality of original speech signals and the noise signal can be added in the time domain according to a certain signal-to-noise ratio to generate a noisy speech signal sample.

[0104] The second acquisition unit 402 can acquire a preset first retention distance value and a target noise reduction speech signal corresponding to the noisy speech signal sample. Here, the preset first retention distance value means retaining the speech signal within the preset first retention distance value range from the target microphone position and suppressing the speech signal outside the range. The retention factor of each original speech signal can be obtained based on the distance value of each original speech signal and the preset first retention distance value, and the retention factor of each original speech signal is used as the weight of each original speech signal to perform weighted summation on the multiple original speech signals to obtain the retention factor.

[0105] According to an exemplary embodiment of the present disclosure, each original voice signal can be softly suppressed using a preset first retention distance value, that is, the retention factor of each original voice signal can be determined based on a comparison result of the distance value of each original voice signal with the preset first retention distance value. In principle, it is desirable to retain the original voice signal whose distance value is within the preset first retention distance value, and to suppress the original voice signal whose distance value is outside the preset first retention distance value. Therefore, the retention factor of the original voice signal whose distance value is within the preset first retention distance value can be set to be greater than the original voice signal whose distance value is outside the preset first retention distance value.

[0106] For example, the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value can be determined as 1. The retention factor of the original speech signal whose distance value is greater than the preset first retention distance value can be determined as less than 1, wherein the larger the distance value, the smaller the retention factor. In other words, the original speech signal whose distance value is within the preset first retention distance value is completely retained, while the original speech signal whose distance value is outside the preset first retention distance value is suppressed, and the larger the distance value, the more suppressed.

[0107] For example, the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second retention distance value can be determined as a value greater than 0 and less than 1, wherein the larger the distance value, the smaller the retention factor, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; and the retention factor of the original speech signal whose distance value is above the preset second retention distance value is determined as 0. That is, the original speech signal whose distance value is within the preset first retention distance value is completely retained, while the original speech signal whose distance value is between the preset first retention distance value and the preset second retention distance value is partially suppressed, and the original speech signal whose distance value is above the preset second retention distance value is completely suppressed.

[0108] For example, the preservation factor of the original speech signal is expressed as:

[0109]

[0110] Wherein, m is the distance value of each original speech signal, m0 is the preset first reserved distance value, m0+a is the preset second reserved distance value, and a is the predetermined distance. For example, a can be set to, but not limited to, 1.

[0111] The estimation unit 403 may input the amplitude spectrum of the noisy speech signal sample and the preset first reserved distance value into the speech denoising model to obtain the estimated amplitude spectrum of the denoised speech signal. Here, the speech denoising model may be implemented by any possible neural network model.

[0112] According to an exemplary embodiment of the present disclosure, the amplitude spectrum of the noisy speech signal sample can be used as an input of the speech denoising model, and the preset first reserved distance value can be input into the speech denoising model according to any position set by the user in the speech denoising model. For example, the amplitude spectrum of the noisy speech signal sample can be spliced ​​with the preset first reserved distance value and then input into the speech denoising model together. For another example, the preset first reserved distance value can be input into the middle layer or output layer of the speech denoising model.

[0113] According to an exemplary embodiment of the present disclosure, the output of the speech denoising model may be an estimated amplitude spectrum of the denoised speech signal or an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample. In the case where the output of the speech denoising model is an estimated mask ratio, the estimation unit 403 may input the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain an estimated mask ratio; and multiply the estimated mask ratio by the amplitude spectrum of the noisy speech signal sample to obtain an estimated amplitude spectrum of the denoised speech signal.

[0114] The calculation unit 404 may calculate a value of the loss function based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal.

[0115] According to an exemplary embodiment of the present disclosure, the calculation unit 404 can calculate the mean square error (MSE), root mean square error (RMSE) or mean absolute error (MAE) between the amplitude spectrum of the estimated denoised speech signal and the amplitude spectrum of the target denoised speech signal as the value of the loss function. Of course, the present disclosure does not limit the loss function, and any possible loss function can be used to train the speech denoising model of the present disclosure.

[0116] The training unit 405 can adjust the parameters of the speech denoising model based on the calculated value of the loss function to train the speech denoising model. That is, the training unit 405 can adjust the parameters of the speech denoising model by back propagating the loss calculated by the loss function. In addition, during the model training process, the training unit 405 can use batches of noisy speech signal samples to adjust (or update) the parameters of the speech denoising model, and iteratively adjust (or update) the parameters of the speech denoising model with the goal of minimizing the value of the loss function until the speech denoising model converges.

[0117] Figure 5 is a block diagram showing a speech noise reduction device according to an exemplary embodiment of the present disclosure. Here, the speech noise reduction device according to the exemplary embodiment of the present disclosure can be based on the above reference Figure 2 The speech noise reduction model trained by the training method is used to perform operations.

[0118] Reference Figure 5According to an exemplary embodiment of the present disclosure, a speech noise reduction device 500 may include a first acquisition unit 501, a second acquisition unit 502, an estimation unit 503, a calculation unit 504 and an input unit 505.

[0119] The first acquisition unit 501 can acquire the voice signal to be denoised at the target microphone for collecting the voice signal to be denoised in the video conferencing scenario. For example, in an audio and video conferencing scenario, the target microphone is used to receive the voice signal to be sent to other terminals participating in the audio and video conference through the network. Therefore, before the target microphone receives the voice signal, the voice signal can be denoised first, and then the denoised voice signal can be input into the target microphone, thereby improving the voice signal quality of the audio and video conference. Therefore, the voice signal to be denoised can be collected by a sound receiving device at the target microphone, and then the voice signal to be denoised can be denoised according to the voice denoising method disclosed in the present invention to obtain a denoised voice signal, and the denoised voice signal can be input into the target microphone and sent to other terminals participating in the audio and video conference through the network.

[0120] The second acquisition unit 502 may acquire a preset first reserved distance value. The preset first reserved distance value may be a preset first reserved distance value used during training.

[0121] The estimation unit 503 may input the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal.

[0122] According to an exemplary embodiment of the present disclosure, the amplitude spectrum of the speech signal to be denoised can be used as an input of the speech denoising model, and the preset first reserved distance value can be input into the speech denoising model according to any position set by the user in the speech denoising model. For example, the amplitude spectrum of the speech signal to be denoised can be spliced ​​with the preset first reserved distance value, and then input into the speech denoising model together. For another example, the preset first reserved distance value can be input into the middle layer or output layer of the speech denoising model.

[0123] According to an exemplary embodiment of the present disclosure, the output of the speech denoising model may be an estimated amplitude spectrum of the denoised speech signal or an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the speech signal to be denoised. In the case where the output of the speech denoising model is an estimated mask ratio, the estimation unit 503 may input the amplitude spectrum of the speech signal to be denoised and the preset first retention distance value into the speech denoising model to obtain an estimated mask ratio; and multiply the estimated mask ratio by the amplitude spectrum of the speech signal to be denoised to obtain an estimated amplitude spectrum of the denoised speech signal.

[0124] The calculation unit 504 may obtain an estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal.

[0125] According to an exemplary embodiment of the present disclosure, the estimated amplitude spectrum of the denoised speech signal and the phase spectrum of the speech signal to be denoised may be combined and an inverse time-frequency transform (eg, short-time Fourier transform (ISTFT)) may be performed to obtain an estimated denoised speech signal.

[0126] The input unit 505 may input the estimated noise reduction speech signal into the target microphone for sending to other terminals participating in the audio and video conference via the network.

[0127] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment of the present disclosure.

[0128] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 1202, wherein the at least one memory 601 stores a set of computer executable instructions. When the computer executable instruction set is executed by the at least one processor 602, a training method for a speech denoising model or a speech denoising method according to an exemplary embodiment of the present disclosure is executed.

[0129] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0130] In the electronic device 600, the processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0131] The processor 602 may execute instructions or codes stored in the memory 601, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0132] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1201 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 602 can read files stored in the memory.

[0133] In addition, the electronic device 600 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0134] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the training method of the speech noise reduction model or the speech noise reduction method according to the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0135] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including computer instructions, which may be executed by at least one processor to complete the training method of the speech denoising model or the speech denoising method according to the exemplary embodiment of the present disclosure.

[0136] According to the training method and training device of the speech noise reduction model as well as the speech noise reduction method and speech noise reduction device disclosed in the present invention, different characteristics of near and far speech, such as reverberation characteristics and energy characteristics, are utilized, different soft suppression factors are set according to the distance of the speech signal and the specific distance set by the user to generate training labels, and the speech noise reduction model is trained, so that the trained speech noise reduction model can perfectly retain the human voice within a specific distance from the microphone, and suppress the human voice outside the specific distance, and as the distance becomes farther, the suppression effect becomes stronger until it is completely eliminated.

[0137] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0138] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a speech noise reduction model, characterized in that: include: Acquire a noisy speech signal sample, wherein the noisy speech signal sample is obtained by mixing a plurality of original speech signals and a noise signal, each original speech signal having a distance value, the distance value indicating the distance between a sounding position of the original speech signal and a target microphone position for collecting the original speech signal, the distance value being recorded when the original speech signal is collected; Acquire a preset first retention distance value and a target noise reduction speech signal corresponding to the noisy speech signal sample, wherein the preset first retention distance value indicates that the speech signal within the preset first retention distance value range from the target microphone position is retained and the speech signal outside the range is suppressed; the target noise reduction speech signal is obtained by obtaining a retention factor of each original speech signal based on the distance value of each original speech signal and the preset first retention distance value, and performing weighted summation on the multiple original speech signals by taking the retention factor of each original speech signal as the weight of each original speech signal; Inputting the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal, wherein the speech denoising model is used to suppress speech outside the preset first retention distance value by utilizing different characteristics of nearby and distant speech; Calculating a value of a loss function based on the estimated amplitude spectrum of the denoised speech signal and the amplitude spectrum of the target denoised speech signal; The parameters of the speech denoising model are adjusted based on the calculated value of the loss function to train the speech denoising model.

2. The training method according to claim 1, characterized in that: The method of obtaining a retention factor of each original speech signal based on the distance value of each original speech signal and a preset first retention distance value includes: The retention factor of each original speech signal is determined according to the comparison result of the distance value of each original speech signal and the preset first retention distance value.

3. The training method according to claim 2, characterized in that: The step of determining the retention factor of each original speech signal according to a comparison result between the distance value of each original speech signal and the preset first retention distance value comprises: Determine the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value as 1; The retention factor of the original speech signal whose distance value is greater than the preset first retention distance value is determined to be less than 1, wherein the larger the distance value is, the smaller the retention factor is.

4. The training method according to claim 3, characterized in that: The step of determining the retention factor of the original speech signal whose distance value is greater than the preset first retention distance value to be less than 1 includes: Determine the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second retention distance value as a value greater than 0 and less than 1, wherein the larger the distance value is, the smaller the retention factor is, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; The retention factor of the original speech signal whose distance value is greater than the preset second retention distance value is determined to be 0.

5. The training method according to claim 4, characterized in that: The retention factor of each original speech signal is expressed as: in, is the distance value of each original speech signal, is the preset first reserved distance value, is the preset second reserved distance value, is the predetermined distance.

6. The training method according to claim 1, characterized in that: The output of the speech denoising model is an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample; The step of inputting the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal comprises: Inputting the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain the estimated mask ratio; The estimated mask ratio is multiplied by the amplitude spectrum of the noisy speech signal sample to obtain the estimated amplitude spectrum of the denoised speech signal.

7. A speech noise reduction method, characterized in that: The speech denoising method is performed by using a speech denoising model trained by the training method according to any one of claims 1 to 6, wherein the speech denoising method comprises: Acquire the speech signal to be de-noised at a target microphone for collecting the speech signal to be de-noised in an audio and video conference scenario; Obtaining the preset first reserved distance value; Inputting the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal; Obtaining an estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal; The estimated noise-reduced speech signal is input to the target microphone.

8. The speech noise reduction method according to claim 7, characterized in that: The output of the speech denoising model is an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the to-be-denoised speech signal; The step of inputting the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal comprises: Inputting the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain the estimated mask ratio; The estimated mask ratio is multiplied by the amplitude spectrum of the speech signal to be denoised to obtain the estimated amplitude spectrum of the denoised speech signal.

9. The speech noise reduction method according to claim 7, characterized in that: The step of obtaining an estimated denoised speech signal based on the estimated amplitude spectrum of the denoised speech signal comprises: The estimated amplitude spectrum of the denoised speech signal and the phase spectrum of the speech signal to be denoised are combined and an inverse time-frequency transform is performed to obtain the estimated denoised speech signal.

10. The speech noise reduction method according to claim 7, characterized in that: The speech signal to be de-noised is collected by a sound receiving device at the target microphone.

11. A training device for a speech noise reduction model, characterized in that: include: A first acquisition unit is configured to acquire a noisy speech signal sample, wherein the noisy speech signal sample is obtained by mixing a plurality of original speech signals and a noise signal, each original speech signal having a distance value, the distance value indicating a distance between a sounding position of the original speech signal and a target microphone position for collecting the original speech signal, the distance value being recorded when the original speech signal is collected; A second acquisition unit is configured to acquire a preset first retention distance value and a target noise reduction speech signal corresponding to the noisy speech signal sample, wherein the preset first retention distance value represents retaining a speech signal within the preset first retention distance value range from the target microphone position and suppressing speech signals outside the range, and the target noise reduction speech signal is obtained by obtaining a retention factor of each original speech signal based on the distance value of each original speech signal and the preset first retention distance value, and by taking the retention factor of each original speech signal as the weight of each original speech signal to weighted sum the multiple original speech signals; an estimating unit, configured to input the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal, wherein the speech denoising model is used to suppress speech outside the preset first retention distance value by utilizing different characteristics of near and far speech; A calculation unit, configured to calculate a value of a loss function based on the estimated amplitude spectrum of the noise-reduced speech signal and the amplitude spectrum of the target noise-reduced speech signal; A training unit is configured to adjust the parameters of the speech denoising model based on the calculated value of the loss function to train the speech denoising model.

12. The training device according to claim 11, characterized in that The retention factor of each original speech signal is determined according to a comparison result between the distance value of each original speech signal and the preset first retention distance value.

13. The training device according to claim 12, characterized in that Determine the retention factor of the original speech signal whose distance value is equal to or less than the preset first retention distance value as 1; The retention factor of the original speech signal whose distance value is greater than the preset first retention distance value is determined to be less than 1, wherein the larger the distance value is, the smaller the retention factor is.

14. The training device according to claim 13, characterized in that Determine the retention factor of the original speech signal whose distance value is within the range of the preset first retention distance value to the preset second retention distance value as a value greater than 0 and less than 1, wherein the larger the distance value is, the smaller the retention factor is, and the preset second retention distance value is greater than the preset first retention distance value by a predetermined distance; The retention factor of the original speech signal whose distance value is greater than the preset second retention distance value is determined to be 0.

15. The training device according to claim 14, characterized in that The retention factor of each original speech signal is expressed as: in, is the distance value of each original speech signal, is the preset first reserved distance value, is the preset second reserved distance value, is the predetermined distance.

16. The training device according to claim 11, characterized in that The output of the speech denoising model is an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the noisy speech signal sample; The estimation unit is configured as: Inputting the amplitude spectrum of the noisy speech signal sample and the preset first retention distance value into the speech denoising model to obtain the estimated mask ratio; The estimated mask ratio is multiplied by the amplitude spectrum of the noisy speech signal sample to obtain the estimated amplitude spectrum of the denoised speech signal.

17. A speech noise reduction device, characterized in that: The speech denoising device is operated by a speech denoising model trained by the training method according to any one of claims 1 to 6, wherein the speech denoising device comprises: A first acquisition unit is configured to acquire the speech signal to be denoised at a target microphone for collecting the speech signal to be denoised in an audio and video conference scenario; A second acquisition unit is configured to acquire the preset first reserved distance value; an estimating unit, configured to input the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain an estimated amplitude spectrum of the denoised speech signal; a calculation unit configured to obtain an estimated noise-reduced speech signal based on the estimated amplitude spectrum of the noise-reduced speech signal; An input unit is configured to input the estimated noise reduction speech signal into the target microphone.

18. The speech noise reduction device according to claim 17, characterized in that: The output of the speech denoising model is an estimated mask ratio, wherein the mask ratio represents the ratio of the amplitude spectrum of the target denoised speech signal to the amplitude spectrum of the to-be-denoised speech signal; Among them, the estimation unit is configured as: Inputting the amplitude spectrum of the speech signal to be denoised and the preset first reserved distance value into the speech denoising model to obtain the estimated mask ratio; The estimated mask ratio is multiplied by the amplitude spectrum of the speech signal to be denoised to obtain the estimated amplitude spectrum of the denoised speech signal.

19. The speech noise reduction device according to claim 17, characterized in that: The signal acquisition unit is configured as follows: The estimated amplitude spectrum of the denoised speech signal and the phase spectrum of the speech signal to be denoised are combined and an inverse time-frequency transform is performed to obtain the estimated denoised speech signal.

20. The speech noise reduction device according to claim 17, characterized in that: The speech signal to be de-noised is collected by a sound receiving device at the target microphone.

21. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, Wherein, when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the training method of the speech denoising model as described in any one of claims 1 to 6 or the speech denoising method as described in any one of claims 7 to 10.

22. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the training method of the speech denoising model as described in any one of claims 1 to 6 or the speech denoising method as described in any one of claims 7 to 10.

23. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by at least one processor, the training method of the speech denoising model according to any one of claims 1 to 6 or the speech denoising method according to any one of claims 7 to 10 is implemented.

Images