A speech noise reduction method, electronic device and medium

Abstract

The present application provides a speech noise reduction method, electronic device and medium, the method comprising: obtaining noisy speech including original speech and noise, then selecting a corresponding speech noise reduction mode according to the noise intensity of the noisy speech, and then performing noise reduction on the noisy speech based on the speech noise reduction mode to obtain the original speech. Among them, by determining the noise intensity of the noisy speech to determine the strength of the noise in the noisy speech, then selecting different speech noise reduction modes according to the noise intensity, and performing noise reduction on the noisy speech according to the selected speech noise reduction mode, adaptive adjustment of speech noise reduction is achieved, waste of system resources is reduced, and the delay of speech interaction is reduced.

Description

[0001] This application is a divisional application for the Chinese patent application with application number 202311400580.9, application date October 26, 2023, and invention name “A speech noise reduction method, electronic device and medium”. Technical Field

[0002] The present application relates to the field of audio processing technology, and in particular to a speech noise reduction method, electronic equipment and medium. Background Art

[0003] With the development of artificial intelligence, voice interaction (such as voice wake-up and voice recognition) is widely used in mobile devices or wearable devices (such as tablets, mobile phones, and smart watches). Taking mobile phones as an example, users can wake up the mobile assistant through specific voice content, or the device can convert the user's input voice into corresponding text through voice recognition applications.

[0004] During the voice interaction process, the input voice may be affected by the noisy environment, resulting in the original voice and noise. If the input voice contains noise, it may result in a low wake-up rate of voice wake-up or a low recognition rate of voice recognition. In order to solve this technical problem, the relevant technology generally uses a voice noise reduction model to perform noise reduction processing on the input voice to remove the impact of noise on voice interaction.

[0005] However, when the device performs noise reduction on noisy speech, system resources are wasted and a high delay occurs during voice interaction. Summary of the invention

[0006] The speech noise reduction method, electronic device and medium provided in the present application solve the problem of wasting system resources and high voice interaction delay when the device performs noise reduction on noisy speech.

[0007] In order to achieve the above objectives, this application adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a speech denoising method, comprising: obtaining a first noisy speech including a first original speech and a first noise, then determining a first signal-to-noise ratio corresponding to the first noisy speech according to a first noise intensity of the first noisy speech, further determining a first number of speech enhancement modules in a speech denoising model based on the first signal-to-noise ratio, and then using the first number of speech enhancement modules to denoise the first noisy speech to obtain the first original speech; obtaining a second noisy speech including a second original speech and a second noise, and then determining a second signal-to-noise ratio corresponding to the second noisy speech according to a second noise intensity of the second noisy speech, wherein the second noise intensity is higher than the first noise intensity, and the second signal-to-noise ratio is lower than the first signal-to-noise ratio; then determining a second number of speech enhancement modules in the speech denoising model based on the second signal-to-noise ratio, wherein the second number is greater than the first number; and further using the second number of speech enhancement modules to denoise the second noisy speech to obtain the second original speech.

[0009] It should be noted that in the embodiment of the present application, the corresponding signal-to-noise ratio (first signal-to-noise ratio and second signal-to-noise ratio) is determined by noisy speech (first noisy speech and second noisy speech) with different noise intensities, so that different numbers of speech enhancement modules are selected according to different signal-to-noise ratios, and the corresponding number of speech enhancement modules are used to perform noise reduction, so as to obtain the original speech (first speech and second speech), realize adaptive adjustment of speech noise reduction, reduce waste of system resources, and reduce the delay of voice interaction.

[0010] As an example, the speech noise reduction model includes multiple speech enhancement modules connected in series; wherein, the input of the i+1th speech enhancement module is the output of the ith speech enhancement module, and i is a positive integer greater than or equal to 1; when i=0, the input of the first speech enhancement module is the first noisy speech or the second noisy speech. That is, the speech enhancement modules in the speech noise reduction model in the embodiment of the present application are connected in series, and the output of the previous speech enhancement module is used as the input of the next speech enhancement module to achieve different degrees of noise reduction for noisy speech, that is, the noise intensity of each segment of noisy speech may be different, and the number of corresponding speech enhancement modules is also different. By processing the noisy speech with the number of speech enhancement modules that matches the noise intensity of the noisy speech, adaptive adjustment of speech noise reduction can be achieved, reducing the waste of system resources and reducing the delay of speech interaction.

[0011] As an example, the speech denoising model also includes a noise intensity determination module, which is used to process the noisy speech feature based on the multi-head attention mechanism to obtain a first intermediate feature, then perform residual connection and layer normalization operations on the first intermediate feature to obtain a second intermediate feature, further process the second intermediate feature based on the feedforward layer to obtain a third intermediate feature, then perform residual connection and layer normalization operations on the third intermediate feature to obtain an output speech feature, and further process the output speech feature using an activation function to obtain the noise intensity of the noisy speech. Wherein, the noisy speech feature is obtained based on the noisy speech, the noisy speech is the first noisy speech or the second noisy speech, the noisy speech feature is the first noisy speech feature corresponding to the first noisy speech or the second noisy speech feature corresponding to the second noisy speech feature, the output speech feature is the first output speech feature corresponding to the first noisy speech feature or the second output speech feature corresponding to the second noisy speech feature, and the noise intensity is the first noise intensity corresponding to the first noisy speech or the second noise intensity corresponding to the second noisy speech.

[0012] It should be understood that through two residual connections and layer normalization operations, the difference part in the first intermediate feature or the third intermediate feature can be focused on, and through processing through the feedback layer, deeper feature information in the second intermediate feature can be obtained, thereby improving the accuracy of noise intensity prediction.

[0013] As an example, after obtaining the output speech feature, the method provided in the embodiment of the present application further includes: performing global average pooling on the output speech feature to obtain a one-dimensional vector, and then converting the one-dimensional vector into a noisy feature based on a fully connected layer. Correspondingly, the noisy feature is mapped to a preset interval using an activation function to obtain the noise intensity. Among them, by performing dimensionality reduction and mapping processing on the output speech feature, the output speech feature can be compressed from the time dimension to obtain a one-dimensional vector, and the one-dimensional vector is converted into a noisy feature, so that the noisy feature can be mapped to a preset interval using an activation function to predict the noise intensity of the noisy speech.

[0014] As an example, the speech denoising model also includes a pre-processing convolution module, which is used to obtain a noisy spectrogram corresponding to the noisy speech, wherein the noisy spectrogram is obtained by performing a short-time Fourier transform on the noisy speech, the noisy speech is a first noisy speech or a second noisy speech, and the noisy spectrogram is a first noisy spectrogram corresponding to the first noisy speech or a second noisy spectrogram corresponding to the second noisy speech; then, based on the noisy spectrogram, the noisy speech feature corresponding to the noisy speech is determined, wherein the noisy speech feature is a first noisy speech feature corresponding to the first noisy speech or a second noisy speech feature corresponding to the second noisy speech feature. By converting the noisy speech into a noisy spectrogram, the amount of calculation when determining the noisy speech feature can be reduced, and the efficiency of determining the noisy speech feature can be improved.

[0015] As an example, using a first number of speech enhancement modules to reduce noise on a first noisy speech to obtain a first original speech includes: using a first number of speech enhancement modules to process the first noisy speech to obtain a first processing feature, then determining a first mask corresponding to the first processing feature, wherein the first mask has the same size as the first noisy spectrogram, further multiplying the first mask by the first noisy spectrogram to obtain a first original spectrogram, and then performing an inverse Fourier transform on the first original spectrogram to obtain the first original speech corresponding to the first noisy speech. The first mask is used to filter the noise in the first noisy spectrogram to obtain the first original spectrogram, thereby achieving speech noise reduction.

[0016] As an example, using a second number of speech enhancement modules to reduce noise on a second noisy speech to obtain a second original speech includes: using a second number of speech enhancement modules to process the second noisy speech to obtain a second processing feature, determining a second mask corresponding to the second processing feature, wherein the second mask has the same size as the second noisy spectrogram, multiplying the second mask by the second noisy spectrogram to obtain a second original spectrogram, and further performing an inverse Fourier transform on the second original spectrogram to obtain a second original speech corresponding to the second noisy speech. The second mask is used to filter the noise in the second noisy spectrogram to obtain the second original spectrogram, thereby achieving speech noise reduction.

[0017] In a second aspect, an embodiment of the present application provides a method for reducing speech noise, including:

[0018] A noisy speech including original speech and noise is obtained, and then a corresponding speech noise reduction mode is selected according to the noise intensity of the noisy speech. Then, the noisy speech is denoised based on the speech noise reduction mode to obtain the original speech.

[0019] In an embodiment of the present application, the noise intensity of the noisy speech is determined to determine the strength of the noise in the noisy speech, and then different speech noise reduction modes are selected according to the noise intensity, and the noisy speech is denoised according to the selected speech noise reduction mode, thereby achieving adaptive adjustment of speech noise reduction, reducing the waste of system resources, and reducing the delay of voice interaction.

[0020] As an example, by determining the noisy speech feature corresponding to the noisy speech, then enhancing the noisy speech feature to obtain the output speech feature, and then processing the output speech feature using the activation function to obtain the noise intensity of the noisy speech. The noise intensity of the noisy speech can be predicted by processing the output speech with the activation function.

[0021] As an example, after the noisy speech feature is enhanced to obtain the output speech feature, it also includes: performing global average pooling on the output speech feature to obtain a one-dimensional vector, and then converting the one-dimensional vector into a noisy feature based on a fully connected layer. Correspondingly, the noisy feature is mapped to a preset interval using an activation function to obtain the noise intensity. Among them, by performing dimensionality reduction and mapping processing on the output speech feature, the output speech feature can be compressed from the time dimension to obtain a one-dimensional vector, and the one-dimensional vector is converted into a noisy feature, so that the noisy feature can be mapped to a preset interval using an activation function to predict the noise intensity of the noisy speech.

[0022] As an example, the noisy speech feature is enhanced to obtain the output speech feature, including: firstly, the noisy speech feature is processed based on the multi-head attention mechanism to obtain the first intermediate feature, then the first intermediate feature is subjected to residual connection and layer normalization operations to obtain the second intermediate feature, and the second intermediate feature is further processed based on the feedforward layer to obtain the third intermediate feature, and finally the third intermediate feature is subjected to residual connection and layer normalization operations to obtain the output speech feature. Among them, the difference part in the first intermediate feature or the third intermediate feature can be focused on through two residual connection and layer normalization operations, and the deeper feature information in the second intermediate feature can be obtained through the processing of the feedback layer, thereby improving the accuracy of noise intensity prediction.

[0023] As an example, before the noisy speech is denoised based on the speech denoising mode to obtain the original speech, the method further includes: obtaining a noisy spectrogram obtained by performing a short-time Fourier transform on the noisy speech, and then determining the noisy speech features corresponding to the noisy speech based on the noisy spectrogram. Correspondingly, based on the speech denoising mode, the noisy speech is denoised according to the noisy spectrogram and the noisy speech features to obtain the original speech. By converting the noisy speech into a noisy spectrogram, the amount of calculation when determining the noisy speech features can be reduced, and the efficiency of determining the noisy speech features can be improved.

[0024] As an example, according to the noisy spectrogram and the noisy speech features, the noisy speech is denoised to obtain the original speech, including: determining a mask corresponding to the noisy speech features and having the same size as the noisy spectrogram, multiplying the mask by the noisy spectrogram to obtain the original spectrogram, and performing an inverse Fourier transform on the original spectrogram to obtain the original speech corresponding to the noisy speech. The mask is used to filter the noise in the noisy spectrogram to obtain the original spectrogram, thereby achieving speech denoising.

[0025] As an example, the noise intensity of the noisy speech is determined by the noise intensity determination module in the speech noise reduction model, the number of speech enhancement modules in the speech noise reduction model is determined according to the noise intensity, and the noisy speech is denoised using the determined number of speech enhancement modules to obtain the original speech, where the number is at least one. The noise intensity of the noisy speech can be determined by the speech noise reduction model, and the number of speech enhancement modules in the speech noise reduction model can be adjusted according to the noise intensity to achieve adaptive adjustment of speech noise reduction.

[0026] As an example, the speech denoising model is trained in the following manner: obtaining a training noisy speech including a training original speech and training noise, determining a corresponding training processing result according to the training noisy speech through the speech denoising model to be trained; and adjusting the model parameters of the speech denoising model and the model parameters of the noise intensity determination module in the speech denoising model according to the training processing result and the training original speech. Among them, by training the speech denoising model, the speech denoising model can denoise noisy speech with different noise intensities.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0028] Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the electronic device executes any method of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes any method as in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a speech noise reduction model provided for related technologies;

[0031] Figure 2 A flow chart of a speech noise reduction method provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of a different form of speech provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a speech noise reduction model provided in an embodiment of the present application;

[0035] Figure 6aA schematic diagram of a noise intensity determination process provided in an embodiment of the present application;

[0036] Figure 6b A schematic diagram of a pre-processing convolution module provided in an embodiment of the present application;

[0037] Figure 6c A schematic diagram of converting a noisy speech into a noisy spectrogram provided in an embodiment of the present application;

[0038] Figure 6d A schematic diagram of a first output feature determination process provided in an embodiment of the present application;

[0039] Figure 6e A schematic diagram of a post-processing convolution module provided in an embodiment of the present application;

[0040] Figure 6f A schematic diagram of a mask provided in an embodiment of the present application;

[0041] Figure 7 A schematic diagram of filtering a noisy speech spectrogram using a mask provided in an embodiment of the present application;

[0042] Figure 8 A schematic diagram of a noisy speech noise reduction process provided in an embodiment of the present application;

[0043] Fig. 9 A schematic diagram of a scenario of a speech noise reduction method provided in an embodiment of the present application;

[0044] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the description of the following embodiments clear, several terms that may appear in the following are first explained:

[0046] Original speech refers to speech excluding noise, and noise refers to sound that does not change with the original sound. As an example, the sound of a user speaking is the original speech. If the user is near a TV that is playing a video, then the sound of the video played by the TV is noise. As another example, the music played by a speaker is the original speech. If there is a computer playing a video near the speaker, then the sound of the video played by the computer is noise.

[0047] Noisy speech means speech containing noise. In the following embodiments, the speech input into the electronic device is all noisy speech. The noisy speech is the sound emitted by the external device or the user's speech obtained by the microphone in the electronic device based on the noisy environment. The sound contains noise other than the original sound. The original sound refers to the sound emitted by the external device or the user's speech.

[0048] In the related art, the noisy speech is generally processed by a speech noise reduction model to obtain the original speech, see Figure 1 , the figure shows a speech denoising model 100 of the related technology, and the speech denoising model 100 can mainly include a pre-processing convolution module 101, N serially connected speech enhancement modules 102 and a post-processing convolution module 103.

[0049] The speech denoising model 100 generally converts the input noisy speech into noisy speech features through a pre-processing convolution module 101, and inputs the noisy speech features into N series-connected speech enhancement modules 102; the N series-connected speech enhancement modules process the noisy speech features to obtain processed features; the post-processing convolution module 103 processes the input processed features to obtain a mask corresponding to the noisy speech; the speech denoising model 100 uses the noisy speech and the mask to obtain the original speech.

[0050] In the study of speech noise reduction models, the present application found that noisy speech may contain noise of different noise intensities. When the noise intensity of the noisy speech is relatively small, the speech noise reduction model 100 still requires N series-connected speech enhancement modules to process the noisy speech with relatively small noise intensity, which may waste system resources and cause the processing time of speech noise reduction to be too long.

[0051] In order to solve the above technical problems, the present application provides a method for reducing speech noise, see Figure 2 , the method may include:

[0052] S201: Acquire noisy speech.

[0053] The noisy speech includes the original speech and the noise. Figure 3 As shown, the noisy speech can be collected by a microphone in the hardware layer.

[0054] S202: Determine the noise intensity of the noisy speech.

[0055] The noise intensity is used to indicate the strength of the noise in the noisy speech. The greater the noise intensity, the stronger the noise intensity in the noisy speech. In some examples, the noise intensity of the noisy speech can be divided into four levels, namely low noise intensity (0-0.25), medium-low noise intensity (0.25-0.5), medium-high noise intensity (0.5-0.75), and high noise intensity (0.75-1).

[0056] In some examples, the noise intensity of noisy speech can be determined by a speech noise reduction model in an application layer.

[0057] S203: Determine a corresponding speech noise reduction mode based on the noise intensity.

[0058] The speech noise reduction mode refers to the processing method of speech noise reduction corresponding to the noise intensity of noisy speech. For example, low noise intensity (0-0.25) can use the first speech noise reduction mode, and medium and low noise intensity (0.25-0.5) can use the second speech noise reduction mode. Different speech noise reduction modes require different system resources. For example, the first speech noise reduction mode may require 1% of system resources, and the second speech noise reduction mode may require 5% of system resources.

[0059] It should be understood that in the embodiment of the present application, the noise intensity can be divided into multiple levels, and different levels can be processed using different speech noise reduction modes to achieve adaptive speech noise reduction.

[0060] S204: De-noising the noisy speech based on the speech noise reduction mode to obtain the original speech.

[0061] It should be understood that in the embodiments of the present application, by determining the noise intensity of the noisy speech, selecting the corresponding speech noise reduction mode according to different noise intensities, and reducing the noise of the noisy speech according to the speech noise reduction mode, adaptive adjustment is achieved, the waste of system resources is reduced, and the voice interaction delay is reduced.

[0062] The speech noise reduction method provided in the above embodiment can be applied to an electronic device installed with an application program. Figure 3 In this embodiment, the Android system of the mobile phone can be divided into four layers, namely, the application layer, the application framework layer, the kernel layer and the hardware abstraction layer from top to bottom. In order to facilitate the understanding of this solution, a hardware layer is added after the kernel layer.

[0063] The application layer can include a series of application packages. Figure 3 As shown, the application package can include: system applications, such as recorder, call, note, video; cloud voice interaction applications, such as map applications, super notes, input method, AI subtitles; local voice interaction applications, including visible and speakable applications, voice assistants. Among them, system applications refer to applications that come with the electronic device's system; cloud voice interaction applications refer to applications that implement voice interaction through the cloud platform; local voice interaction applications refer to applications that implement voice interaction through the local computing power of the electronic device.

[0064] In some examples, system applications, cloud-based voice interaction applications, and local voice interaction applications can all use the voice noise reduction model provided in the embodiments of the present application, and can execute the process of the above-mentioned voice noise reduction method. The embodiments of the present application only take the voice assistant in the local voice interaction application as an example for exemplary explanation. For example, the voice assistant can include a voice noise reduction model and scenario-based voice instructions.

[0065] The voice assistant recognizes the audio, converts it into text, and then replies with the answer corresponding to the text or performs the corresponding action.

[0066] It should be noted that different applications require different forms of voice input. In some examples, such as Figure 4 The voices 1 to 6 shown in the figure can be divided into original microphone signals, audio uplink data stream, audio downlink data stream, call audio, video audio, and voice recognition audio. The system application can input Figure 4 The voice 1 to 6 shown in the figure; both the cloud voice interaction application and the local voice interaction application can be Figure 4 Voice 6 shown.

[0067] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a content provider, a resource manager, an audio manager, and the like.

[0068] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0069] The resource manager provides various resources for applications, such as localized strings, audio files, video files, and so on.

[0070] The audio manager enables applications to record and collect voices generated by external devices or user voices, and can pass the collected voice signals to speech recognition applications or voice assistants to achieve voice interaction.

[0071] The kernel layer is the layer between hardware and software. The kernel layer contains at least the audio driver.

[0072] The hardware abstraction layer is an interface layer between the kernel layer and the hardware layer, and its purpose is to abstract the hardware. Figure 3 As shown, the hardware abstraction layer can be used to convert the noisy speech collected by the microphone in the hardware layer into an audio stream for input.

[0073] Combination Figure 3 As shown, illustratively, the hardware layer provided by the embodiment of the present application includes at least an advanced digital signal processor, a microphone and a speaker.

[0074] The advanced digital signal processor can perform speech pre-processing on the noisy speech input through the microphone and input the processed noisy speech into the hardware abstraction layer. The speech pre-processing may include echo cancellation processing, noise reduction processing, gain control processing, etc.

[0075] Among them, the noise in noisy speech is divided into steady-state noise and transient noise. The spectrum of steady-state noise is stable; transient noise is extremely sudden, in the form of oscillation attenuation in the time domain, and the duration ranges from tens of milliseconds to hundreds of milliseconds; it is widely distributed in the frequency domain, and the spectrum of transient noise is basically mixed with the spectrum of normal speech. Noise reduction processing in speech pre-processing means the operation of denoising steady-state noise in noisy speech. The speech noise reduction method provided in the embodiment of the present application is to reduce the transient noise in noisy speech.

[0076] Figure 3 The speech denoising model in the voice assistant shown here is a model that extracts original speech from noisy speech. The following content introduces how to train the original speech denoising model to obtain the speech denoising model.

[0077] In some examples, the speech noise reduction model of the embodiment of the present application can be trained in the following ways:

[0078] Step 11: Obtain training noisy speech; the training noisy speech includes training original speech and training noise.

[0079] In some examples, the Libirspeech (an audiobook dataset containing text and speech) dataset or the Aishell (an eight-channel Chinese Mandarin conference scene speech dataset recorded through a microphone array) dataset can be used as training data for training a speech noise reduction model, and the Musha dataset can be used as noise data. For example, multiple data in the Libirspeech dataset are used as training original speech, and multiple data in the Musha dataset are used as training noise.

[0080] During the training process, the signal-to-noise ratio can be a random value between [0, 20]. The signal-to-noise ratio (SNR or S / N, SIGNAL-NOISE RATIO, also known as signal-to-noise ratio) refers to the ratio of speech signal to noise in an electronic device or electronic system. The higher the signal-to-noise ratio, the weaker the noise intensity, that is, the less noise in the noisy speech.

[0081] Step 12: Determine the corresponding training processing result according to the training noisy speech through the speech denoising model to be trained; and adjust the model parameters of the speech denoising model and the model parameters of the signal-to-noise ratio determination module in the speech denoising model according to the training processing result and the training original speech.

[0082] It should be understood that the speech denoising model obtained by executing step 11 and step 12 is applicable to the speech denoising method provided in the embodiment of the present application, and the speech denoising models involved in the following parts all refer to the trained speech denoising models.

[0083] In some examples, it is assumed that the speech denoising model includes 4 groups of speech enhancement modules, each group includes 4 speech enhancement modules, that is, the speech denoising model includes 16 denoising modules. The preset number of test noisy speech in the Libirspeech data set is used as a test set to test the speech denoising model, by determining the number of speech enhancement modules required when the speech denoising model denoises noisy speech at different signal-to-noise ratios, and the test result evaluation index; and determining the test result evaluation index of the speech denoising model in the related art containing 16 speech enhancement modules, the results are shown in Table 1 below:

[0084] Table 1

[0085]

[0086] In combination with Table 1 above, it can be seen that under the same signal-to-noise ratio, the speech noise reduction model provided by the present application can achieve the same noise reduction effect through fewer speech enhancement modules. For example, when the signal-to-noise ratio is [15, 20], the speech noise reduction model provided by the embodiment of the present application only needs 5.28 speech enhancement modules for speech noise reduction, while the speech noise reduction model in the related art requires 16 speech enhancement modules for speech noise reduction.

[0087] Compared with the speech noise reduction model in the related art, the speech noise reduction model provided in the embodiment of the present application reduces the waste of system resources and reduces the delay of voice interaction.

[0088] See also Figure 5 The speech noise reduction model provided in the embodiment of the present application may include a pre-processing convolution module, M groups of speech enhancement module groups, a noise intensity determination module and a post-processing convolution module. Each of the M groups of speech enhancement module groups includes N speech enhancement modules, M is a positive integer greater than or equal to 2, and N is a positive integer greater than or equal to 1.

[0089] The noise intensity determination module can determine the noise intensity of noisy speech, see Figure 6a The process of determining the noise intensity of the noisy speech provided in the embodiment of the present application may be:

[0090] S601: After the noise intensity determination module receives the noisy speech feature corresponding to the noisy speech, the noisy speech feature is processed by the multi-head attention mechanism in the noise intensity determination module to obtain the output first intermediate feature.

[0091] Among them, the multi-head attention mechanism can increase the weight of the original speech features in the noisy speech features, which facilitates the subsequent extraction of the original speech features.

[0092] Among them, the noisy speech features are obtained by the pre-processing convolution module. The relevant explanation can be found below.

[0093] S602: Perform residual connection and layer normalization operations (Add&Norm) on the noisy speech feature and the first intermediate feature to obtain the second intermediate feature.

[0094] Through the residual connection operation, the noise intensity determination module can focus only on the difference part in the first intermediate feature during the training process; through the layer normalization operation, the convergence speed of the noise intensity determination module during the training process can be accelerated.

[0095] S603: Process the second intermediate feature based on the feedforward layer to obtain a third intermediate feature.

[0096] By processing the second intermediate feature through the feedforward layer, deeper feature information of the second intermediate feature can be obtained, thereby improving the expression ability of the speech denoising model.

[0097] S604: Perform a second residual connection and layer normalization operation (Add&Norm) on the second intermediate feature and the third intermediate feature to obtain output speech features.

[0098] Among them, the residual connection and layer normalization operations can avoid the problem of gradient disappearance when the network gradient backpropagates to update parameters.

[0099] S605: Perform a global average pooling (Global-Avg Pool) operation on the output speech features, compress the output speech features from the time dimension, and convert the two-dimensional output speech features into a one-dimensional vector.

[0100] S606: Convert the one-dimensional vector into a noisy feature based on a fully connected layer.

[0101] S607: Use the activation function (Sigmoid function) in the activation function sigmoid layer to process the noisy features and map the noisy features to the [0,1] interval.

[0102] The noise intensity of the noisy speech can be predicted by processing the noisy features through the activation function. The noise intensity is any number between [0,1], such as 0, 1, 0.5, 0.2, 0.1, 0.8, etc.

[0103] In an embodiment of the present application, by using the sigmoid layer as the last layer of the noise intensity determination module, a noise intensity between 0 and 1 can be output, and the noise intensity is used to indicate the strength of the noise in the noisy speech. The greater the noise intensity, the stronger the noise intensity in the noisy speech, and the more noise the noisy speech contains.

[0104] The noise intensity of the noisy speech is determined, and the number of speech enhancement modules in the speech noise reduction model is determined based on the noise intensity to achieve adaptive adjustment of speech noise reduction. That is, the noise intensity of each noisy speech may be different, and the number of corresponding speech enhancement modules is also different. The noisy speech is processed by the number of speech enhancement modules that matches the noise intensity of the noisy speech to achieve adaptive adjustment of speech noise reduction. That is, the greater the noise intensity, the more speech enhancement modules are required; conversely, the fewer the number of speech enhancement modules are.

[0105] In some embodiments, the noisy speech features are obtained by a pre-processing convolution module in a speech denoising model, see Figure 6b The pre-processing convolution module provided in the embodiment of the present application may include two one-dimensional convolutional neural networks. The process of determining the noisy speech features using the pre-processing convolution module may be: the noisy speech spectrogram corresponding to the noisy speech is processed twice by the one-dimensional convolutional neural network to obtain the noisy speech features corresponding to the noisy speech.

[0106] One-dimensional convolutional neural networks are used to perform convolution on the width of two-dimensional data. For example, the size of the input data is d*s, where d is the dimension of the word vector and s is the maximum length of the sentence. The convolution kernel window slides in the direction of the sentence length to perform the convolution operation.

[0107] Among them, the noisy speech features determined by the pre-processing convolution module can be used as inputs of the speech enhancement module and the noise intensity determination module respectively.

[0108] In some examples, the process of converting noisy speech into a noisy spectrogram may be:

[0109] Step 21: Read the noisy speech of preset duration in real time, such as Figure 6c Noisy speech shown.

[0110] It should be understood that the duration of the input noisy speech may be long. In order to reduce the amount of calculation, the noisy speech of a preset duration can be read for processing. In some examples, assuming that the noisy speech is a real-time input speech, 20 milliseconds of noisy speech can be read as input, and 30 milliseconds of noisy speech can be read as input the next time. The duration of reading the noisy speech can be the same or different, and is not specifically limited here.

[0111] Step 22: Perform signal pre-emphasis processing on the noisy speech.

[0112] It should be understood that generally, the high-frequency component of an audio signal has a smaller intensity, while the low-frequency component has a larger intensity. The intensity of the high-frequency and low-frequency components of noisy speech can be made similar through the signal pre-emphasis processing method.

[0113] Step 23: performing frame processing on the noisy speech after the signal pre-emphasis processing.

[0114] It should be understood that the duration of noisy speech is relatively long. If the noisy speech of the original duration is used for Fourier transform, only the relationship between the signal frequency and intensity can be obtained, and the information of the time dimension is lost. Therefore, in order to obtain the relationship between frequency and time, it is necessary to divide the noisy speech into several frames, perform short-time Fourier transform on each frame, and then splice the obtained transformation results in chronological order.

[0115] Step 24: After the framing is completed, a window function is added to each frame to obtain the frames variable (the property returns all the named frames in the window) to obtain a better sidelobe reduction amplitude to avoid a large difference in the spectrum between the noisy signal and the corresponding part after windowing.

[0116] Step 25: Perform short-time Fourier transform on each frame of windowed noisy speech to obtain a spectrum diagram corresponding to each frame of windowed noisy speech.

[0117] Step 26: Splice multiple frames of spectrograms to obtain a noisy spectrogram corresponding to the noisy speech, such as Figure 6c shown.

[0118] The data on the noisy spectrogram is used to represent the characteristics of the noisy speech. The horizontal axis of the noisy spectrogram is time, the vertical axis is frequency, and the coordinate point value is the energy of the noisy speech. In the embodiment of the present application, by converting the noisy speech into a two-dimensional noisy spectrogram, the noisy speech can be better processed, the amount of calculation can be reduced, and the noise reduction efficiency can be improved.

[0119] Based on the above embodiment, after determining the noisy speech features through the pre-processing convolution module and determining the noise intensity of the noisy speech through the noise intensity determination module, the speech denoising model can determine the number of speech enhancement modules according to the noise intensity of the noisy speech, and process the noisy speech features according to the multiple speech enhancement modules to obtain output features. Figure 6d , the process of determining the first output feature by a speech enhancement module provided in the embodiment of the present application may be:

[0120] S611: After the speech enhancement module receives the noisy speech feature, the noisy speech feature can be processed through a multi-head attention mechanism to obtain a first intermediate feature of the output.

[0121] S612: After obtaining the first intermediate feature, perform residual connection and layer normalization operations (Add&Norm) on the first intermediate feature to obtain the second intermediate feature.

[0122] S613: Based on the feed-forward layer, the second intermediate feature is processed to obtain the third intermediate feature.

[0123] S614: Perform a second residual connection and layer normalization operation (Add&Norm) on the third intermediate feature to obtain the first output feature of the speech enhancement module.

[0124] It should be noted that the implementation process of step S611 to step S614 is the same as the implementation process of the above-mentioned step S601 to step S604. The relevant explanations can be found above and will not be repeated here.

[0125] It should be understood that the multiple speech enhancement modules provided in the embodiment of the present application all perform the operations of step S611 to step S614. Among them, the multiple speech enhancement modules are connected in series, so the input of the i+1th speech enhancement module is the output of the ith speech enhancement module, where i is a positive integer greater than or equal to 1. That is, the output feature corresponding to the last speech enhancement module is used as the output feature.

[0126] It should be noted that by processing the noisy speech features through multiple speech enhancement modules, an output feature can be determined, and the mask corresponding to the noisy speech can be determined using the output feature. Figure 6e , the process of generating a mask by a post-processing convolution module provided in the embodiment of the present application may be:

[0127] S621: Process the output features through a one-dimensional convolutional neural network to obtain a one-dimensional vector.

[0128] S622: Through the activation function layer, the one-dimensional vector is processed based on the sigmoid function, the value of the element corresponding to the original speech feature is set to 1, and the value of the element corresponding to the noisy speech feature is set to 0, so as to obtain Figure 6f The mask shown.

[0129] It should be noted that the mask can be understood as a layer of film covering the noisy speech spectrogram of the noisy speech, thereby shielding or selecting some characteristic elements. That is, the noise elements on the noisy speech spectrogram can be filtered by the mask. It should be understood that the noisy speech spectrogram can include m×n elements, including original speech elements and noise elements, the original speech elements correspond to the original speech part, and the noise elements correspond to the noise part.

[0130] It should be understood that, through the processing of the aforementioned multiple speech enhancement modules, the output features obtained distinguish the original speech features from the noisy speech features, so the value of the element corresponding to the original speech feature can be set to 1 and the value of the element corresponding to the noisy speech feature can be set to 0 through the sigmoid function to obtain the mask corresponding to the output feature. The mask is used to filter the noise in the noisy speech spectrogram. The original speech feature refers to the feature corresponding to the original speech, and the noisy speech feature refers to the feature corresponding to the noise.

[0131] After determining the mask corresponding to the output feature, the noisy spectrogram corresponding to the noisy speech can be filtered through the mask to obtain the original spectrogram, see Figure 7 ,The process of filtering the noisy spectrogram using the mask can be: multiplying the mask with the noisy spectrogram, and the noise elements in the noisy spectrogram can be filtered to obtain the original spectrogram.

[0132] The size of the mask is the same as that of the noisy spectrogram. By multiplying the elements with a value of 0 in the mask with the elements at the same position in the noisy spectrogram, the values ​​of the elements at the same position in the original spectrogram are obtained to be 0, thereby filtering the noise elements in the noisy spectrogram. As an example, Figure 7 As shown in the X position in , the value of the element at the X position of the mask is 0, the value of the element at the X position of the noisy speech feature is 0.2, and the value of the element at the X position is multiplied, and the value of the element at the X position in the original spectrogram is 0.

[0133] In some examples, combined Figure 8 As shown, based on the speech noise reduction model, the noise reduction process of noisy speech can be:

[0134] Step 31: The noisy speech signal corresponding to the noisy speech is subjected to short-time Fourier transform to obtain a noisy spectrogram.

[0135] Step 32: Input the noisy spectrogram into the speech denoising model, and obtain a mask based on the speech denoising model.

[0136] Step 33: Filter the noise elements in the noisy spectrogram through the mask to obtain the original spectrogram.

[0137] Step 34: Perform inverse Fourier transform on the original spectrogram to obtain the original speech.

[0138] The sizes of the noisy spectrogram, mask, and original spectrogram are all the same.

[0139] In an embodiment of the present application, noisy speech is converted into a two-dimensional noisy spectrogram; a mask is generated based on a speech enhancement module in a speech noise reduction model, and the mask is used to filter the noise in the noisy spectrogram to obtain the original spectrogram; and the original spectrogram is converted back into original speech to achieve adaptive noise reduction of noisy speech.

[0140] The speech noise reduction method provided in the above embodiment of the present application can be applied to a variety of application scenarios. The following takes the user waking up the voice assistant and the user driving scenario as examples to introduce the implementation process of the speech noise reduction method.

[0141] In the scenario where the user is watching TV, see Fig. 9 As shown in B in the figure, the user is watching TV and audio is playing on the TV. The user can wake up the smart speaker by saying "Hello, speaker assistant".

[0142] In the user driving scenario, see Fig. 9 As shown in A in the figure, the user is driving a vehicle and can wake up the voice assistant of the mobile phone by saying "Hi, voice assistant".

[0143] The mobile phone or smart speaker can receive the specific voice through the microphone, such as "Hi, voice assistant" or "Hello, speaker assistant"; then, the voice noise reduction model of the mobile phone or smart speaker will perform noise reduction on the input noisy voice to obtain the original voice; the voice recognition model of the mobile phone or smart speaker will perform voice recognition on the original voice; after successful recognition, the mobile phone voice assistant or speaker assistant will perform relevant operations or responses.

[0144] Among them, the speech noise reduction model of the mobile phone or smart speaker will process the noisy speech features of the noisy speech through the noise intensity determination module to obtain the noise intensity of the noisy speech; and select different numbers of speech enhancement modules based on the noise intensity to perform noise reduction on the noisy speech features to achieve adaptive noise reduction.

[0145] In some examples, when the noise intensity of noisy speech is 0.00-0.25, the switch of the speech noise reduction model can be switched to node ①; when the noise intensity of noisy speech is 0.25-0.50, the switch of the speech noise reduction model can be switched to node ②; when the noise intensity of noisy speech is 0.50-0.75, the switch of the speech noise reduction model can be switched to node ③; when the noise intensity of noisy speech is 0.75-1.00, the switch of the speech noise reduction model can be switched to node ④.

[0146] For example, assuming Fig. 9The environment of A is relatively quiet. Fig. 9 The environment of B in is relatively noisy, that is, Fig. 9 The noise intensity corresponding to A in is smaller. Fig. 9 The noise intensity corresponding to B in is larger, so Fig. 9 In the environment of A in , based on the smaller noise intensity, node ② can be selected, that is, two groups of speech enhancement modules are selected to process the noisy speech features; Fig. 9 In the environment of B in , based on the greater noise intensity, node ③ can be selected, that is, 3 groups of speech enhancement modules are selected to process the noisy speech features. Adaptive noise reduction is achieved, the waste of system resources is reduced, the noise reduction efficiency is improved, and the delay of voice interaction is further reduced.

[0147] The speech noise reduction method provided in the above embodiments can be applied to electronic devices, which may be mobile phones, tablet computers, laptop computers, wearable electronic devices (such as smart watches), devices with voice interaction functions (such as smart speakers, smart TVs, smart refrigerators, smart access control systems) and vehicle-mounted devices, etc. This application does not impose any special restrictions on the specific forms of the above electronic devices.

[0148] The following uses a mobile phone as an example to illustrate the hardware structure of an electronic device.

[0149] like Fig.10 As shown, the electronic device may include a processor 1010, an internal memory 1020, a universal serial bus (USB) interface 1030, an audio module 1040, a speaker 1040A, a receiver 1040B, a microphone 1040C, and an earphone interface 1040D.

[0150] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0151] The processor 1010 is used to execute the speech noise reduction method provided in the above embodiment.

[0152] The processor 1010 may include one or more processing units, for example, the processor 1010 may include an advanced digital signal processor (ADSP) and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0153] The advanced digital signal processor is used to process audio signals. For example, when a user records noisy speech through the microphone 1040C of the mobile phone, the advanced digital signal processor is used to process and analyze the noisy speech. In some examples, the advanced digital signal processor can perform speech pre-processing on the noisy speech and input the noisy speech into the hardware abstraction layer.

[0154] NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as speech recognition, voice wake-up, etc. In some examples, the NPU computing processor can be used in the reasoning of speech noise reduction models.

[0155] In some embodiments, the processor 1010 may include one or more interfaces. The interface may include an inter-integrated circuit sound (I2S) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0156] In some embodiments, the processor 1010 may include multiple I2S buses. The processor 1010 may be coupled to the audio module 1040 via the I2S bus to implement communication between the processor 1010 and the audio module 1040.

[0157] The USB interface 1030 is an interface that complies with USB standard specifications, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 1030 can be used to connect headphones to play audio through the headphones. The USB interface 1030 can also be used to connect wired headphones to receive noisy voices through a microphone provided on the wired headphones.

[0158] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0159] The internal memory 1020 can be used to store computer executable program codes, which include instructions. The processor 1010 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 1020. The internal memory 1020 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 1020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 1010 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 1020, and / or the instructions stored in the memory provided in the processor.

[0160] The electronic device can implement audio functions such as music playing and recording through the audio module 1040, the speaker 1040A, the receiver 1040B, the microphone 1040C, the headphone jack 1040D, etc.

[0161] The audio module 1040 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 1040 can also be used to encode and decode audio signals. In some embodiments, the audio module 1040 can be arranged in the processor 1010, or some functional modules of the audio module 1040 can be arranged in the processor 1010.

[0162] Speaker 1040A, also called "speaker", is used to convert audio electrical signals into sound signals. In some examples, speaker 1040A can play the audio corresponding to the voice recognition result, or the audio corresponding to the successful voice wake-up. In some examples, the user can use the speaker 1040A of the mobile phone.

[0163] Receiver 1040B, also called a "earpiece", is used to convert audio electrical signals into sound signals. In some examples, the user can place receiver 1040B close to the ear and listen to the audio corresponding to the speech recognition result.

[0164] Microphone 1040C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. In some embodiments, microphone 1040C of the mobile phone can collect noisy speech. Figure 3 As shown, microphone 1040C can collect noisy speech and transmit it to processor 1010; the advanced digital signal processor in processor 1010 can perform speech pre-processing on the noisy speech and input the noisy speech into the hardware abstraction layer. Figure 3 The hardware abstraction layer in the kernel layer sends the input noisy speech as an audio stream to the audio processor of the application layer based on the audio driver of the kernel layer. The audio processor inputs the obtained audio stream as the input noisy speech into the speech recognition application or voice assistant.

[0165] The earphone interface 1040D is used to connect a wired earphone and can be a USB interface 1030 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0166] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0167] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0168] The technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, etc., various media that can store program codes.

Claims

1. A speech noise reduction method, characterized in that: include: Acquire a first noisy speech; the first noisy speech includes a first original speech and a first noise; Determining a first signal-to-noise ratio corresponding to the first noisy speech according to a first noise intensity of the first noisy speech; Determine a first number of speech enhancement modules in a speech noise reduction model based on the first signal-to-noise ratio; Using the first number of speech enhancement modules to reduce noise on the first noisy speech to obtain the first original speech; Acquire a second noisy speech; the second noisy speech includes a second original speech and a second noise; determining a second signal-to-noise ratio corresponding to the second noisy speech according to a second noise intensity of the second noisy speech; the second noise intensity is higher than the first noise intensity; the second signal-to-noise ratio is lower than the first signal-to-noise ratio; Determine a second number of speech enhancement modules in the speech noise reduction model based on the second signal-to-noise ratio; the second number is greater than the first number; The second noisy speech is denoised using the second number of speech enhancement modules to obtain the second original speech.

2. The method according to claim 1, characterized in that: The speech denoising model comprises a plurality of speech enhancement modules connected in series; wherein the input of the i+1th speech enhancement module is the output of the ith speech enhancement module, and i is a positive integer greater than or equal to 1; when i=0, the input of the first speech enhancement module is the first noisy speech or the second noisy speech.

3. The method according to claim 1, characterized in that The speech denoising model further includes a noise intensity determination module; the noise intensity determination module is used to process the noisy speech feature based on a multi-head attention mechanism to obtain a first intermediate feature; the noisy speech feature is obtained based on the noisy speech; the noisy speech is the first noisy speech or the second noisy speech; the noisy speech feature is the first noisy speech feature corresponding to the first noisy speech or the second noisy speech feature corresponding to the second noisy speech feature; Performing residual connection and layer normalization operations on the first intermediate features to obtain second intermediate features; Processing the second intermediate feature based on the feed-forward layer to obtain a third intermediate feature; Performing residual connection and layer normalization operations on the third intermediate feature to obtain an output speech feature; the output speech feature is a first output speech feature corresponding to the first noisy speech feature or a second output speech feature corresponding to the second noisy speech feature; The output speech feature is processed using an activation function to obtain a noise intensity of the noisy speech; the noise intensity is a first noise intensity corresponding to the first noisy speech or a second noise intensity corresponding to the second noisy speech.

4. The method according to claim 3, characterized in that After obtaining the output speech feature, the method further includes: Performing global average pooling on the output speech features to obtain a one-dimensional vector; Converting the one-dimensional vector into a noisy feature based on a fully connected layer; the noisy feature is a first noisy feature corresponding to the first output speech feature or a second noisy feature corresponding to the second output speech feature; The step of processing the output speech feature by using an activation function to obtain the noise intensity of the noisy speech includes: The noisy feature is mapped to a preset interval using an activation function to obtain the noise intensity.

5. The method according to claim 3, characterized in that: The speech denoising model further includes a pre-processing convolution module; the pre-processing convolution module is used to obtain a noisy spectrogram corresponding to the noisy speech; the noisy spectrogram is obtained by performing a short-time Fourier transform on the noisy speech; the noisy speech is the first noisy speech or the second noisy speech; the noisy spectrogram is a first noisy spectrogram corresponding to the first noisy speech or a second noisy spectrogram corresponding to the second noisy speech; A noisy speech feature corresponding to the noisy speech is determined based on the noisy speech spectrogram; the noisy speech feature is a first noisy speech feature corresponding to the first noisy speech or a second noisy speech feature corresponding to the second noisy speech feature.

6. The method according to claim 5, characterized in that The step of using the first number of speech enhancement modules to reduce noise on the first noisy speech to obtain the first original speech includes: Processing the first noisy speech using the first number of speech enhancement modules to obtain a first processing feature; Determining a first mask corresponding to the first processing feature; the first mask has the same size as the first noisy spectrogram; multiplying the first mask by the first noisy spectrogram to obtain a first original spectrogram; Performing an inverse Fourier transform on the first original spectrogram to obtain a first original speech corresponding to the first noisy speech.

7. The method according to claim 5, characterized in that The step of reducing noise on the second noisy speech by using the second number of speech enhancement modules to obtain the second original speech includes: Processing the second noisy speech using the second number of speech enhancement modules to obtain a second processing feature; Determining a second mask corresponding to the second processing feature; the second mask has the same size as the second noisy spectrogram; multiplying the second mask by the second noisy spectrogram to obtain a second original spectrogram; Perform an inverse Fourier transform on the second original spectrogram to obtain a second original speech corresponding to the second noisy speech.

8. The method according to claim 1, characterized in that The speech noise reduction model is trained in the following way: Acquire training noisy speech; the training noisy speech includes training original speech and training noise; Determine the corresponding training processing result according to the training noisy speech by the speech noise reduction model to be trained; And according to the training processing result and the training original speech, the model parameters of the speech noise reduction model and the model parameters of the noise intensity determination module in the speech noise reduction model are adjusted.

9. An electronic device, characterized in that: include: Processor and memory; Wherein, one or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the electronic device performs the method as described in any one of claims 1-8.

10. A computer storage medium, characterized in that: The method comprises computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

Images