Training method, device, equipment, storage medium and product of speech noise reduction model

Abstract

The present application provides a method, device, equipment, storage medium and product for training a speech noise reduction model, which belongs to the field of audio signal processing technology. The method includes: based on multiple groups of first sample data pairs in simulation scenarios of multiple scenarios, training a first speech noise reduction model; in a target scenario among multiple scenarios, reverberation processing is performed on the initial speech sample data and the initial noise sample data in each group of initial sample data pairs respectively, to obtain second speech sample data and target noise data containing target environment reverberation, and the second speech sample data and the target noise data are mixed to obtain second noisy speech sample data; based on multiple groups of third sample data pairs including second speech sample data and target noise data, the first speech noise reduction model is trained to obtain a target speech noise reduction model; the method makes the target speech noise reduction model suitable for the target scenario, and can improve the noise reduction effect of the noisy speech data in the target scenario.

Description

Technical Field

[0001] The present application relates to the technical field of audio signal processing, and in particular to a training method, device, equipment, storage medium and product for a speech noise reduction model. Background Art

[0002] Currently, speech noise reduction models are used to reduce the noise of noisy speech data in scenarios such as meetings, lectures, and teaching, in order to obtain clean speech data after noise is removed.

[0003] In the related art, generally, based on the simulation scenarios of these scenarios, noisy speech sample data and clean speech sample data are obtained to perform model training to obtain a speech noise reduction model that is common to these scenarios. However, when performing speech noise reduction in any scenario based on the speech noise reduction model, since the scenario is not exactly the same as its simulation scenario, performing noise reduction on the noisy speech data in the scenario based on the speech noise reduction model will result in poor noise reduction effect. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment, storage medium and product for training a speech noise reduction model, which can improve the noise reduction effect on noisy speech data. The technical solution is as follows:

[0005] On the one hand, a method for training a speech noise reduction model is provided, the method comprising:

[0006] A first speech denoising model is obtained by training a plurality of groups of first sample data pairs in simulation scenarios based on a plurality of scenarios, wherein each group of first sample data pairs comprises first speech sample data and first noisy speech sample data, wherein the first speech sample data is the first noisy speech sample data after denoising;

[0007] Acquire multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is the first speech sample data without simulated environment reverberation, and the initial noise sample data is the data after removing speech data and simulated environment reverberation from the first noisy speech sample data;

[0008] In a target scene among the multiple scenes, performing reverberation processing on initial speech sample data in the multiple groups of initial sample data pairs to obtain second speech sample data in multiple groups of second sample data pairs containing target environment reverberation, and performing reverberation processing on initial noise sample data in the multiple groups of initial sample data pairs to obtain target noise data in the multiple groups of second sample data pairs;

[0009] Mixing the second speech sample data and the target noise data in each group of second sample data pairs to obtain a plurality of second noisy speech sample data;

[0010] Based on multiple groups of third sample data pairs, the first speech denoising model is trained to obtain a target speech denoising model, and the target speech denoising model is used to denoise the noisy speech data in the target scene, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

[0011] In some embodiments, performing reverberation processing on the initial speech sample data in the plurality of groups of initial sample data pairs to obtain second speech sample data in the plurality of groups of second sample data pairs containing target environment reverberation includes:

[0012] For each group of initial sample data pairs, the initial voice sample data is played in the target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain the second voice sample data containing the target environment reverberation; or

[0013] For each group of initial sample data pairs, impulse response data in the target scene is obtained, and convolution processing is performed on the impulse response data and the initial speech sample data to obtain the second speech sample data containing the target environment reverberation.

[0014] In some embodiments, performing reverberation processing on the initial noise sample data in the plurality of groups of initial sample data pairs to obtain target noise data in the plurality of groups of second sample data pairs includes:

[0015] For each group of initial sample data pairs, determine the noise type corresponding to the initial noise sample data, collect the noise corresponding to the noise type in the target scene, and obtain the target noise data containing the target environment reverberation; or

[0016] For each group of initial sample data pairs, the initial noise sample data is played in the target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain the target noise data containing the target environment reverberation.

[0017] In some embodiments, the mixing of the second speech sample data and the target noise data in each plurality of groups of second sample data pairs to obtain a plurality of second noisy speech sample data includes:

[0018] For each group of second sample data pairs, based on a target signal-to-noise ratio, the second speech sample data and the target noise data are mixed to obtain the second noisy speech sample data.

[0019] In some embodiments, the target noise data includes target noise data of multiple different noise types, the second noisy speech sample data is multiple, and the second speech sample data and the target noise data are mixed based on the target signal-to-noise ratio to obtain the second noisy speech sample data, including at least one of the following implementations:

[0020] Based on the target signal-to-noise ratio, respectively mixing the target noise data of the plurality of different noise types with the second speech sample data to obtain a plurality of second noisy speech sample data;

[0021] For target noise data of each noise type, the target noise data is mixed with at least one target noise data of a different noise type to obtain a plurality of mixed noise data; based on the target signal-to-noise ratio, the plurality of mixed noise data are respectively mixed with the second speech sample data to obtain a plurality of second noisy speech sample data.

[0022] In some embodiments, the second speech sample data includes second speech sample data of multiple different groups of people, the second noisy speech sample data is multiple, and the mixing of the second speech sample data and the target noise data based on the target signal-to-noise ratio to obtain the second noisy speech sample data includes at least one of the following implementations:

[0023] Based on the target signal-to-noise ratio, the second speech sample data of the multiple different groups of people are mixed with the target noise data respectively to obtain a plurality of second noisy speech sample data;

[0024] For the second speech sample data of each group of people, the second speech sample data is mixed with at least one second speech sample data different from the group of people to obtain a plurality of mixed speech sample data; based on the target signal-to-noise ratio, the plurality of mixed speech sample data are respectively mixed with the target noise data to obtain a plurality of second noisy speech sample data.

[0025] On the other hand, a training device for a speech noise reduction model is provided, the device comprising:

[0026] A first training module is used to train a first speech denoising model based on a plurality of groups of first sample data pairs in simulation scenarios of a plurality of scenarios, wherein each group of first sample data pairs comprises first speech sample data and first noisy speech sample data, wherein the first speech sample data is the first noisy speech sample data after denoising;

[0027] An acquisition module, used to acquire multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is the first speech sample data without simulated environment reverberation, and the initial noise sample data is the data obtained by removing speech data and simulated environment reverberation from the first noisy speech sample data;

[0028] a processing module, configured to perform reverberation processing on the initial speech sample data in the plurality of groups of initial sample data pairs in a target scene among the plurality of scenes to obtain second speech sample data in a plurality of groups of second sample data pairs containing target environment reverberation, and perform reverberation processing on the initial noise sample data in the plurality of groups of initial sample data pairs to obtain target noise data in the plurality of groups of second sample data pairs;

[0029] A mixing module, used for mixing the second speech sample data and the target noise data in each group of second sample data pairs to obtain a plurality of second noisy speech sample data;

[0030] The second training module is used to train the first speech noise reduction model based on the multiple groups of third sample data pairs to obtain a target speech noise reduction model, wherein the target speech noise reduction model is used to denoise the noisy speech data in the target scene, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

[0031] In some embodiments, the processing module is used to:

[0032] For each group of initial sample data pairs, the initial voice sample data is played in the target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain the second voice sample data containing the target environment reverberation; or

[0033] For each group of initial sample data pairs, impulse response data in the target scene is obtained, and convolution processing is performed on the impulse response data and the initial speech sample data to obtain the second speech sample data containing the target environment reverberation.

[0034] In some embodiments, the processing module is used to:

[0035] For each group of initial sample data pairs, determine the noise type corresponding to the initial noise sample data, collect the noise corresponding to the noise type in the target scene, and obtain the target noise data containing the target environment reverberation; or

[0036] For each group of initial sample data pairs, the initial noise sample data is played in the target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain the target noise data containing the target environment reverberation.

[0037] In some embodiments, the mixing module is used to mix the second speech sample data and the target noise data based on a target signal-to-noise ratio for each group of second sample data pairs to obtain the second noisy speech sample data.

[0038] In some embodiments, the target noise data includes target noise data of multiple different noise types, the second noisy speech sample data is multiple, and the mixing module is used to:

[0039] Based on the target signal-to-noise ratio, respectively mixing the target noise data of the plurality of different noise types with the second speech sample data to obtain a plurality of second noisy speech sample data;

[0040] For target noise data of each noise type, the target noise data is mixed with at least one target noise data of a different noise type to obtain a plurality of mixed noise data; based on the target signal-to-noise ratio, the plurality of mixed noise data are respectively mixed with the second speech sample data to obtain a plurality of second noisy speech sample data.

[0041] In some embodiments, the second speech sample data includes second speech sample data of multiple different groups of people, the second noisy speech sample data is multiple, and the mixing module is used to:

[0042] Based on the target signal-to-noise ratio, the second speech sample data of the multiple different groups of people are mixed with the target noise data respectively to obtain a plurality of second noisy speech sample data;

[0043] For the second speech sample data of each group of people, the second speech sample data is mixed with at least one second speech sample data different from the group of people to obtain a plurality of mixed speech sample data; based on the target signal-to-noise ratio, the plurality of mixed speech sample data are respectively mixed with the target noise data to obtain a plurality of second noisy speech sample data.

[0044] On the other hand, a computer device is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the training method of the speech noise reduction model described in any of the above implementation methods.

[0045] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the training method of the speech noise reduction model described in any of the above implementations.

[0046] On the other hand, a computer program product is provided, which includes a computer program code, the computer program code is stored in a computer-readable storage medium, a processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the training method of the speech noise reduction model described in any of the above-mentioned implementations.

[0047] An embodiment of the present application provides a method for training a speech denoising model. The method obtains a basic speech denoising model that is common to a variety of scenarios based on sample data in a simulation scenario, and in a target scenario, performs reverberation processing on initial speech sample data and initial noise sample data to obtain speech sample data and noise data containing target environmental reverberation, and then obtains noisy speech sample data based on the speech sample data and noise data containing environmental reverberation, that is, obtains speech sample data and noisy speech sample data adapted to the target scenario; and then trains the basic speech denoising model based on the speech sample data and noisy speech sample data adapted to the target scenario, so that the trained target speech denoising model can be applicable to the target scenario, thereby denoising the noisy speech data in the target scenario based on the target speech denoising model, and improving the denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0050] Figure 2 It is a flow chart of a method for training a speech noise reduction model provided in an embodiment of the present application;

[0051] Figure 3 is a flowchart of another method for training a speech noise reduction model provided in an embodiment of the present application;

[0052] Figure 4 is a flowchart of another method for training a speech noise reduction model provided in an embodiment of the present application;

[0053] Figure 5 It is a block diagram of a training device for a speech noise reduction model provided in an embodiment of the present application;

[0054] Figure 6 is a block diagram of a terminal provided in an embodiment of the present application;

[0055] Figure 7 This is a block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0057] The terms "first", "second", "third" and "fourth" in the specification and claims of this application and the drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0058] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the sample data involved in this application were obtained with full authorization.

[0059] The training method of the speech noise reduction model provided in the embodiment of the present application can be executed by a computer device. In some embodiments, the computer device is at least one of a terminal or a server. Figure 1 , Figure 1A schematic diagram of the implementation environment of the training method of the speech noise reduction model provided in the embodiment of the present application, the implementation environment includes at least one of the terminal 10 and the server 20, the terminal 10 and the server 20 can be directly or indirectly connected by wired or wireless communication, and the present application does not limit this. The training method of the speech noise reduction model provided in the embodiment of the present application can be executed by the terminal 10 alone, or by the server 20, or by the terminal 10 or the server 20 through data interaction, and the embodiment of the present application does not limit this. In some embodiments, the server 20 undertakes the main computing work and the terminal 10 undertakes the secondary computing work; or, the server 20 undertakes the secondary computing work and the terminal 10 undertakes the main computing work; or, the server 20 and the terminal 10 adopt a distributed computing architecture for collaborative computing.

[0060] In some embodiments, the training method of the speech noise reduction model provided in the embodiments of the present application is applied in scenarios such as meetings, lectures, classes, workshops, etc. where noisy speech data needs to be denoised. For example, in a speech scenario, the collected noisy speech data of the speaker can be denoised based on the speech noise reduction model to obtain clean speech data with noise removed.

[0061] The terminal 10 is at least one of a mobile phone, a tablet computer, a PC (Personal Computer), etc. The server 20 can be at least one of a server, a server cluster consisting of multiple servers, a cloud server, a cloud computing platform, and a virtualization center.

[0062] Figure 2 is a flowchart of a method for training a speech noise reduction model provided in an embodiment of the present application. The method can be implemented by at least one of a terminal and a server. In the embodiment of the present application, the terminal and the server are collectively referred to as computer devices. Therefore, the method is executed by using a computer device as the execution subject. Figure 2 , the method comprising:

[0063] 201. A computer device trains a first speech noise reduction model based on multiple groups of first sample data pairs in simulation scenarios of multiple scenarios, each group of first sample data pairs includes first speech sample data and first noisy speech sample data, and the first speech sample data is the first noisy speech sample data after noise reduction.

[0064] In the embodiment of the present application, the first speech sample data and the first noisy speech sample data are both speech signals. The first speech sample data is clean speech sample data that does not contain noise but contains simulated environmental reverberation, and the first noisy speech sample data is noisy speech sample data that contains noise and simulated environmental reverberation. The simulated environmental reverberation is the environmental reverberation in a simulated scene. The simulated environmental reverberation of any simulated scene is generated based on the sound rebound phenomenon in the simulated scene, which is related to the size and shape of the simulated scene and the material in the simulated scene.

[0065] In an embodiment of the present application, the computer device learns a first noise reduction rule based on the multiple groups of first sample data pairs, and the first noise reduction rule refers to a rule for performing noise reduction on the first noisy speech sample data to obtain the first speech sample data, and then generates the first speech noise reduction model based on the first noise reduction rule. In an embodiment of the present application, the various scenarios include lectures, meetings, lectures, factories, subways, etc., which are not specifically limited here; the computer device trains the first speech noise reduction model based on the multiple groups of first sample data pairs in the simulation scenarios of the various scenarios, that is, obtains a basic speech noise reduction model that is common to various scenarios.

[0066] 202. The computer device obtains multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is first speech sample data that does not contain simulated environment reverberation, and the initial noise sample data is data obtained by removing speech data and simulated environment reverberation from the first noisy speech sample data.

[0067] In the embodiment of the present application, the initial speech sample data is clean speech sample data that does not contain noise and does not contain simulated environment reverberation.

[0068] 203. The computer device performs reverberation processing on initial voice sample data in multiple groups of initial sample data pairs in a target scene among multiple scenes to obtain second voice sample data in multiple groups of second sample data pairs containing target environment reverberation, and performs reverberation processing on initial noise sample data in multiple groups of initial sample data pairs to obtain target noise data in multiple groups of second sample data pairs.

[0069] In the embodiment of the present application, the target scene can be any scene in the multiple scenes for which speech noise reduction is to be performed. The target environmental reverberation is the environmental reverberation under the target scene, which is generated based on the sound rebound phenomenon under the target scene and is related to the size and shape of the target scene and the material in the target scene.

[0070] 204. The computer device mixes the second speech sample data and the target noise data in each group of second sample data pairs to obtain a plurality of second noisy speech sample data.

[0071] 205. The computer device trains the first speech noise reduction model based on multiple groups of third sample data pairs to obtain a target speech noise reduction model, where the target speech noise reduction model is used to reduce noise on noisy speech data in a target scenario, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

[0072] In an embodiment of the present application, the computer device learns a second noise reduction rule based on the multiple groups of third sample data pairs, and the second noise reduction rule refers to a rule for denoising the second noisy speech sample data to obtain second speech sample data, and then generates the target speech noise reduction model based on the second noise reduction rule.

[0073] An embodiment of the present application provides a method for training a speech denoising model. The method obtains a basic speech denoising model that is common to a variety of scenarios based on sample data in a simulation scenario, and in a target scenario, performs reverberation processing on initial speech sample data and initial noise sample data to obtain speech sample data and noise data containing target environmental reverberation, and then obtains noisy speech sample data based on the speech sample data and noise data containing environmental reverberation, that is, obtains speech sample data and noisy speech sample data adapted to the target scenario; and then trains the basic speech denoising model based on the speech sample data and noisy speech sample data adapted to the target scenario, so that the trained target speech denoising model can be applicable to the target scenario, thereby denoising the noisy speech data in the target scenario based on the target speech denoising model, and improving the denoising effect.

[0074] Figure 3 is a flowchart of a method for training a speech noise reduction model provided in an embodiment of the present application. The method can be implemented by at least one of a terminal and a server. In the embodiment of the present application, the terminal and the server are collectively referred to as computer devices. Therefore, the method is executed by using a computer device as the execution subject. Figure 3 , the method comprising:

[0075] 301. A computer device trains a first speech noise reduction model based on multiple groups of first sample data pairs in simulation scenarios of multiple scenarios, where each group of first sample data pairs includes first speech sample data and first noisy speech sample data, and the first speech sample data is the first noisy speech sample data after noise reduction.

[0076] Optionally, the process of training the first speech denoising model based on multiple groups of first sample data pairs by the computer device includes: for each group of first sample data pairs, the computer device inputs the first noisy speech sample data therein into the initial speech denoising model to obtain the predicted first speech data, and then adjusts the model parameters of the initial speech denoising model based on the loss value between the first speech data and the first speech sample data. The computer device iteratively performs the above steps based on the multiple groups of first sample data pairs until the current iteration reaches the stop condition to obtain the first speech denoising model.

[0077] In some embodiments, the computer device is configured as a terminal, which can obtain multiple groups of first sample data pairs based on simulation scenarios of multiple scenarios; optionally, the terminal is a simulator, which can simulate multiple scenarios, and then based on the simulator, it can generate noisy speech sample data and speech sample data to obtain multiple groups of first sample data pairs; or the terminal is a sound collection device, which is used to collect noisy speech sample data and speech sample data in simulation scenarios to obtain multiple groups of first sample data pairs.

[0078] 302. The computer device obtains multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is first speech sample data that does not contain simulated environment reverberation, and the initial noise sample data is data obtained by removing speech data and simulated environment reverberation from the first noisy speech sample data.

[0079] In one implementation, the initial sample data pairs and the first sample data pairs are stored correspondingly, and the computer device directly obtains the multiple groups of initial sample data pairs corresponding to the first sample data pairs, to obtain the multiple groups of initial sample data pairs.

[0080] 303. For each group of initial sample data pairs, the computer device plays the initial voice sample data in the target scene through the target playback device, collects sound in the target scene through the target sound collection device, obtains voice data containing environmental noise and target environmental reverberation, performs noise reduction processing on the voice data, and obtains second voice sample data containing target environmental reverberation.

[0081] Optionally, the computer device plays the initial voice sample data in the target scene through the target playback device in the designated environment corresponding to the target scene, and collects sound in the target scene through the target sound collection device. The designated environment only has environmental noise, but no other interference noise. The environmental noise is the inevitable noise in the target scene, and the other interference noise is the noise that can be avoided in the target scene. For example, in a meeting scene, the designated environment is a meeting room. At this time, the background noise in the meeting room is the inevitable environmental noise, and the sounds of keyboard tapping, door opening and closing, and clapping belong to other interference noise. In this way, when sound collection is performed in the presence of only environmental noise, the noise included in the obtained voice data is only environmental noise. The general noise reduction method can achieve stable noise reduction of the voice data, and then obtain the second voice sample data in the target scene.

[0082] It should be noted that different sound collection devices have different frequency responses, which indicate the difference in processing capabilities for signals of different frequencies. In the embodiment of the present application, in order to further obtain the second voice sample data matching the target scene, the target sound collection device is the sound collection device actually used in the target scene, which is used to collect the noisy voice data to be denoised, thereby making the second voice sample data more compatible with the target scene. The target playback device can also be the playback device actually used in the target scene, avoiding the need to configure other playback devices, which improves the convenience of playing the initial voice sample data.

[0083] In an embodiment of the present application, step 302 is used to implement a process of performing reverberation processing on initial voice sample data in a target scenario to obtain second voice sample data; however, for some complex scenarios, it cannot be guaranteed that only ambient noise exists in the specified environment. In order to ensure the efficiency and accuracy of obtaining the second voice sample data, in some embodiments, a computer device performs reverberation processing on initial voice sample data in multiple groups of initial sample data pairs to obtain second voice sample data in multiple groups of second sample data pairs containing target environment reverberation, including the following implementation method: the computer device obtains impulse response data in the target scenario for each group of initial sample data pairs, and performs convolution processing on the impulse response data and the initial voice sample data to obtain second voice sample data containing target environment reverberation.

[0084] The impulse response data reflects the target environment reverberation of the target scene; optionally, the computer device obtains the impulse response data of the target scene by measuring the impulse response of a room in the target scene. In this implementation, by obtaining the second voice sample data containing the target environment reverberation based on the impulse response data, it is avoided to perform noise reduction processing on the voice data containing environmental noise, thereby improving the efficiency of obtaining the second voice sample data.

[0085] 304. The computer device determines, for each group of initial sample data pairs, a noise type corresponding to the initial noise sample data, collects noise corresponding to the noise type in the target scene, and obtains target noise data including target environment reverberation.

[0086] The target noise data includes not only the target environment reverberation, but also the environmental noise in the target scene and the noise corresponding to the noise type of the initial noise sample data. The noise types include walking sounds, crying and laughing sounds, keyboard tapping sounds, moving tables and chairs sounds, wind sounds, rain sounds, tidal sounds, etc., which are not specifically limited here.

[0087] Among them, the computer device collects the noise corresponding to the noise type in the specified environment corresponding to the target scene. The specified environment only has the noise corresponding to the noise type, and no other sounds exist. The other sounds include the sounds generated by people talking and the sounds generated by the target playback device playing voice data, etc., so that pure noise data can be obtained. Optionally, the noise corresponding to the noise type is artificially made in the target scene, so that the noise can be flexibly made as needed, thereby improving the flexibility and efficiency of obtaining target noise data by collecting noise. Optionally, the computer device collects noise through the target sound collection device, so that the target noise data is more matched with the target scene.

[0088] In an embodiment of the present application, noise is collected in a target scene to obtain target noise data including target environment reverberation, so that the target noise data matches the target scene, and then noisy speech sample data matching the target scene can be obtained based on the target noise data.

[0089] In an embodiment of the present application, step 304 is used to implement reverberation processing on initial noise sample data in a plurality of initial sample data pairs to obtain target noise data in a plurality of second sample data pairs; in other embodiments, the process in which a computer device performs reverberation processing on initial noise sample data in a plurality of initial sample data pairs to obtain target noise data in a plurality of second sample data pairs also includes the following implementation method: for each group of initial sample data pairs, the computer device plays the initial noise sample data in a target scene through a target playback device, and performs sound collection in a target scene through a target sound collection device to obtain target noise data containing target environment reverberation.

[0090] In this implementation, since the target noise data is obtained by collecting sound in the target scene where the initial noise sample data is played, artificial noise is avoided and the convenience of obtaining the target noise data is improved.

[0091] 305. The computer device mixes the second speech sample data and the target noise data in each group of second sample data pairs to obtain a plurality of second noisy speech sample data.

[0092] In one implementation, the computer device mixes the second speech sample data and the target noise data based on the target signal-to-noise ratio for each group of second sample data pairs to obtain second noisy speech sample data.

[0093] Among them, the target signal-to-noise ratio refers to the ratio between the signal strength of the second speech sample data and the signal strength of the target noise data; optionally, the target signal-to-noise ratio in the embodiment of the present application is determined based on the signal-to-noise ratio of the noisy speech data in the target scene, so that the second noisy speech sample data is more matched with the target scene. And the noisy speech data in the target scene can have a variety of signal-to-noise ratios, so the target signal-to-noise ratio can be multiple. Accordingly, the computer device mixes the second speech sample data and the target noise data based on each target signal-to-noise ratio, and obtains multiple second noisy speech sample data, which further enriches the data volume of the second noisy speech sample data, thereby improving the noise reduction effect of the trained speech noise reduction model.

[0094] The initial noise sample data includes multiple noise types, and the target noise data includes multiple target noise data of different noise types, and the obtained second noisy speech sample data is multiple. Accordingly, in some embodiments, the computer device mixes the second speech sample data and the target noise data based on the target signal-to-noise ratio to obtain the second noisy speech sample data, including at least one of the following implementation methods:

[0095] In one implementation, the computer device mixes target noise data of multiple different noise types with the second speech sample data based on the target signal-to-noise ratio to obtain multiple second noisy speech sample data. In this implementation, by mixing target noise data of multiple different noise types with the second speech sample data, the multiple second noisy speech sample data cover multiple types of noise, enriching the noise types of the noisy speech sample data, and then performing model training based on the noisy speech sample data of multiple noise types, the universal applicability of the trained speech denoising model can be improved.

[0096] In another implementation, the computer device mixes the target noise data of each noise type with at least one target noise data of a different noise type to obtain a plurality of mixed noise data, and based on a target signal-to-noise ratio, mixes the plurality of mixed noise data with the second speech sample data respectively to obtain a plurality of second noisy speech sample data.

[0097] Optionally, the computer device mixes multiple target noise data based on a signal energy ratio, where the signal energy ratio refers to the ratio between the signal energies corresponding to the multiple target noise data respectively. The signal energy ratio can be set and changed as needed. For example, in order to make the signal energy distribution corresponding to the multiple target noise data in the mixed noise data more uniform, the signal energy ratio can be set to 1.

[0098] It should be noted that there may be multiple different types of noise at the same time in the target scene. In this implementation, by mixing target noise data of multiple different noise types, the mixed noise data is made to better match the target scene, thereby obtaining second noisy speech sample data that better matches the target scene.

[0099] It should be noted that the second speech sample data includes second speech sample data of multiple different groups of people, and thus the second noisy speech sample data obtained is multiple. Accordingly, in some embodiments, the computer device mixes the second speech sample data and the target noise data based on the target signal-to-noise ratio to obtain the second noisy speech sample data, including at least one of the following implementation methods:

[0100] In one implementation, the computer device mixes second speech sample data of multiple different groups of people with target noise data based on a target signal-to-noise ratio to obtain multiple second noisy speech sample data.

[0101] The crowd can be divided based on different criteria. If divided based on age, the crowd includes infants, teenagers, young people, middle-aged people, the elderly, etc.; if divided based on gender, the crowd includes men, women, etc. In the embodiments of the present application, no specific limitation is made to this.

[0102] In this implementation, by mixing the second speech sample data of multiple different groups of people with the target noise data respectively, the multiple second noisy speech sample data cover the speech of multiple groups of people. In this way, a third sample data pair is formed based on the second speech sample data and the second noisy speech sample data, thereby enriching the data amount of the third sample data pair. Then, model training is performed based on the third sample data pair, which can improve the universal applicability of the trained speech denoising model.

[0103] In another implementation, the computer device mixes the second speech sample data of each group of people with at least one second speech sample data different from that group of people to obtain multiple mixed speech sample data, and based on the target signal-to-noise ratio, mixes the multiple mixed speech sample data with the target noise data respectively to obtain multiple second noisy speech sample data.

[0104] Optionally, the computer device mixes the plurality of second voice sample data based on a signal energy ratio, and the signal energy ratio can be set and changed as required, which is not specifically limited herein.

[0105] It should be noted that in the target scene, there may be voice data of multiple different groups of people at the same time, such as in a chorus scene; and in this implementation method, by mixing different second voice sample data of multiple groups of people, the mixed voice sample data is made to better match the target scene, and then the second noisy speech sample data that better matches the target scene can be obtained.

[0106] It should be noted that the numbering of the steps in the embodiment of the present application is for the convenience of description and does not limit the execution order of the steps. For example, the above steps 302-305 can be executed after the above step 301, or before the above step 301, or can be executed synchronously with step 301. That is, in the embodiment of the present application, the first speech noise reduction model can be obtained based on the first sample data pair, or the third sample data pair can be obtained based on the initial sample data pair, or the first speech noise reduction model and the third sample data pair can be obtained at the same time. The embodiment of the present application does not limit this.

[0107] 306. The computer device trains the first speech noise reduction model based on multiple groups of third sample data pairs to obtain a target speech noise reduction model, where the target speech noise reduction model is used to reduce noise on noisy speech data in a target scenario, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

[0108] Optionally, the process of the computer device training the first speech noise reduction model based on multiple groups of third sample data pairs includes: the computer device inputs the second noisy speech sample data in each group of third sample data pairs into the first speech noise reduction model to obtain predicted second speech data, and then adjusts the model parameters of the first speech noise reduction model based on the loss value between the second speech data and the second speech sample data; the computer device iteratively executes the above steps based on multiple groups of third sample data pairs until this iteration reaches the stopping condition to obtain the target speech noise reduction model.

[0109] See also Figure 4 , Figure 4This is a training flow chart of a speech noise reduction model provided by an embodiment of the present application; wherein, a computer device plays and collects initial speech sample data in an actual interference-free environment of a target scene to obtain speech data, and after denoising the speech data, obtains clean second speech sample data, and the actual interference-free environment refers to an environment that only includes environmental noise; then the computer device collects noise in the actual environment of the target scene to obtain target noise data, and the noise is of the same type as the noise of the initial noise sample data; then the computer device mixes the target noise data and the second speech sample data to obtain second noisy speech sample data; finally, the computer device optimizes and trains the trained basic speech noise reduction model based on a specified data set to obtain a target speech noise reduction model suitable for the target scene, the basic speech noise reduction model is a first speech noise reduction model trained based on a first sample data pair in a simulation scene, and the specified data set includes multiple groups of third sample data pairs consisting of second speech sample data and second noisy speech sample data.

[0110] In an embodiment of the present application, a general speech noise reduction model is first trained based on sample data pairs in simulation scenarios of various scenarios, and then used as a basic speech noise reduction model, and fine-tuned by a training data set composed of sample data pairs in a target scenario, thereby realizing the application of data actually collected in the target scenario to model training, so that the target speech noise reduction model is suitable for the target scenario; and the speech noise reduction model in the simulation scenario is used as the basic speech noise reduction model, and then optimized and trained based on the sample data pairs in the target scenario, which can improve the efficiency of optimizing and training based on the sample data pairs in the target scenario, so as to quickly complete the optimization of the basic speech noise reduction model in the specified target scenario, and then can relatively quickly obtain speech noise reduction models suitable for multiple scenarios.

[0111] An embodiment of the present application provides a method for training a speech denoising model. The method obtains a basic speech denoising model that is common to a variety of scenarios based on sample data in a simulation scenario, and in a target scenario, performs reverberation processing on initial speech sample data and initial noise sample data to obtain speech sample data and noise data containing target environmental reverberation, and then obtains noisy speech sample data based on the speech sample data and noise data containing environmental reverberation, that is, obtains speech sample data and noisy speech sample data adapted to the target scenario; and then trains the basic speech denoising model based on the speech sample data and noisy speech sample data adapted to the target scenario, so that the trained target speech denoising model can be applicable to the target scenario, thereby denoising the noisy speech data in the target scenario based on the target speech denoising model, and improving the denoising effect.

[0112] The present application also provides a training device for a speech noise reduction model, see Figure 5 , the device comprises:

[0113] A first training module 501 is used to train a first speech denoising model based on a plurality of groups of first sample data pairs in simulation scenarios of a plurality of scenarios, wherein each group of first sample data pairs includes first speech sample data and first noisy speech sample data, and the first speech sample data is the first noisy speech sample data after denoising;

[0114] An acquisition module 502 is used to acquire multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is first speech sample data without simulated environment reverberation, and the initial noise sample data is data obtained by removing speech data and simulated environment reverberation from the first noisy speech sample data;

[0115] The processing module 503 is used to perform reverberation processing on the initial speech sample data in the multiple groups of initial sample data pairs in a target scene among multiple scenes to obtain the second speech sample data in the multiple groups of second sample data pairs containing the target environment reverberation, and perform reverberation processing on the initial noise sample data in the multiple groups of initial sample data pairs to obtain the target noise data in the multiple groups of second sample data pairs;

[0116] A mixing module 504 is used to mix the second speech sample data and the target noise data in each group of second sample data pairs to obtain a plurality of second noisy speech sample data;

[0117] The second training module 505 is used to train the first speech denoising model based on multiple groups of third sample data pairs to obtain a target speech denoising model. The target speech denoising model is used to denoise the noisy speech data in the target scene. Each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

[0118] In some embodiments, the processing module 503 is used to:

[0119] For each group of initial sample data pairs, the initial voice sample data is played in a target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain second voice sample data containing target environment reverberation; or

[0120] For each group of initial sample data pairs, impulse response data in a target scene is obtained, and convolution processing is performed on the impulse response data and the initial speech sample data to obtain second speech sample data containing target environment reverberation.

[0121] In some embodiments, the processing module 503 is used to:

[0122] For each group of initial sample data pairs, determine the noise type corresponding to the initial noise sample data, collect the noise corresponding to the noise type in the target scene, and obtain the target noise data containing the target environment reverberation; or

[0123] For each group of initial sample data pairs, the initial noise sample data is played in a target scene by a target playback device, and sound is collected in a target scene by a target sound collection device to obtain target noise data containing target environment reverberation.

[0124] In some embodiments, the mixing module 504 is configured to mix the second speech sample data and the target noise data based on the target signal-to-noise ratio for each group of second sample data pairs to obtain second noisy speech sample data.

[0125] In some embodiments, the target noise data includes target noise data of multiple different noise types, the second noisy speech sample data is multiple, and the mixing module 504 is used to:

[0126] Based on the target signal-to-noise ratio, target noise data of multiple noise types are mixed with the second speech sample data respectively to obtain multiple second noisy speech sample data;

[0127] For target noise data of each noise type, the target noise data is mixed with at least one target noise data of a different noise type to obtain a plurality of mixed noise data; based on a target signal-to-noise ratio, the plurality of mixed noise data are respectively mixed with the second speech sample data to obtain a plurality of second noisy speech sample data.

[0128] In some embodiments, the second speech sample data includes second speech sample data of multiple different groups of people, and the second noisy speech sample data is multiple, and the mixing module 504 is used to:

[0129] Based on the target signal-to-noise ratio, second speech sample data of multiple different groups of people are mixed with target noise data to obtain multiple second noisy speech sample data;

[0130] For the second speech sample data of each group of people, the second speech sample data is mixed with at least one second speech sample data different from the group of people to obtain multiple mixed speech sample data. Based on the target signal-to-noise ratio, the multiple mixed speech sample data are respectively mixed with the target noise data to obtain multiple second noisy speech sample data.

[0131] The embodiment of the present application provides a training device for a speech noise reduction model, which obtains a basic speech noise reduction model that is common to multiple scenarios based on sample data in a simulation scenario, and in a target scenario, performs reverberation processing on initial speech sample data and initial noise sample data to obtain speech sample data and noise data containing target environment reverberation, and then obtains noisy speech sample data based on the speech sample data and noise data containing environment reverberation, that is, obtains speech sample data and noisy speech sample data adapted to the target scenario; and then trains the basic speech noise reduction model based on the speech sample data and noisy speech sample data adapted to the target scenario, so that the target speech noise reduction model obtained by training can be applicable to the target scenario, thereby performing noise reduction on the noisy speech data in the target scenario based on the target speech noise reduction model, and improving the noise reduction effect.

[0132] It should be noted that the training device for the speech denoising model provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the training device for the speech denoising model provided in the above embodiment and the training method embodiment of the speech denoising model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0133] In some embodiments, the computer device is provided as a terminal. Figure 6 The structure block diagram of a terminal 600 provided by an exemplary embodiment of the present application is shown. The terminal 600 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 600 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0134] Typically, the terminal 600 includes a processor 601 and a memory 602 .

[0135] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0136] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one program code, which is used to be executed by the processor 601 to implement the training method of the speech noise reduction model provided in the method embodiment of the present application.

[0137] In some embodiments, the terminal 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608 and a power supply 609.

[0138] The peripheral device interface 603 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0139] The radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 604 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0140] The display screen 605 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to collect touch signals on the surface or above the surface of the display screen 605. The touch signal can be input to the processor 601 as a control signal for processing. At this time, the display screen 605 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 605 can be one, set on the front panel of the terminal 600; in other embodiments, the display screen 605 can be at least two, respectively set on different surfaces of the terminal 600 or in a folding design; in other embodiments, the display screen 605 can be a flexible display screen, set on the curved surface or folding surface of the terminal 600. Even, the display screen 605 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0141] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0142] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 601 for processing, or input them into the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 600. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 607 may also include a headphone jack.

[0143] The positioning component 608 is used to locate the current geographical location of the terminal 600 to implement navigation or LBS (Location Based Service). The positioning component 608 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0144] The power supply 609 is used to power various components in the terminal 600. The power supply 609 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0145] In some embodiments, the terminal 600 further includes one or more sensors 610 . The one or more sensors 610 include but are not limited to: an acceleration sensor 611 , a gyroscope sensor 612 , a pressure sensor 613 , an optical sensor 614 , and a proximity sensor 615 .

[0146] The acceleration sensor 611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 600. For example, the acceleration sensor 611 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for collecting game or user motion data.

[0147] The gyro sensor 612 can detect the body direction and rotation angle of the terminal 600, and the gyro sensor 612 can cooperate with the acceleration sensor 611 to collect the user's 3D actions on the terminal 600. The processor 601 can implement the following functions based on the data collected by the gyro sensor 612: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0148] The pressure sensor 613 can be set on the side frame of the terminal 600 and / or the lower layer of the display screen 605. When the pressure sensor 613 is set on the side frame of the terminal 600, it can detect the user's holding signal of the terminal 600, and the processor 601 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is set on the lower layer of the display screen 605, the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 605. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0149] The optical sensor 614 is used to collect the ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the display screen 605 according to the ambient light intensity collected by the optical sensor 614. Specifically, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is reduced. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 614.

[0150] The proximity sensor 615, also called a distance sensor, is usually arranged on the front panel of the terminal 600. The proximity sensor 615 is used to collect the distance between the user and the front of the terminal 600. In one embodiment, when the proximity sensor 615 detects that the distance between the user and the front of the terminal 600 is gradually decreasing, the processor 601 controls the display screen 605 to switch from the screen-on state to the screen-off state; when the proximity sensor 615 detects that the distance between the user and the front of the terminal 600 is gradually increasing, the processor 601 controls the display screen 605 to switch from the screen-off state to the screen-on state.

[0151] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the terminal 600, and the terminal 600 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0152] In some embodiments, the computer device is provided as a server, Figure 7It is a block diagram of a server provided in an embodiment of the present application. The server 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 701 and one or more memories 702, wherein the memory 702 is used to store executable program codes, and the processor 701 is configured to execute the above executable program codes to implement the training methods of the speech noise reduction models provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0153] In an exemplary embodiment, a storage medium including a program code is also provided, such as a memory 702 including a program code, and the program code can be executed by a processor 701 of a server 700 to complete the training method of the speech noise reduction model. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0154] An embodiment of the present application also provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the training method of the speech noise reduction model of any of the above-mentioned implementation methods.

[0155] An embodiment of the present application also provides a computer program product, which includes a computer program code, the computer program code is stored in a computer-readable storage medium, a processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the training method of the speech noise reduction model of any of the above-mentioned implementation methods.

[0156] In some embodiments, the computer program product involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0157] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a speech noise reduction model, characterized in that: The method comprises: A first speech denoising model is obtained by training a plurality of groups of first sample data pairs in simulation scenarios based on a plurality of scenarios, wherein each group of first sample data pairs comprises first speech sample data and first noisy speech sample data, wherein the first speech sample data is the first noisy speech sample data after denoising; Acquire multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is the first speech sample data without simulated environment reverberation, and the initial noise sample data is the data after removing speech data and simulated environment reverberation from the first noisy speech sample data; In a target scene among the multiple scenes, reverberation processing is performed on the initial speech sample data in the multiple groups of initial sample data pairs to obtain second speech sample data in multiple groups of second sample data pairs containing target environment reverberation, and reverberation processing is performed on the initial noise sample data in the multiple groups of initial sample data pairs to obtain target noise data in the multiple groups of second sample data pairs; the second speech sample data includes second speech sample data of multiple different groups of people; For each group of second sample data pairs, based on a target signal-to-noise ratio, the second speech sample data of the multiple different groups of people are respectively mixed with the target noise data to obtain a plurality of second noisy speech sample data; or, for the second speech sample data of each group of people, the second speech sample data is mixed with at least one second speech sample data of a different group of people to obtain a plurality of mixed speech sample data, and based on a target signal-to-noise ratio, the plurality of mixed speech sample data are respectively mixed with the target noise data to obtain a plurality of second noisy speech sample data; Based on multiple groups of third sample data pairs, the first speech denoising model is trained to obtain a target speech denoising model, and the target speech denoising model is used to denoise the noisy speech data in the target scene, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

2. The method according to claim 1, characterized in that The performing reverberation processing on the initial speech sample data in the plurality of groups of initial sample data pairs to obtain second speech sample data in a plurality of groups of second sample data pairs containing target environment reverberation includes: For each group of initial sample data pairs, the initial voice sample data is played in the target scene by a target playback device, sound is collected in the target scene by a target sound collection device to obtain voice data containing environmental noise and target environmental reverberation, and noise reduction processing is performed on the voice data to obtain the second voice sample data containing the target environmental reverberation; or For each group of initial sample data pairs, impulse response data in the target scene is obtained, and convolution processing is performed on the impulse response data and the initial speech sample data to obtain the second speech sample data containing the target environment reverberation.

3. The method according to claim 1, characterized in that The performing reverberation processing on the initial noise sample data in the plurality of groups of initial sample data pairs to obtain the target noise data in the plurality of groups of second sample data pairs includes: For each group of initial sample data pairs, determine the noise type corresponding to the initial noise sample data, collect the noise corresponding to the noise type in the target scene, and obtain the target noise data containing the target environment reverberation; or For each group of initial sample data pairs, the initial noise sample data is played in the target scene by a target playback device, and sound is collected in the target scene by a target sound collection device to obtain the target noise data containing the target environment reverberation.

4. The method according to claim 1, characterized in that: The target noise data includes target noise data of multiple different noise types, and the method further includes at least one of the following implementations: Based on the target signal-to-noise ratio, respectively mixing the target noise data of the plurality of different noise types with the second speech sample data to obtain a plurality of second noisy speech sample data; For target noise data of each noise type, the target noise data is mixed with at least one target noise data of a different noise type to obtain a plurality of mixed noise data; based on the target signal-to-noise ratio, the plurality of mixed noise data are respectively mixed with the second speech sample data to obtain a plurality of second noisy speech sample data.

5. A training device for a speech noise reduction model, characterized in that: The device comprises: A first training module is used to train a first speech denoising model based on a plurality of groups of first sample data pairs in simulation scenarios of a plurality of scenarios, wherein each group of first sample data pairs comprises first speech sample data and first noisy speech sample data, wherein the first speech sample data is the first noisy speech sample data after denoising; An acquisition module, used to acquire multiple groups of initial sample data pairs, each group of initial sample data pairs includes initial speech sample data and initial noise sample data, the initial speech sample data is the first speech sample data without simulated environment reverberation, and the initial noise sample data is the data obtained by removing speech data and simulated environment reverberation from the first noisy speech sample data; a processing module, configured to perform reverberation processing on the initial speech sample data in the plurality of groups of initial sample data pairs in a target scene among the plurality of scenes to obtain second speech sample data in a plurality of groups of second sample data pairs containing target environment reverberation, and perform reverberation processing on the initial noise sample data in the plurality of groups of initial sample data pairs to obtain target noise data in the plurality of groups of second sample data pairs; the second speech sample data includes second speech sample data of a plurality of different groups of people; a mixing module, configured to, for each group of second sample data pairs, mix the second speech sample data of the plurality of different groups of people with the target noise data respectively based on a target signal-to-noise ratio to obtain a plurality of second noisy speech sample data; or, for the second speech sample data of each group of people, mix the second speech sample data with at least one second speech sample data of a group different from the second speech sample data of the group to obtain a plurality of mixed speech sample data, and, based on a target signal-to-noise ratio, mix the plurality of mixed speech sample data with the target noise data respectively to obtain a plurality of second noisy speech sample data; The second training module is used to train the first speech noise reduction model based on multiple groups of third sample data pairs to obtain a target speech noise reduction model, wherein the target speech noise reduction model is used to reduce noise on noisy speech data in the target scene, and each group of third sample data pairs includes second speech sample data and second noisy speech sample data.

6. A computer device, characterized in that: The computer device includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the training method of the speech noise reduction model as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the training method of the speech noise reduction model as described in any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product includes a computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the training method of the speech noise reduction model as described in any one of claims 1 to 4.

Images