Speech enhancement method, computer device and storage medium

Abstract

The present application relates to the field of speech processing technology, and discloses a speech enhancement method, a computer device and a storage medium. The method comprises extracting a speech signal in a target scene, inputting a first acoustic feature signal in the speech signal into a trained speech enhancement model for speech enhancement, and obtaining the first target acoustic feature signal. Then, the first target acoustic feature signal and the phase of the first acoustic feature signal are synthesized to obtain a target speech signal. Since the trained speech enhancement model includes a preset number of short-term memory modules, the preset number of short-term memory modules are stacked into a network structure with a stacked structure, and the loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part and a cross-domain constraint loss part, the trained speech enhancement model performs speech enhancement on the first acoustic feature information, which can not only suppress or eliminate stationary noise and non-stationary noise, but also suppress or eliminate impact noise.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a speech enhancement method, a computer device and a storage medium. Background Art

[0002] With the rapid development of speech processing technology, speech enhancement technology has been widely used in different fields, such as live audio and video broadcasting, human-computer interaction, conference calls, etc. Speech enhancement refers to the process of extracting the purest original speech from noisy speech. The frequency domain single-channel model denoising method is often used. The frequency domain single-channel model denoising method suppresses or eliminates stationary noise and various non-stationary noises through masking. This method does not suppress or eliminate impact noise.

[0003] In actual application scenarios, the mixing of multiple sounds is inevitable, especially in scenarios where teaching resources are shared. The classroom needs to be equipped with multiple audio and video equipment to meet the needs of simultaneous classes in multiple locations. During the teaching process, the teacher's footsteps, the dragging of tables and chairs, the sound of the microphone rubbing against clothes, and the sound of car horns outside the classroom will inevitably be recorded by the microphone to form impact noise, which in turn affects the quality of voice received in remote classrooms.

[0004] Therefore, existing speech enhancement methods are still unable to be applied to scenes with impact noise and have the problem of poor adaptability. Summary of the invention

[0005] The present application provides a speech enhancement method, a computer device and a storage medium, which can not only suppress or eliminate stationary noise and non-stationary noise, but also suppress or eliminate impact noise, so that the speech enhancement method can be applied to scenes with impact noise, thereby improving the adaptability of the speech enhancement method.

[0006] In a first aspect, the present application provides a speech enhancement method, the method comprising:

[0007] Acquire speech signals in the target scene;

[0008] Extracting a first acoustic feature signal from the speech signal, inputting the first acoustic feature signal into a trained speech enhancement model for enhancement processing, and obtaining a first target acoustic feature signal;

[0009] The first target acoustic feature signal and the phase of the first acoustic feature signal are synthesized to obtain a target speech signal.

[0010] In a second aspect, the present application further provides a computer device, including:

[0011] Memory and processor;

[0012] The memory is used to store computer programs;

[0013] The processor is used to execute the computer program and implement the steps of the speech enhancement method described in the first aspect when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the speech enhancement method as described in the first aspect above.

[0015] The present application discloses a speech enhancement method, a computer device and a storage medium, which extracts a speech signal in a target scene, inputs a first acoustic feature signal in the speech signal into a trained speech enhancement model for speech enhancement, obtains the first target acoustic feature signal, and synthesizes the phase of the first target acoustic feature signal and the first acoustic feature signal to obtain a target speech signal; since the trained speech enhancement model includes a preset number of short-term memory modules, the preset number of short-term memory modules are stacked into a network structure with a stacked structure, and the loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part and a cross-domain constraint loss part, the trained speech enhancement model performs speech enhancement on the first acoustic feature information, which can not only suppress or eliminate stationary noise and non-stationary noise, but also suppress or eliminate impact noise. Therefore, the speech enhancement method provided in the embodiment of the present application can be applied to scenes with impact noise to improve the quality of speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a schematic flow chart of a speech enhancement method provided in an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of the training process of the speech enhancement model provided in the embodiment of the present application;

[0019] Figure 3 It is a schematic diagram of an application scenario of a speech enhancement method;

[0020] Figure 4 is a schematic diagram of the structure of a short-term memory module provided in an embodiment of the present application;

[0021] Figure 5 It is a structural diagram of a speech enhancement model provided in an embodiment of the present application;

[0022] Figure 6 It is a schematic block diagram of the structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide a speech enhancement method, apparatus, device and storage medium. The speech enhancement method provided in the embodiments of the present application can be used in speech enhancement scenarios with impulsive noise, such as speech enhancement scenarios for classroom impulsive noise, and can effectively improve the quality of speech.

[0028] For example, the speech enhancement method provided in the embodiment of the present application can be applied to a terminal or a server, by extracting the speech signal in the target scene, inputting the first acoustic feature signal in the speech signal into the trained speech enhancement model for speech enhancement, and obtaining the first target acoustic feature signal, synthesizing the phase of the first target acoustic feature signal and the first acoustic feature signal to obtain the target speech signal; since the trained speech enhancement model includes a preset number of FSMN modules, the preset number of FSMN modules are stacked into a network structure with a stacked structure, and the loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part, and a cross-domain constraint loss part, the trained speech enhancement model performs speech enhancement on the first acoustic feature information, which can not only suppress or eliminate stationary noise and non-stationary noise, but also suppress or eliminate impact noise. Therefore, the speech enhancement method provided in the embodiment of the present application can be applied to scenes with impact noise to improve the quality of speech.

[0029] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0030] See also Figure 1 , Figure 1 : is a schematic flow chart of a speech enhancement method provided in an embodiment of the present application. The speech enhancement method can be implemented by a terminal or a server, and the terminal can be a handheld terminal, a personal computer, a laptop, a wearable smart device or a robot, etc.; the server can be a single server or a server cluster, and the single server or the server cluster can be a local server, a cloud server, etc.

[0031] like Figure 1 As shown, the speech enhancement method provided in this embodiment specifically includes: Step S101 to Step S103. The details are as follows:

[0032] S101: Acquire a speech signal in a target scene.

[0033] The target scenario is an application scenario with impact noise, such as live audio and video broadcasting, human-computer interaction, and online conference call scenarios, where various noises are recorded by the microphone to form impact noise. Impact noise refers to noise whose energy increases sharply in a short period of time (usually tens to hundreds of milliseconds) compared to steady noise.

[0034] S102: extracting a first acoustic feature signal from the speech signal, inputting the first acoustic feature signal into a trained speech enhancement model for speech enhancement, and obtaining a first target acoustic feature signal.

[0035] Among them, the trained speech enhancement model includes a preset number of short-term memory FSMN (Feedforward Sequential Memory Networks, FSMN) modules, and the preset number of FSMN modules are stacked into a network structure with a stacked structure; the loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part, and a cross-domain constraint loss part. Since the FSMN module has a short-term memory function, adding the FSMN module to the speech enhancement model can effectively utilize the short-term memory function of FSMN, meet the purpose of real-time noise processing, and achieve a better noise reduction effect. The use of a loss function including a frequency domain loss part, a signal constraint loss part, and a cross-domain constraint loss part can effectively make the speech enhancement model pay more attention to the impact noise part, ensure the speech quality, and make the noise reduction output of the speech enhancement model more robust.

[0036] Specifically, the frequency domain loss part is adjusted using the data sample distribution correction parameter. There is a translation linear mapping relationship between the data sample distribution correction parameter of the frequency domain loss part and the energy value of the impact noise. The signal constraint loss part is used to retain the value of the loss function of the network model with a stacked structure when the expected output value of the network model with a stacked structure is greater than the actual output value, and to make the value of the loss function of the network model with a stacked structure zero when the expected output value is less than or equal to the actual output value. The cross-domain constraint loss part is used to perform cross-domain constraints from the frequency domain to the time domain when the frequency domain signal of the network model with a stacked structure is transformed into the time domain signal. It can output an auditory perception that is more in line with the human ear, while reducing the parameter quantity and complexity of the speech enhancement model, and enabling the speech enhancement model to have a certain dereverberation capability, which can meet the requirements of actual use in actual reverberation scenarios.

[0037] It should be understood that before inputting the first acoustic feature signal into the trained speech enhancement model for speech enhancement, a step of training the speech enhancement model is also included. The step of training the speech enhancement model can be completed before acquiring the speech signal in the target scene, or after acquiring the speech signal in the target scene. In other words, the step of training the speech enhancement model and the step of acquiring the speech signal in the target scene are in a parallel execution relationship, and one of them can be executed selectively.

[0038] For example, Figure 2 As shown, Figure 2 is a schematic diagram of the training process of the speech enhancement model provided in the embodiment of the present application. It should be understood that the training process of the speech enhancement model provided in the embodiment of the present application can be implemented by the terminal or the server. Specifically, when Figure 1When the speech enhancement method is implemented by a terminal, if the computing power of the terminal is limited, the training process of the speech enhancement model can be implemented by a server.

[0039] Correspondingly, if Figure 3 As shown, Figure 3 It is a schematic diagram of an application scenario of the speech enhancement method. In this embodiment, the speech enhancement method is implemented by terminal 301. The terminal 301 is an audio and video device in the classroom, which is used to meet the needs of taking classes in multiple locations at the same time. Due to the limited computing power of various audio and video devices, the speech enhancement model can be completed by the server 302 that is connected to the audio and video device. The server 302 sends the trained speech enhancement model to the terminal 301, and the terminal 301 performs speech enhancement processing based on the speech enhancement model. It should be understood that when Figure 1 When the speech enhancement method is implemented by a terminal, if the computing power of the terminal is sufficient, the training process of the speech enhancement model can be implemented by the terminal. The server can be a local server or a cloud server.

[0040] In addition, when Figure 1 When the speech enhancement method is implemented by a local server, the training process of the speech enhancement model can be implemented by a cloud server in addition to the local server, which is not limited in the embodiments of the present application.

[0041] Depend on Figure 2 It can be seen that the training steps of the speech enhancement model provided in the embodiment of the present application include: step S201 to step S204. The details are as follows:

[0042] S201: Acquire noisy speech signals and clean speech signals in multiple preset scenarios.

[0043] Specifically, noisy audio signals in multiple preset scenarios can be obtained as the noisy speech signal. The clean speech signal is an audio signal obtained when there is no noise in multiple preset scenarios.

[0044] S202: extracting a second acoustic feature signal from the noisy speech signal, and inputting the second acoustic feature signal into a preset network model to predict a second target acoustic feature signal.

[0045] The second acoustic characteristic signal is extracted from the noisy speech signal, including: performing a short-time Fourier transform (STFT) on the noisy audio signal, or processing the noisy audio signal by other methods such as wavelet transform, to extract the second acoustic characteristic signal from the noisy audio signal. The second acoustic characteristic signal includes but is not limited to an amplitude characteristic signal.

[0046] The preset network model may be a network model with a stacked structure, and the second acoustic feature signal is input into the network model with a stacked structure to calculate an early reverberation (or early reflection) convolution clean speech signal, and the early reverberation convolution clean speech signal is used as the second target acoustic feature signal.

[0047] Exemplarily, the inputting the second acoustic feature signal into the network model with a stacked structure to predict the second target acoustic feature includes: generating a first sequence of the second acoustic feature signal, inputting the first sequence into the network model with a stacked structure for analysis, and obtaining a second sequence of predicted acoustic feature signals output by any FSMN module in the network model with the stacked structure; after splicing the first sequence and the second sequence, inputting the next FSMN module adjacent to the any FSMN module for training until the second target acoustic feature signal is obtained.

[0048] Specifically, the FSMN module includes a network layer and a short-term memory module, and the short-term memory module stores a first sequence of the acoustic feature signal. The first sequence includes a current frame and at least one historical frame, or includes a current frame and at least one future frame; the at least one historical frame is located before the current frame in the time domain, and the at least one future frame is located after the current frame in the time domain, and the current frame is a frame analyzed by the network layer.

[0049] For example, Figure 4 As shown, Figure 4 Schematic diagram of the structure of the short-term memory module provided in the embodiment of the present application. Figure 4 It can be seen that the output of a certain layer in the FSMN module is h represents the output, the superscript l represents the output of the lth layer, and the subscript t represents the input as the acoustic features of the tth frame. It is concatenated with a certain number of historical output features (or future features) and then obtained through convolution or other methods. Then and Merge the input to the next layer of the network.

[0050] It should be understood that due to the characteristics of impact noise, the speech enhancement model needs to pay more attention to short-term memory to meet the requirement of suppressing noise in a shorter period of time. Therefore, the network structure of the speech enhancement model refers to the FSMN module and uses the short-term memory of limited historical frames. At the same time, in order to improve performance, some future frame information can also be introduced into the short-term memory. At the same time, multiple FSMN modules are stacked, so that the speech enhancement model has better performance in suppressing impact noise.

[0051] In some embodiments, the inputting the first sequence into the network model with a stacked structure for analysis includes: inputting the first sequence into the network model with a stacked structure; for any FSMN module, analyzing a current frame in the first sequence through a network layer of the FSMN module; and calculating a predicted acoustic feature signal of the current frame after splicing the analysis result with at least one future frame or at least one historical frame in the short-term memory module.

[0052] S203. Iteratively update the parameters of the network model based on the clean speech signal and the predicted second target acoustic feature.

[0053] Among them, the loss function of the network model with a stacked structure in the target optimization function Loss can be expressed as:

[0054] Loss=α*Freq mse +β*Freq sdr +γ*Temp mae

[0055] Among them, α, β, γ are loss adjustment factors, which can be any real number; Freq mse is the frequency domain loss part, Freq sdr is the signal constraint loss part, Temp mae is the cross-domain constraint loss part.

[0056] Specifically, in the process of iteratively updating the parameters of the network model with a stacked structure, in order to effectively suppress the impact noise, in the iterative update process, the training width (weight) of the model is designed and acts on the frequency domain loss part of the loss function of the network model, so as to achieve the optimization of the frequency domain loss part during model training. Specifically, the frequency domain loss part can be expressed as:

[0057] Freq mse =weigt*mse(pred,label)

[0058]

[0059] Among them, pred is the model output prediction target speech signal, w min , w max are the lower and upper limits of weight, respectively, N is the noise acoustic feature after standardization (minus the mean and divided by the variance); where α w and β w Represent the translation and linear mapping of affine transformation respectively. mse During the iterative update process, according to the characteristics of the impact noise, first adjust αw Make the center of the weight value range located at w min and w max Near the center, then adjust β w The value of weight makes most of the value range fall within w min and w max The distribution characteristics of the impulse noise can be retained to the greatest extent, so that the weight is close to w where the energy of the impulse noise is larger. max , the energy is smaller and close to w min , so as to achieve the suppression of impact noise by the trained speech enhancement model.

[0060] In addition, Freq sdr Part of this is to constrain the speech enhancement model to suppress the speech signal and reduce the damage to the speech signal. Since the amplitude of the impulse noise is generally large in the part where the impulse noise and speech are aliased, it is necessary to suppress the impulse noise with a large amplitude to achieve the effect of constraining the speech enhancement model to suppress the speech signal and effectively prevent the loss of the speech signal. Specifically, Freq sdr It can be expressed as:

[0061] Freq sdr =mse(label-pred,>0,0)

[0062] Among them, when the expected output speech signal label value is greater than the speech signal pred value predicted by the speech enhancement model, when the speech signal is damaged, retain Freq sdr When the label value is less than or equal to the pred value, the speech signal will not be damaged. At this time, Freq sdr The value of is 0.

[0063] In addition, the Temp of the loss function of the network model mae Temp is the cross-domain constraint loss part, which is used to perform cross-domain constraints from frequency domain to time domain on the speech enhancement model after STFT conversion from frequency domain to time domain and overlap addition, so as to achieve better noise suppression effect of the speech enhancement model. mae It can be expressed as:

[0064] Temp mae =mae(istft(noisy*pred),istft(noisy*label))

[0065] Among them, mae represents the mean absolute error, noisy is the noise signal, label is the expected output speech signal, pred is the speech signal predicted by the speech enhancement model, and isftt is the inverse transform of the short-time Fourier transform.

[0066] S204: According to preset model training conditions, determine that the training of the network type is completed to obtain the speech enhancement model.

[0067] The preset model training condition includes determining the end of training the network model with a stacked structure according to the value of the loss function or the number of iterations to obtain the speech enhancement model. Specifically, determining the end of training the network model according to the preset model training condition to obtain the speech enhancement model includes:

[0068] If the value of the loss function is less than a preset loss function threshold, or the number of iterations of the loss function is greater than a preset iteration threshold, it is determined that the training of the network model with a stacked structure is completed, and the speech enhancement model is obtained. Figure 5 As shown, Figure 5 It is a structural diagram of the speech enhancement model provided in the embodiment of the present application.

[0069] From the above analysis, it can be seen that the speech enhancement model provided in this embodiment is based on the characteristics of impact noise, takes advantage of the short-term memory of FSMN, and stacks multiple FSMN modules to obtain a better model for removing impact noise. And based on the characteristics of impact noise, in the loss function of model training, weight is used to adjust the frequency domain loss part. At the same time, to ensure the voice quality, the signal constraint loss part is constrained. In addition, adding cross-domain constraints from frequency domain to time domain further improves the performance of the speech enhancement model. Considering the reverberation factor of the actual scene, the expected output of the speech enhancement model is changed to early reverberation convolution clean speech, so that the speech enhancement model has a certain ability to remove reverberation and conforms to the auditory perception of the human ear.

[0070] S103: synthesize the first target acoustic feature signal and the phase of the first acoustic feature signal to obtain a target speech signal.

[0071] In some embodiments, the first target acoustic feature signal and the phase of the first acoustic feature signal are synthesized to obtain a target speech signal, including: multiplying the first target acoustic feature signal with the first acoustic feature signal to obtain an amplitude spectrum of the denoised speech signal; and synthesizing the amplitude spectrum with the phase of the first acoustic feature signal to obtain the target speech signal.

[0072] Through the above analysis, it can be known that the speech enhancement method provided in the embodiment of the present application extracts the speech signal in the target scene, inputs the first acoustic feature signal in the speech signal into the trained speech enhancement model for speech enhancement, and after obtaining the first target acoustic feature signal, synthesizes the phase of the first target acoustic feature signal and the first acoustic feature signal to obtain the target speech signal; since the trained speech enhancement model includes a preset number of FSMN modules, the preset number of FSMN modules are stacked into a network structure with a stacked structure, and the loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part and a cross-domain constraint loss part, the trained speech enhancement model performs speech enhancement on the first acoustic feature information, which can not only suppress or eliminate stationary noise and non-stationary noise, but also suppress or eliminate impact noise. Therefore, the speech enhancement method provided in the embodiment of the present application can be applied to scenes with impact noise to improve the quality of speech.

[0073] See also Figure 6 , Figure 6 1 is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device includes a processor, a memory and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0074] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the speech enhancement methods.

[0075] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0076] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any speech enhancement method.

[0077] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0078] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0079] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0080] Acquire speech signals in the target scene;

[0081] Extracting a first acoustic feature signal from the speech signal, inputting the first acoustic feature signal into a trained speech enhancement model for speech enhancement, and obtaining a first target acoustic feature signal;

[0082] The first target acoustic feature signal and the phase of the first acoustic feature signal are synthesized to obtain a target speech signal.

[0083] In some embodiments, before inputting the speech signal into a preset speech enhancement model for speech enhancement, the method further includes:

[0084] Obtain noisy speech signals and clean speech signals under multiple preset scenarios;

[0085] Extracting a second acoustic feature signal from the noisy speech signal, and inputting the second acoustic feature signal into a preset network model to predict a second target acoustic feature;

[0086] Iteratively updating the parameters of the network model based on the clean speech signal and the predicted second target acoustic feature signal;

[0087] According to the preset model training conditions, it is determined that the training of the network model with the stacking structure is completed, and the speech enhancement model is obtained.

[0088] In some embodiments, the step of inputting the second acoustic feature signal into a preset network model to predict a second target acoustic feature signal includes:

[0089] generating a first sequence of the second acoustic feature signal, inputting the first sequence into the network model for analysis, and obtaining a second sequence of predicted acoustic feature signals output by any short-term memory module in the network model;

[0090] After the first sequence and the second sequence are concatenated, the next short-term memory module adjacent to any one of the short-term memory modules is input for training until the second target acoustic feature signal is obtained.

[0091] In some embodiments, the short-term memory module includes a network layer and a short-term memory module, and the short-term memory module stores the first sequence of the acoustic feature signal;

[0092] Among them, the first sequence includes a current frame and at least one historical frame, or includes a current frame and at least one future frame; the at least one historical frame is located before the current frame in the time domain, and the at least one future frame is located after the current frame in the time domain, and the current frame is the frame analyzed by the network layer.

[0093] In some embodiments, inputting the first sequence into the network model for analysis includes:

[0094] inputting the first sequence into the network model;

[0095] For any short-term memory module, analyzing the current frame in the first sequence through the network layer of the short-term memory module;

[0096] After splicing the analysis result with at least one future frame or at least one historical frame in the short-term memory module, the predicted acoustic feature signal of the current frame is calculated.

[0097] In some embodiments, the frequency domain loss portion is adjusted using a data sample distribution correction parameter, and there is a translation linear mapping relationship between the data sample distribution correction parameter and the energy value of the impulse noise.

[0098] In some embodiments, the signal constraint loss part is used to retain the value of the loss function of the network model with a stacked structure when the expected output value of the network model with a stacked structure is greater than the actual output value, and to make the value of the loss function of the network model with a stacked structure zero when the expected output value is less than or equal to the actual output value.

[0099] In some embodiments, the cross-domain constraint loss part is used to perform cross-domain constraints from the frequency domain to the time domain when the frequency domain signal of the network model is transformed into the time domain signal.

[0100] In some embodiments, synthesizing the first target acoustic feature signal and the phase of the first acoustic feature signal to obtain the target speech signal includes:

[0101] Multiplying the first target acoustic feature signal by the first acoustic feature signal to obtain an amplitude spectrum of the denoised speech signal;

[0102] The amplitude spectrum is synthesized with the phase of the first acoustic feature signal to obtain the target speech signal.

[0103] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement the speech enhancement method provided in any embodiment of the present application.

[0104] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0105] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A speech enhancement method, characterized in that: The method comprises: Acquire speech signals in the target scene; Extracting a first acoustic feature signal from the speech signal, inputting the first acoustic feature signal into a trained speech enhancement model for speech enhancement, and obtaining a first target acoustic feature signal; synthesizing the first target acoustic feature signal and the phase of the first acoustic feature signal to obtain a target speech signal; The speech enhancement model includes a preset number of FSMN modules, which are stacked into a network structure with a stacked structure. The loss function of the network structure with a stacked structure includes a frequency domain loss part, a signal constraint loss part and a cross-domain constraint loss part. The frequency domain loss part is adjusted using a data sample distribution correction parameter, and there is a translation linear mapping relationship between the data sample distribution correction parameter and the energy value of the impact noise.

2. The method according to claim 1, characterized in that Before inputting the first acoustic feature signal into the trained speech enhancement model for speech enhancement, the method further includes: Obtain noisy speech signals and clean speech signals under multiple preset scenarios; Extracting a second acoustic feature signal from the noisy speech signal, and inputting the second acoustic feature signal into a preset network model to predict a second target acoustic feature signal; Iteratively updating the parameters of the network model based on the clean speech signal and the predicted second target acoustic feature signal; According to the preset model training conditions, it is determined that the training of the network model is completed, and the speech enhancement model is obtained.

3. The method according to claim 2, characterized in that The step of inputting the second acoustic feature signal into a preset network model to predict a second target acoustic feature signal includes: generating a first sequence of the second acoustic feature signal, inputting the first sequence into the network model for analysis, and obtaining a second sequence of predicted acoustic feature signals output by any FSMN module in the network model; After concatenating the first sequence and the second sequence, inputting the next FSMN module adjacent to any one of the FSMN modules for training; The second target acoustic feature signal is obtained according to the preset model training conditions.

4. The method according to claim 3, characterized in that The FSMN module includes a network layer and a short-term memory module, and the short-term memory module stores a first sequence of the acoustic feature signal; Among them, the first sequence includes a current frame and at least one historical frame, or includes a current frame and at least one future frame; the at least one historical frame is located before the current frame in the time domain, and the at least one future frame is located after the current frame in the time domain, and the current frame is the frame analyzed by the network layer.

5. The method according to claim 4, characterized in that The step of inputting the first sequence into the network model for analysis comprises: inputting the first sequence into the network model; For any FSMN module, analyzing the current frame in the first sequence through the network layer of the FSMN module; After splicing the analysis result with at least one future frame or at least one historical frame in the short-term memory module, the predicted acoustic feature signal of the current frame is calculated.

6. The method according to claim 2, characterized in that The signal constraint loss part is used to retain the value of the loss function of the network model when the expected output value of the network model is greater than the predicted output value, and to make the value of the loss function of the network model zero when the expected output value is less than or equal to the predicted output value.

7. The method according to claim 2, characterized in that The cross-domain constraint loss part is used to perform cross-domain constraints from the frequency domain to the time domain when the frequency domain signal of the network model is transformed into the time domain signal.

8. The method according to claim 1, characterized in that The synthesizing the first target acoustic feature signal and the phase of the first acoustic feature signal to obtain a target speech signal includes: Multiplying the first target acoustic feature signal by the first acoustic feature signal to obtain an amplitude spectrum of the denoised speech signal; The amplitude spectrum is synthesized with the phase of the first acoustic feature signal to obtain the target speech signal.

9. A computer device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the steps of the speech enhancement method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the steps of the speech enhancement method according to any one of claims 1 to 8.

Images