Howling suppression method, device and equipment and readable storage medium

Abstract

The embodiment of the invention provides a howling suppression method, device and equipment and a readable storage medium, a pre-trained neural network model is deployed on electronic equipment, and the neural network model comprises a first stage model and a second stage model which are connected in series. After the electronic device obtains a first voice signal, the first voice signal is sequentially subjected to howling suppression through the first stage model and the second stage model which are connected in series so as to obtain a second voice signal, and the second voice signal is played by the loudspeaker. According to the scheme, the first stage model and the second stage model which are connected in series are used for sequentially carrying out howling suppression on the first voice signal to obtain the second voice signal, the howling suppression and noise reduction functions are combined to enhance the first voice signal to obtain the second voice signal, and the purpose of improving the howling suppression and noise reduction effect is achieved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech processing, in particular to a howling suppression method and device, equipment and readable storage medium. BACKGROUND

[0002] In a sound amplification system, the voice signal collected by the microphone is transmitted to the loudspeaker for amplification and broadcast, and the voice signal played by the loudspeaker will be picked up by the microphone again. The transmission and feedback of voice signals between the loudspeaker and the microphone form an acoustic loop.

[0003] During the transmission of the voice signal, when the volume is large, the feedback loop of the sound forms positive feedback, that is, the acoustic loop gain is greater than 1. The sound is amplified step by step in the continuous feedback and produces a piercing howling sound, which seriously affects the user's auditory experience. To suppress the howling, common methods include frequency shifting and phase shifting method, notch method, adaptive feedback control method, etc. Among them, the frequency shifting and phase shifting method mainly changes the frequency or phase of the sound in real time during sound processing to destroy the phase characteristics required for positive feedback to occur. The notch method suppresses the howling by forcibly lowering the acoustic loop gain of the frequency point where the howling occurs through a notch filter. The adaptive feedback control method needs to converge to a very deep depth to suppress the howling.

[0004] However, the above howling suppression methods have poor quality and cannot completely eliminate the howling. SUMMARY

[0005] Embodiments of the present application provide a howling suppression method, device, equipment and readable storage medium, which suppresses howling and reduces noise in the first voice signal by using a first stage model and a second stage model connected in series to obtain a second voice signal, thereby achieving the purpose of improving the howling suppression effect.

[0006] In a first aspect, the embodiments of the present application provide a howling suppression method, comprising:

[0007] obtaining a first voice signal;

[0008] suppressing howling in the first voice signal by using a neural network model to obtain a second voice signal; wherein the neural network model comprises a first stage model and a second stage model connected in series, the first stage model is obtained by training a first initial model with a sample howling signal as input, and the second stage model is obtained by training a second initial model with a sample mixed signal as input of the first stage model after the first stage model is trained and frozen;

[0009] playing the second voice signal.

[0010] In a second aspect, the embodiments of the present application provide a method for training a neural network model for howling suppression, which comprises the following steps:

[0011] training a first initial model by using sample howling signals in a first training set to obtain the first-stage model, wherein the first initial model and the second initial model are connected in series;

[0012] freezing the weights of the first-stage model;

[0013] training the second initial model by inputting sample mixed signals in a second training set into the first-stage model to obtain a second-stage model;

[0014] generating a neural network model for howling suppression according to the first-stage model and the second-stage model.

[0015] In a third aspect, the embodiments of the present application provide a howling suppression device, which comprises:

[0016] an acquisition module, configured to acquire a first voice signal;

[0017] a processing module, configured to perform howling suppression on the first voice signal by using a neural network model to obtain a second voice signal, wherein the neural network model comprises a first-stage model and a second-stage model connected in series, the first-stage model is obtained by training a first initial model by using sample howling signals as input, and the second-stage model is obtained by training a second initial model by inputting sample mixed signals as input of the first-stage model after the first-stage model is trained and frozen;

[0018] a playing module, configured to play the second voice signal.

[0019] In a fourth aspect, the embodiments of the present application provide a training device, which comprises:

[0020] a first training module, configured to train a first initial model by using sample howling signals in a first training set to obtain the first-stage model, wherein the first initial model and the second initial model are connected in series;

[0021] a freezing module, configured to freeze the weights of the first-stage model;

[0022] a second training module, configured to train the second initial model by inputting sample mixed signals in a second training set into the first-stage model to obtain a second-stage model;

[0023] a generating module, configured to generate a neural network model for howling suppression according to the first-stage model and the second-stage model.

[0024] In a fifth aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the method according to the first aspect or any possible implementation manner of the first aspect. Alternatively, the electronic device implements the method according to the second aspect or any possible implementation manner of the second aspect.

[0025] In a sixth aspect, a computer readable storage medium is provided, which stores computer instructions. When the computer instructions are executed by a processor, the computer instructions are used to implement the method according to the first aspect or any possible implementation manner of the first aspect. Alternatively, the computer instructions are used to implement the method according to the second aspect or any possible implementation manner of the second aspect.

[0026] In a seventh aspect, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the computer program implements the method according to the first aspect or any possible implementation manner of the first aspect. Alternatively, the computer program implements the method according to the second aspect or any possible implementation manner of the second aspect.

[0027] The echo suppression method, device, electronic device, and readable storage medium provided in the embodiments of the present application deploy a pre-trained neural network model on the electronic device. The neural network model includes a first-stage model and a second-stage model connected in series. The first-stage model is obtained by training a first initial model with a sample echo signal as input. The second-stage model is obtained by training a second initial model with a sample mixed signal as input of the first-stage model after the first-stage model is trained and frozen. After the electronic device obtains a first voice signal, the first voice signal is sequentially subjected to echo suppression by the first-stage model and the second-stage model connected in series to obtain a second voice signal. The second voice signal is played by a loudspeaker. With this scheme, the first voice signal is enhanced by combining echo suppression and noise reduction functions to obtain the second voice signal, so as to achieve the purpose of improving echo suppression and noise reduction effects. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0029] Figure 1 is a schematic diagram of an echo generation process;

[0030] Figure 2 is a flowchart of a howling suppression method provided by an embodiment of the present application;

[0031] Figure 3 is a flowchart of howling suppression by a neural network model in a howling suppression method provided by an embodiment of the present application;

[0032] Figure 4 is a flowchart of a training method of a neural network model for howling suppression provided by an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of a neural network model used in a howling suppression method provided by an embodiment of the present application;

[0034] Figure 6 is a schematic diagram of a process of simulating howling generation in a howling suppression method provided by an embodiment of the present application;

[0035] Figure 7 is a schematic diagram of a training process of a neural network model in a howling suppression method provided by an embodiment of the present application;

[0036] Figure 8 is a structural schematic diagram of a first-stage model in a howling suppression method provided by an embodiment of the present application;

[0037] Figure 9A is a structural schematic diagram of an encoding module in Figure 8 ;

[0038] Figure 9B is a structural schematic diagram of a decoding module in Figure 8 ;

[0039] Figure 10 is a structural schematic diagram of a second-stage model in a howling suppression method provided by an embodiment of the present application;

[0040] Figure 11 is a schematic diagram of a howling suppression apparatus provided by an embodiment of the present application;

[0041] Figure 12 is a schematic diagram of a training apparatus provided by an embodiment of the present application;

[0042] Figure 13 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] Howling is an oscillation caused by positive feedback in a sound reinforcement system. The howling phenomenon refers to: a microphone in a sound reinforcement system collects a sound signal, converts the sound signal into an electrical signal, and the electrical signal is amplified by a power amplifier. After the amplified electrical signal is converted into a sound signal, it is played by a loudspeaker. Part of the sound signal played by the loudspeaker is collected by the microphone through various paths, forming a feedback path such as "loudspeaker → microphone → amplifier → loudspeaker", and the cycle is repeated to form positive feedback. When the amplitude balance condition and the phase balance condition are met at the same time, oscillation occurs, which is manifested as howling.

[0044] No matter in a local sound reinforcement system or an online conference, when there is positive feedback between the microphone for picking up sound and the loudspeaker for amplifying sound, that is, the signal recorded by the microphone is played by the loudspeaker placed in the same space, and then picked up again by the same microphone to form a closed acoustic loop, it is easy to produce a howling signal. For example, in a conference room or a teaching scene, a microphone collects a speaker's voice signal, which is converted into an electrical signal through analog-to-digital conversion and amplified, and finally played by a loudspeaker on site. The voice signal played by the loudspeaker is collected again by the microphone, and the cycle is repeated to produce howling.

[0045] For another example, in a conference scene, device A is located in a remote conference room, and devices B and C are in a local conference room. The microphones of devices B and C are turned on at the same time. The voice signal collected by device A in the remote conference room is played by the loudspeaker of device B, and the sound played by the loudspeaker of device B is picked up by the microphone of device C and played, and the cycle is repeated to produce howling.

[0046] Figure 1 is a schematic diagram of the howling occurrence process. Please refer to Figure 1 , the voice signal collected by the microphone reaches the loudspeaker after amplitude control and delay processing, the voice signal played by the loudspeaker is picked up again by the microphone, and the cycle is repeated to produce howling. Once the howling phenomenon occurs, it produces a harsh, sharp sound that can damage electronic devices. Moreover, it can easily harm the user's auditory system.

[0047] In order to suppress howling, common howling suppression algorithms include frequency shifting and phase shifting method, notch method, and adaptive feedback control method. Among them, the frequency shifting and phase shifting method is to destroy the positive feedback path, but the effect is limited and the sound quality is easily damaged. The notch method detects the howling frequency point after howling occurs, and then does notch filtering on the howling frequency point to remove the howling signal. The howling suppression effect depends heavily on whether the detection of the howling frequency point is accurate, and it is difficult to completely remove the howling signal. The adaptive feedback control method removes howling through a low-consumption feedback path. This method needs to converge very deeply to suppress howling, and when the relative positions of the loudspeaker and the microphone change, the filter needs to be re-converged.

[0048] Although the current research on howling suppression schemes is combined with deep learning, the deep learning-based howling suppression method is offline data, which does not match the actual real-time process, resulting in the effect of howling suppression being affected. Moreover, in some far-field pickup scenes, such as ring microphone ceiling scenes, the voice signal picked up by the microphone will also have a low signal-to-noise ratio problem, and noise reduction processing needs to be performed on the voice signal in order to improve the intelligibility of the voice.

[0049] Based on this, the embodiment of the application provides a howling suppression method, device, equipment and readable storage medium, which uses the first-stage model and the second-stage model in series to sequentially perform howling suppression on the first voice signal to obtain the second voice signal, thereby achieving the purpose of improving the howling suppression effect.

[0050] The execution subject of the embodiment of the application is an electronic device, which may be, for example, a conference central control device, a server, etc. In one way, the electronic device integrates a loudspeaker and a microphone, and in another way, the microphone, the loudspeaker and the electronic device are three independent devices. In yet another way, the electronic device integrates a microphone, and the loudspeaker is a separate device, or the electronic device integrates a loudspeaker, and the microphone is a separate device.

[0051] The howling suppression method provided by the embodiment is suitable for a local sound amplification system or an online conference system. When used for the local sound amplification system, the neural network model including the first-stage model and the second-stage model in series is pre-deployed on the electronic device. After the first voice signal of the speaker is collected by the microphone, the first-stage model and the second-stage model in series are used by the electronic device to sequentially perform howling suppression on the first voice signal to obtain the second voice signal, and the second voice signal is played by the loudspeaker.

[0052] When used for the online conference, the neural network model including the first-stage model and the second-stage model in series is pre-deployed on the device B and the device C in the local conference room. The voice signal from the A device in the remote conference room is played by the loudspeaker of the device B, the microphone of the device C picks up the voice signal played by the loudspeaker of the device B and / or other voice signals on site to obtain the first voice signal, and the first-stage model and the second-stage model in series are used by the device C to sequentially perform howling suppression on the first voice signal to obtain the second voice signal, and the second voice signal is played by the loudspeaker.

[0053] In the embodiment of the application, the first voice signal is, for example, a voice signal obtained by far-field pickup of the microphone. Far-field pickup refers to that the distance between the microphone and the sound source is greater than a preset distance, for example, 30 centimeters, and the distance between the microphone and the sound source is, for example, 3 meters, 5 meters, etc., which is not limited in the embodiment of the application.

[0054] Figure 2is a flowchart of a howling suppression method provided by an embodiment of the present application. The embodiment includes

[0055] 201, acquire a first speech signal.

[0056] In an embodiment of the present application, an electronic device acquires a first speech signal picked up by a microphone. For example, in a teaching scenario or a conference scenario, a microphone and a loudspeaker are located in the same space, and a speaker speaks during the speaking process, the microphone continuously picks up the first speech signal, and the electronic device acquires the first speech signal. For another example, in an online conference scenario, a device B in a local conference room plays a speech signal from a remote device A, a microphone of a device C picks up the sound played by the loudspeaker of the device B and other sounds in the field, thereby obtaining the first speech signal.

[0057] 202, suppress howling of the first speech signal by using a neural network model to obtain a second speech signal.

[0058] The neural network model includes a first-stage model and a second-stage model connected in series, the first-stage model is obtained by training a first initial model with a sample howling signal as input, and the second-stage model is obtained by training a second initial model with a sample mixed signal as input of the first-stage model after the first-stage model is trained and frozen.

[0059] In an embodiment of the present application, a pre-trained neural network model is deployed on an electronic device. The model, also known as a two-stage model, includes a first-stage model and a second-stage model connected in series, as well as a Fourier transform module, a filtering module, a point multiplication module, and the like. The first-stage model is used to suppress howling of a first speech signal, and the second-stage model is used to further suppress howling and reduce noise of the output of the first-stage model.

[0060] 203, play the second speech signal.

[0061] In this step, the signal after howling suppression, i.e., the second speech signal, is played by a loudspeaker.

[0062] The whistle suppression method provided in the embodiments of the present application is to deploy a pre-trained neural network model on an electronic device, the neural network model comprising a first-stage model and a second-stage model connected in series, the first-stage model being obtained by training a first initial model with a sample whistle signal as input, and the second-stage model being obtained by training a second initial model with a sample mixed signal as input of the first-stage model after the first-stage model is trained and frozen. After the electronic device obtains a first speech signal, the first speech signal is sequentially subjected to whistle suppression by using the first-stage model and the second-stage model connected in series to obtain a second speech signal, which is played by a loudspeaker. By using the first-stage model and the second-stage model connected in series to sequentially perform whistle suppression on the first speech signal to obtain the second speech signal, i.e., by combining the whistle suppression and noise reduction functions to enhance the first speech signal to obtain the second speech signal, the purpose of improving the whistle suppression and noise reduction effects is achieved.

[0063] Figure 3 The flowchart in FIG. 1 is a process of whistle suppression on a first speech signal by using a neural network model in the whistle suppression method provided in the embodiments of the present application. The embodiments include the following steps.

[0064] 301, determining a first complex frequency spectrum of the first speech signal.

[0065] In this step, the electronic device determines a time domain signal according to the first speech signal, and performs short-time Fourier transform on the time domain signal to obtain a first complex frequency spectrum. The first complex frequency spectrum is also referred to as a first complex spectrum.

[0066] 302, inputting the first complex frequency spectrum into the first-stage model to make the first-stage model output filter coefficients.

[0067] The electronic device inputs the real part and the imaginary part of the first complex frequency spectrum into the first-stage model to make the first-stage model output filter coefficients, which are also referred to as complex filter information.

[0068] 303, filtering the first complex frequency spectrum according to the filter coefficients to obtain a second complex frequency spectrum.

[0069] In this step, the electronic device filters the first complex frequency spectrum according to the filter coefficients to obtain an output complex frequency spectrum of the first stage, i.e., a second complex frequency spectrum.

[0070] 304, inputting the second complex frequency spectrum into the second-stage model to make the second-stage model output a mask matrix.

[0071] In this step, the electronic device inputs the second complex frequency spectrum into the second-stage model to make the second-stage model output a mask matrix.

[0072] 305. generating the second speech signal according to the mask matrix.

[0073] In this step, the electronic device performs point multiplication and other processing on the mask matrix and the second complex spectrum, and further obtains the second speech signal.

[0074] With this scheme, the electronic device combines filtering processing and mask processing to suppress howling and reduce noise of the speech signal, greatly improving the effect of howling suppression and noise reduction.

[0075] Optionally, in the above Figure 3 In the embodiment shown in the figure, in the process of generating the second speech signal according to the mask matrix, first, the electronic device determines a third complex spectrum according to the mask matrix and the second complex spectrum. Then, the electronic device generates a time domain signal according to the third complex spectrum, and generates the second speech signal according to the time domain signal.

[0076] For example, the electronic device performs point multiplication on the mask matrix and the second complex spectrum to obtain the output complex spectrum of the second stage, i.e. the third complex spectrum. Then, the electronic device performs inverse transform of short-time Fourier transform on the third complex spectrum to obtain a time domain signal, which is also called amplitude time domain signal. The electronic device converts the second speech signal according to the time domain signal.

[0077] With this scheme, the electronic device combines point multiplication processing and inverse transform of short-time Fourier transform to determine the second speech signal, achieving the purpose of accurately and efficiently determining the second speech signal.

[0078] The above mainly describes how to use the neural network model to suppress howling and reduce noise of the first speech signal, i.e. mainly describes the application of the neural network model. However, the prerequisite for applying the neural network model is to pre-train the neural network model and deploy it on the electronic device. Next, how to train the neural network model is described in detail.

[0079] Figure 4 is a flowchart of a training method of a neural network model for howling suppression provided by the embodiment of the present application. The execution subject of the embodiment is an electronic device, such as a server, a desktop computer, a notebook computer, etc. The method comprises:

[0080] 401. training a first initial model with the sample howling signal in the first training set to obtain the first stage model, wherein the first initial model and the second initial model are connected in series.

[0081] In the embodiments of the present application, the electronic device pre-constructs a first training set, and the first training set contains multiple sample howling signals. For example, an acoustic loop is pre-constructed, and a clean speech signal is collected by using a microphone in the acoustic loop. The clean speech signal is played by a loudspeaker after passing through the acoustic loop. The speech signal played by the loudspeaker is collected by the microphone again, and the signal collected by the microphone again is referred to as a real-time feedback signal. The electronic device takes the sum signal of the clean speech signal and the real-time feedback signal as the input of the microphone, and takes the output signal of the loudspeaker as the sample howling signal. For another example, the electronic device determines the real-time feedback signal of the clean speech signal in a simulation manner, and takes the sum signal of the clean speech signal and the real-time feedback signal as the input of the microphone, and takes the output signal of the loudspeaker as the sample howling signal.

[0082] After the first training set is constructed, the electronic device trains a first initial model by using the sample howling signals in the first training set to obtain a first-stage model. The first-stage model is, for example, a model containing an encoding module (Encoder Block), a decoding module (Decoder Block), etc. The first-stage model is mainly used for howling suppression in a far-field sound pickup scene.

[0083] 402, freeze the weights of the first-stage model.

[0084] In this step, the electronic device freezes the weights of the first-stage model, that is, the parameters of the first-stage model remain unchanged in the process of training the second-stage model.

[0085] 403, input the sample mixed signals in the second training set into the first-stage model to train the second initial model to obtain a second-stage model.

[0086] In this step, the electronic device pre-constructs a second training set, and the second training set contains multiple sample mixed signals. The sample mixed signal is a signal obtained by mixing a clean speech signal and a sample noise signal.

[0087] After the second training set is constructed, the electronic device connects the first-stage model and the second initial model in series, takes the sample mixed signals in the second training set as the input of the first-stage model, and trains the second initial model to obtain a second-stage model. The second-stage model is, for example, a model containing an encoding module (Encoder Block), a decoding module (Decoder Block), etc. The second-stage model has both howling suppression and noise reduction functions. In addition, the first initial model and the second initial model can be connected in series, the first-stage model is trained first, then the first-stage model is frozen, the sample mixed signals in the second training set are taken as the input of the first-stage model, and the second initial model is trained to obtain the second-stage model.

[0088] 404. generate a neural network model for howling suppression according to the first stage model and the second stage model.

[0089] After the first stage model and the second stage model are trained, the electronic device generates a neural network model for howling suppression according to the first stage model, the second stage model, the Fourier transform module, the filtering module, the point multiplication module, and the like.

[0090] The training method for the neural network model for howling suppression provided in the embodiments of the present application is that the first initial model and the second initial model are connected in series. After the electronic device trains the first initial model by using the sample howling signals in the first training set to obtain the second stage model, the parameter weights of the first stage model are frozen, and the sample mixed signals in the second training set are used as the input of the first stage model to train the second initial model to obtain the second stage model. Then, the electronic device generates a neural network model for howling suppression according to the first stage model and the second stage model. By using this scheme, after the electronic device trains the first stage model, the parameter weights of the first model are frozen and the second stage model is trained, and a neural network model for howling suppression is generated according to the first stage model and the second stage model. The neural network model combines the howling suppression and the noise reduction functions, and the purpose of improving the speech enhancement performance of the neural network model is achieved.

[0091] As can be seen from the above, in the embodiments of the present application, the electronic device simulates the acoustic feedback process to construct the first training set, and trains the first initial model by using the sample howling signals in the first training set to obtain the first stage model. Then, the electronic device mixes the clean speech signals and the sample noise signals to obtain the sample mixed signals, constructs the second training set according to the sample mixed signals, and trains the second initial model by using the sample mixed signals in the second training set to obtain the second stage model. The first stage model and the second stage model are connected in series to generate a neural network model. Finally, the electronic device applies the trained neural network model to the acoustic feedback path, that is, uses the acoustic feedback model to perform howling suppression and noise reduction on the first speech signals in teaching scenes, conference scenes, and the like. For example, please refer to Figure 5 .

[0092] Figure 5 is a schematic diagram of the neural network model used in the howling suppression method provided in the embodiments of the present application. Please refer to Figure 5The neural network model comprising the first-stage model and the second-stage model in series is located in the acoustic feedback path. For example, the microphone, the electronic device, and the loudspeaker are independent devices, the neural network model is deployed on the electronic device, and the electronic device also has functions of electrical signal conversion, amplitude control, delay processing, etc. The first voice signal picked up by the microphone is processed by amplitude control, delay, etc., and then enters the neural network model for howling suppression and noise reduction to obtain the second voice signal. Then, the loudspeaker plays the second voice signal.

[0093] Next, the construction of the first training set, the construction of the second training set, the training of the first-stage model, and the training of the second-stage model are described in detail.

[0094] First, the construction of the first training set.

[0095] The electronic device obtains at least one signal pair, and each signal pair comprises a clean voice signal and a real-time feedback signal. The real-time feedback signal is, for example, a signal simulated or emulated by the clean voice signal picked up by the microphone and played by the loudspeaker and then picked up by the microphone again, so as to obtain the real-time feedback signal. For example, the clean voice signal is x clean The simulated real-time feedback signal xfeedback is:

[0096] After the electronic device obtains at least one signal pair, the electronic device simulates the howling occurrence process for each signal pair to obtain at least one sample howling signal. In the simulation of the howling occurrence process, the electronic device models the acoustic feedback path. The sum of the clean voice signal x clean and the real-time feedback signal xfeedback is taken as the input of the microphone, and the output signal of the loudspeaker is taken as the sample howling signal xhowling. Figure 6 is a schematic diagram of the process of simulating howling occurrence in the howling suppression method provided by the embodiments of the present application.

[0097] Please refer to Figure 6 , the electronic device simulates the howling occurrence process, and takes the sum of the clean voice signal x clean and the real-time feedback signal xfeedback as the input of the microphone. The sum signal picked up by the microphone is processed by gain, delay, and non-linear distortion, and then played by the loudspeaker. The electronic device takes the signal played by the loudspeaker as the sample howling signal x howling . The generation of the sample howling signal x howling is shown in formula (1):

[0098]

[0099] wherein, represents a convolution operation, h(t) represents a room impulse response, G represents a gain, t represents time, Δt represents a time delay which varies, and NL represents a non-linear process. The h(t) can be obtained by measurement or simulation. The non-linear process NL includes a hard clipping function and a memoryless sigmoid function. The non-linear process NL is shown in the following formula (2):

[0100]

[0101] wherein, is a memoryless sigmoid function, and a factor b(t) in the memoryless sigmoid function is shown in the following formula (3):

[0102] b(t) = 1.5 x x(t) - 0.3 x x(t) hard hard 2 x(t) formula (3)

[0103] wherein, x hard (t) is a hard clipping function, and is shown in the following formula (4):

[0104]

[0105] x(t) represents a signal value at time t, and x max represents a threshold value.

[0106] In the formula (2) to formula (4), the gain γ is set to 4, for example, x max is set to 1, for example, and a is a simulation coefficient, and a range of a is [0.2, 4], for example, that is, a is randomly selected in the range of 0.2 to 4 to simulate different degrees of non-linear distortion.

[0107] According to the formula (1) to formula (4), the electronic device simulates a howling generation process for each signal pair to obtain at least one sample howling signal. Then, the electronic device constructs a first training set by using the sample howling signals x howling . Subsequently, the sample howling signals x howling in the first training set are used to train a first stage model.

[0108] By using this scheme, the electronic device constructs the first training set by simulating the howling generation process, and the sample howling signals are generated according to the clean speech signal and the real-time feedback signal, so that the training of the first stage model uses online real-time data, which achieves the purposes of improving the howling suppression performance of the first stage model and quickly constructing a high-quality first training set.

[0109] Secondly, construction of a second training set.

[0110] ​An electronic device mixes sample noise signals and clean speech signals, and uses the resulting mixed signal as the sample mixed signal. In one approach, the electronic device maintains a fixed signal-to-noise ratio (SNR) and mixes the clean speech signal and sample noise signals at this fixed SNR to obtain the sample mixed signal.

[0111] In another approach, the electronic device mixes the sample noise signal and the clean speech signal with different signal-to-noise ratios to obtain a sample mixed signal.

[0112] For example, the sample mixed signal is x mix The clean speech signal is x clean The sample noise signal is x noise Then: x mix =x clean +x noise The clean speech signal is x. clean And the sample noise signal is x noise The following formula must be satisfied:

[0113]

[0114] Here, SNR represents the signal-to-noise ratio, which can take different values, and pow represents power. To obtain different SNR values, we can first fix x. clean Amplitude, then adjust x noise The amplitude of x causes SNR to take different values. Then, adjust x... mix The magnitude of x, thus making x mix The range does not exceed 1.

[0115] By adopting this approach, which takes into account both mixed sample noise signals and clean speech signals with different signal-to-noise ratios, a more universal sample mixed signal is obtained, thereby achieving the goal of training a high-precision second-stage model.

[0116] Next, the training of the model in the first stage.

[0117] Figure 7 This is a schematic diagram illustrating the training process of the neural network model in the howling suppression method provided in this application embodiment. Please refer to... Figure 7 After constructing the first training set, the electronic device uses the sample howling signals from the first training set to train the first initial model to obtain the first stage model. During the training process of the first stage model, the electronic device uses the sample howling signals x from the first training set... howling Input the first initial model, and input the sample howling signal x howling The corresponding clean speech signal x clean The first initial model is trained using the target signal.

[0118] Please refer to Figure 7, the electronic device determines a time-domain howling signal x(t) of the sample howling signal, performs short-time Fourier transform analysis on the time-domain howling signal x(t) to obtain a complex frequency spectrum X1(k, f), and the complex frequency spectrum X1(k, f) is an input feature of the first initial model. Wherein, k represents a time frame, and f represents a frequency. Then, the electronic device takes the real part and the imaginary part of the complex frequency spectrum X1(k, f) as two paths as the input of the first initial model, so that the first initial model outputs filter coefficients C N (k, f), and N represents an order. In the subsequent filtering process, the electronic device filters the complex frequency spectrum X1(k, f) according to the filter coefficients C N (k, f) to obtain a new complex frequency spectrum X2(k, f), and the filtering process is shown in the following formula (5):

[0119]

[0120] Wherein, N represents an order.

[0121] In the training process of the first stage model, the electronic device takes the clean speech signal corresponding to the sample howling signal as a target signal, performs short-time Fourier transform analysis on the target signal to obtain a target complex frequency spectrum, and determines whether the first initial model converges according to the first loss function and the target signal. The first loss function is the mean square error of the target complex frequency spectrum of the target signal and the output complex frequency spectrum of the first stage. Please refer to Figure 7 It can be understood that the output complex frequency spectrum of the first stage is X2(k, f). After the first initial model outputs the filter coefficients C N (k, f), the electronic device filters the complex frequency spectrum X1(k, f) according to the filter coefficients C N (k, f) to obtain a new complex frequency spectrum X2(k, f).

[0122] While the electronic device continuously adjusts the parameters of the first initial model, it determines whether the first initial model converges. If the first initial model does not converge, the electronic device continues to determine the time-domain howling signal x(t) of the sample howling signal and inputs it to the first initial model to continue training the first initial model. If the first initial model converges, the electronic device takes the converged first initial model as the first stage model.

[0123] Figure 8 is a structural diagram of the first stage model in the howling suppression method provided by the embodiment of the application. Please refer to Figure 8 The first stage model includes an encoding module (Encoder Block), a decoding module (Decoder Block), and an intermediate layer that plays a connecting role, the input of the first stage model is a complex frequency spectrum X1(k, f), and the output is a filter coefficient. Wherein, the encoding module and the decoding module are each, for example, 4, such as Figure 8The encoding modules in the first stage model include encoding module 2→16, encoding module 16→32, encoding module 32→64, and encoding module 64→128; and the decoding modules include decoding module 128→64, decoding module 64→32, decoding module 32→16, and decoding module 16→8. The intermediate layer is, for example, a Long Short-Term Memory (LSTM) network. The decoding side further includes a convolutional layer Conv2d8→8 and the like. The number of channels of each layer of the encoding modules and the decoding modules is as shown in Table 1. Figure 8

[0124] It should be noted that, although Figure 8 is described by taking an example in which the first stage model includes four encoding modules and four decoding modules. However, the embodiments of the present application are not limited thereto, and in other feasible implementation manners, the number of encoding modules and decoding modules can also be set according to requirements.

[0125] Figure 9A is Figure 8 a structural diagram of an encoding module in the first stage model. Please refer to Figure 9A An encoding module includes a convolutional layer (Conv2d), a normalization layer (BatchNorm), and an activation layer (Leaky_relu). The convolution kernel size of the encoding module is (3×3), and the step size of the convolution kernel is, for example, (2, 1), that is, the frequency domain step size is 2, and the time domain step size is 1. The convolution is, for example, a causal convolution to meet the requirement of real-time processing.

[0126] Figure 9B is Figure 8 a structural diagram of a decoding module in the first stage model. Please refer to Figure 9B A decoding module includes a transposed convolutional layer (ConvTranspose2d), a normalization layer (BatchNorm), and an activation layer (Leaky_relu). The convolution kernel size of the transposed layer of the decoding module is (3×3), and the step size of the convolution kernel is, for example, (2, 1), that is, the frequency domain step size is 2, and the time domain step size is 1. The convolution is, for example, a causal convolution to meet the requirement of real-time processing.

[0127] By adopting this scheme, the electronic device constructs a first loss function according to the mean square error between the target complex frequency spectrum of the target signal and the output complex frequency spectrum of the first stage, and determines whether the first initial model converges by using the first loss function and the target signal, thereby achieving the purpose of quickly and accurately determining whether the first initial model converges.

[0128] It should be noted that the construction of the first training set and the construction of the second training set do not have a strict sequence. The construction of the second training set and the training of the first stage model also do not have a strict sequence.

[0129] Finally, the training of the second stage model.

[0130] Please refer to​Figure 7 After the second training set is constructed and the first-stage model is trained, the electronic device trains a second initial model using the sample howling signals in the second training set to obtain a second-stage model. In the training process of the second-stage model, the electronic device first freezes the parameter weights of the first-stage model, and then trains the second initial model using the sample mixed signals in the second training set to obtain the second-stage model.

[0131] By adopting this scheme, after the electronic device trains the first-stage model, the weights and parameters of the first-stage model are frozen to train the second-stage model. Through the two-stage freezing and training means, the final neural network model is a two-stage network structure, which simultaneously achieves howling suppression and noise reduction without introducing delay, so that the residual amounts of noise and howling are very small, and the purpose of improving the howling suppression and noise reduction effects of the neural network model is achieved.

[0132] After the electronic device freezes the parameter weights of the first-stage model, in the process of training the second initial model using the sample mixed signals in the second training set to obtain the second-stage model, the short-time Fourier transform analysis is performed on the time domain signal of the sample mixed signal x mix to obtain a complex frequency spectrum, the complex frequency spectrum is input into the first-stage model to output filter coefficients, and the corresponding complex frequency spectrum of the sample mixed signal x mix is filtered according to the filter coefficients to obtain a new complex frequency spectrum, that is, the complex frequency spectrum X2(k, f) in Figure 7 .

[0133] After the electronic device obtains the complex frequency spectrum X2(k, f), the complex frequency spectrum X2(k, f) is input into the second initial model to output a mask matrix M(k, f). Then, the electronic device performs point multiplication processing on the mask matrix and the aforementioned complex frequency spectrum X2(k, f) to obtain the output complex frequency spectrum Y(k, f) of the second stage, and performs inverse Fourier transform on the output complex frequency spectrum Y(k, f) to obtain an output time domain signal y(t). The point multiplication processing is shown in the following formula (6):

[0134] Y(k, f) = M(k, f) x X2(k, f) formula (6)

[0135] In the training process of the second-stage model, the electronic device takes the clean speech signal corresponding to the sample mixed signal as a target signal, performs short-time Fourier transform analysis on the time domain signal of the target signal to obtain a target complex frequency spectrum, and determines whether the first initial model converges according to the second loss function and the target signal. The second loss function is the mean square error of the target complex frequency spectrum of the target signal and the output complex frequency spectrum Y(k, f) of the second stage. Please refer to Figure 7It can be understood that the complex frequency spectrum of the second initial model output is: the second stage output complex frequency spectrum Y(k, f) obtained by the electronic device performing dot multiplication on the aforementioned complex frequency spectrum X2(k, f) and the mask matrix.

[0136] The electronic device continuously adjusts the parameters of the second initial model and determines whether the second initial model converges. If the second initial model does not converge, the electronic device continues to determine the time domain signal of the sample mixed signal and inputs the time domain signal to the first stage model to continue training the second initial model. If the second initial model converges, the electronic device sets the converged second initial model as the second stage model.

[0137] Figure 10 FIG. 1 is a structural diagram of a second stage model in a howling suppression method provided by an embodiment of the present application. Please refer to Figure 10 , compared with Figure 8 , the input complex frequency spectrum X2(k, f) of the second stage model is a mask matrix. The encoding module in the second stage model can refer to the encoding module in the first stage model, and the decoding module in the second stage model can refer to the decoding module in the first stage model, which will not be described here.

[0138] With this scheme, the electronic device constructs a second loss function according to the mean square error of the target complex frequency spectrum of the target signal and the complex frequency spectrum output by the second initial model, and determines whether the second initial model converges by using the second loss function and the target signal, thereby achieving the purpose of quickly and accurately determining whether the second initial model converges.

[0139] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0140] Figure 11 FIG. 1 is a schematic diagram of a howling suppression device provided by an embodiment of the present application. The howling suppression device 1100 includes an acquisition module 111, a processing module 112, and a playing module 113.

[0141] The acquisition module 111 is configured to acquire a first voice signal.

[0142] The processing module 112 is configured to perform howling suppression on the first voice signal by using a neural network model to obtain a second voice signal. The neural network model includes a first stage model and a second stage model connected in series. The first stage model is obtained by training a first initial model with a sample howling signal as input. The second stage model is obtained by training a second initial model with a sample mixed signal as input of the first stage model after the first stage model is trained and frozen.

[0143] The playing module 113 is configured to play the second voice signal.

[0144] In a possible implementation, the processing module 113 is configured to determine a first complex spectrum of the first speech signal; input the first complex spectrum into the first-stage model, so that the first-stage model outputs filter coefficients; filter the first complex spectrum according to the filter coefficients to obtain a second complex spectrum; input the second complex spectrum into the second-stage model, so that the second-stage model outputs a mask matrix; and generate the second speech signal according to the mask matrix, wherein: C N (k, f) represents the filter coefficients, X1(k, f) represents the first complex spectrum, X2(k, f) represents the second complex spectrum, N represents the order, k represents a time frame, and f represents a frequency.

[0145] In a possible implementation, when the processing module 113 generates the second speech signal according to the mask matrix, the processing module 113 is configured to determine a third complex spectrum according to the mask matrix and the second complex spectrum; generate a time-domain signal according to the third complex spectrum; and generate the second speech signal according to the time-domain signal.

[0146] In a possible implementation, before the processing module 113 performs siren suppression on the first speech signal by using the neural network model to obtain the second speech signal, the processing module 113 is further configured to obtain at least one signal pair, each signal pair including a clean speech signal x clean and a real-time feedback signal xfeedback, the real-time feedback signal xfeedback being a signal simulated by the clean speech signal x clean played by a loudspeaker and then collected by a microphone again; simulate a siren generation process for each signal pair to obtain at least one sample siren signal x howling ; construct a first training set according to the at least one sample siren signal; and train the first initial model by using the sample siren signals in the first training set to obtain the first-stage model; and wherein: the real-time feedback signal represents a convolution operation, h(t) represents a room impulse response, G represents a gain, t represents time, △t represents a time delay that changes, and NL represents a nonlinear processing.

[0147] In a possible implementation, when the processing module 113 trains the first initial model by using the sample howling signals in the first training set to obtain the first-stage model, the processing module 113 is configured to input the sample howling signals in the first training set into the first initial model, train the first initial model by using the clean speech signals corresponding to the sample howling signals as target signals, determine whether the first initial model converges according to a first loss function and the target signals, the first loss function being a mean square error of a target complex frequency spectrum of the target signals and a complex frequency spectrum output by the first initial model, and determine the converged first initial model as the first-stage model when the first initial model converges.

[0148] In a possible implementation, before the processing module 113 performs howling suppression on the first speech signal by using the neural network model to obtain a second speech signal, the processing module 113 is further configured to construct a second training set, the second training set including sample mixed signals, the sample mixed signals being signals obtained by mixing sample noise signals and clean speech signals, freeze weights of the first-stage model, and train the second initial model by using the sample mixed signals in the second training set to obtain the second-stage model.

[0149] In a possible implementation, when the processing module 113 trains the second initial model by using the sample mixed signals in the second training set to obtain the second-stage model, the processing module 113 is configured to train the second initial model by using the sample mixed signals in the second training set as input signals of the first-stage model and the clean speech signals corresponding to the sample mixed signals as target signals, determine whether the second initial model converges according to a second loss function and the target signals, the second loss function being a mean square error of a target complex frequency spectrum of the target signals and a complex frequency spectrum output by the second initial model, and determine the converged second initial model as the second-stage model when the second initial model converges.

[0150] In a possible implementation, when the processing module 113 constructs the second training set, the processing module 113 is configured to mix the sample noise signals and the clean speech signals at different signal-to-noise ratios to obtain the sample mixed signals.

[0151] In a feasible implementation manner, the first stage model comprises at least one encoding module, at least one decoding module, and an intermediate layer, the intermediate layer is used to connect the encoding module and the decoding module, the encoding module comprises a convolution layer, a normalization layer, and an activation layer, the decoding module comprises a transposed convolution, a normalization layer, and an activation layer, and an output of the first stage model is a filter coefficient; the second stage model comprises at least one encoding module, at least one decoding module, and an intermediate layer, the intermediate layer is used to connect the encoding module and the decoding module, the encoding module comprises a convolution layer, a normalization layer, and an activation layer, the decoding module comprises a transposed convolution, a normalization layer, and an activation layer, and an output of the second stage model is a mask matrix.

[0152] The howling suppression device provided in the embodiments of the present application can perform the action of the electronic device using the pre-trained neural network model in the above embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0153] Figure 12 A schematic diagram of a training device provided in the embodiments of the present application is shown. The training device 1200 comprises a first training module 121, a freezing module 122, a second training module 123, and a generating module 124.

[0154] The first training module 121 is configured to train a first initial model by using sample howling signals in a first training set to obtain the first stage model, and the first initial model and a second initial model are connected in series.

[0155] The freezing module 122 is configured to freeze the weights of the first stage model.

[0156] The second training module 123 is configured to input sample mixed signals in a second training set into the first stage model to train the second initial model to obtain a second stage model.

[0157] The generating module 124 is configured to generate a neural network model for howling suppression according to the first stage model and the second stage model.

[0158] The training device provided in the embodiments of the present application can perform the action of the electronic device for training the neural network model in the above embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0159] Figure 13 A structural schematic diagram of an electronic device provided in the embodiments of the present application is shown. Please refer to Figure 13 The electronic device 1300 described in the embodiments of the present application comprises at least one processor 131, at least one communication bus 132, a user interface 133, at least one network interface 134, and a memory 135.

[0160] The communication bus 132 is configured to realize the connection communication between the components.

[0161] The user interface 133 can include a display screen, a camera, and optionally a standard wired interface and a wireless interface. The display screen is configured to display an editing interface and a roaming interface.

[0162] The network interface 134 can optionally include a standard wired interface and a wireless interface (e.g., a WI-FI interface).

[0163] The processor 131 can include one or more processing cores. The processor 131 is connected to various parts of the electronic device 1300 through various interfaces and lines, and performs various functions of the electronic device 1300 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 135, and calling data stored in the memory 135. Optionally, the processor 131 can be implemented in at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 131 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU is mainly configured to process an operating system, a user interface, and an application program. The GPU is configured to render and draw a panoramic sphere required to be displayed on the display screen. The modem is configured to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 131, but can be implemented by a separate chip.

[0164] The memory 135 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 135 includes a non-transitory computer-readable storage medium. The memory 135 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 135 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc., and the data storage area can store data involved in the various method embodiments described above, etc. The memory 135 can optionally be at least one storage device located away from the processor 131. As shown in FIG. 13, the memory 135, as a computer storage medium, can include an operating system, a network communication module, a user interface module, and an operating application program of the electronic device. Figure 13

[0165] Embodiments of the present application also provide a computer readable storage medium, which stores computer instructions. The computer instructions, when executed by a processor, implement the howling suppression method or the training method of the neural network model for howling suppression implemented by the electronic device.

[0166] Embodiments of the present application also provide a computer program product, which contains a computer program. The computer program, when executed by a processor, implements the howling suppression method or the training method of the neural network model for howling suppression implemented by the electronic device.

[0167] Those skilled in the art should understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0168] ​The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0169] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0170] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0171] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0172] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM) for storing structural information and / or instruction code. Both volatile and non-volatile memory can be implemented as a flash memory, a magnetic memory, an optical memory, and / or any non-transitory computer readable storage medium.

[0173] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0174] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0175] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method for suppressing howling, characterized in that, Applied to electronic devices, the method includes: Acquire the first speech signal; The first speech signal is suppressed by a neural network model to obtain a second speech signal; wherein, the neural network model includes a first-stage model and a second-stage model in series. The first-stage model is obtained by training a first initial model with sample howling signals as input. The second-stage model is obtained by training a second initial model with sample mixed signals as input after the first-stage model has been trained and frozen. Play the second audio signal.

2. The method according to claim 1, characterized in that, The step of using a neural network model to suppress howling in the first speech signal to obtain the second speech signal includes: Determine the first complex spectrum of the first speech signal; The first complex spectrum is input into the first stage model so that the first stage model outputs filter coefficients. The first complex spectrum is filtered according to the filtering coefficients to obtain the second complex spectrum; The second complex spectrum is input into the second stage model so that the second stage model outputs a mask matrix; The second speech signal is generated according to the mask matrix; wherein: C N (k, f) represents the filter coefficients, X1(k, f) represents the first complex spectrum, X2(k, f) represents the second complex spectrum, N represents the order, k represents the time frame, and f represents the frequency.

3. The method according to claim 2, characterized in that, The step of generating the second speech signal based on the mask matrix includes: The third complex spectrum is determined based on the mask matrix and the second complex spectrum; A time-domain signal is generated based on the third complex spectrum; The second speech signal is generated based on the time-domain signal.

4. The method according to any one of claims 1-3, characterized in that, Before using a neural network model to suppress howling in the first speech signal to obtain the second speech signal, the method further includes: Acquire at least one signal pair, each signal pair containing a clean speech signal x clean and a real-time feedback signal xfeedback, wherein the real-time feedback signal xfeedback is an analog clean speech signal x collected by the microphone. clean The signal that is played by the speaker and then picked up again by the microphone; For each signal pair, simulate the howling generation process to obtain at least one sample howling signal x. howling ; A first training set is constructed based on at least one sample howling signal; The first initial model is trained using sample howling signals from the first training set to obtain the first stage model; wherein: The real-time feedback signal denoted by convolution operation, h(t) represents the room impulse response, G represents the gain, t represents time, Δt represents the variable time delay, and NL represents nonlinear processing.

5. The method according to claim 4, characterized in that, The step of training the first initial model using sample howling signals from the first training set to obtain the first stage model includes: The sample howling signals from the first training set are input into the first initial model, and the clean speech signals corresponding to the sample howling signals are used as the target signals to train the first initial model. Whether the first initial model converges is determined based on the first loss function and the target signal. The first loss function is the mean square error between the target complex spectrum of the target signal and the output complex spectrum of the first stage. When the first initial model converges, the converged first initial model is determined to be the first stage model.

6. The method according to any one of claims 1-3, characterized in that, Before using a neural network model to suppress howling in the first speech signal to obtain the second speech signal, the method further includes: Construct a second training set, which contains a sample mixed signal, which is a signal obtained by mixing sample noise signal and clean speech signal; Freeze the weights of the first-stage model; The second initial model is trained using the mixed signal of samples from the second training set to obtain the second stage model.

7. The method according to claim 6, characterized in that, The step of training the second initial model using the mixed signal of samples from the second training set to obtain the second stage model includes: The sample mixed signal in the second training set is used as the input signal of the first stage model, and the clean speech signal corresponding to the sample mixed signal is used as the target signal to train the second initial model. The convergence of the second initial model is determined based on the target signal according to the second loss function, wherein the second loss function is the mean square error between the target complex spectrum of the target signal and the output complex spectrum of the second stage. When the second initial model converges, the converged second initial model is determined to be the second stage model.

8. The method according to claim 6, characterized in that, The construction of the second training set includes: The sample noise signal and the clean speech signal are mixed with different signal-to-noise ratios to obtain the sample mixed signal.

9. The method according to any one of claims 1-3, characterized in that, The first-stage model includes at least one encoding module, at least one decoding module, and an intermediate layer. The intermediate layer is used to connect the encoding module and the decoding module. The encoding module includes a convolutional layer, a normalization layer, and an activation layer. The decoding module includes a transposed convolutional layer, a normalization layer, and an activation layer. The output of the first-stage model is the filter coefficients. The second-stage model includes at least one encoding module, at least one decoding module, and an intermediate layer. The intermediate layer connects the encoding module and the decoding module. The encoding module includes a convolutional layer, a normalization layer, and an activation layer. The decoding module includes a transposed convolutional layer, a normalization layer, and an activation layer. The output of the second-stage model is a mask matrix.

10. A training method for a neural network model for howling suppression, characterized in that, include: The first initial model is trained using the sample howling signals in the first training set to obtain the first stage model, and the first initial model and the second initial model are concatenated. Freeze the weights of the first-stage model; The mixed signal of samples from the second training set is input into the first-stage model to train the second initial model to obtain the second-stage model; A neural network model for howling suppression is generated based on the first-stage model and the second-stage model.

11. A whistling suppression device, characterized in that, include: Acquisition module, used for the first voice signal; The processing module is used to suppress howling on the first speech signal using a neural network model to obtain a second speech signal; wherein, the neural network model includes a first-stage model and a second-stage model in series, the first-stage model is obtained by training a first initial model with sample howling signals as input, and the second-stage model is obtained by training a second initial model with sample mixed signals as input to the first-stage model after the first-stage model has been trained and frozen; The playback module is used to play the second audio signal.

12. A training device, characterized in that, include: The first training module is used to train a first initial model using sample howling signals from a first training set to obtain the first stage model, wherein the first initial model and the second initial model are connected in series. The freeze module is used to freeze the weights of the model in the first stage. The second training module is used to input the mixed signal of samples from the second training set into the first stage model to train the second initial model to obtain the second stage model; A generation module is used to generate a neural network model for howling suppression based on the first-stage model and the second-stage model.

13. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 10.

Images