A method, device, equipment and storage medium for reducing call noise

Abstract

The present disclosure provides a method, device, equipment and storage medium for reducing the noise of call speech, which relates to the field of computer technology, and in particular to the field of speech technology. The specific implementation scheme is: obtaining a call speech signal and a reference signal, the call speech signal includes an external dual-microphone front-end signal and a built-in microphone signal; performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal; performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal; using a deep neural network DNN noise reduction model, performing noise reduction on the first pre-processed signal to obtain a first noise reduction signal; performing noise reduction on the second pre-processed signal according to voice activity detection information to obtain a second noise reduction signal, and the voice activity detection information comes from the output of the DNN noise reduction model; performing spectrum fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of voice technology, a method, a device, a equipment and a storage medium for reducing call voice noise. Background Art

[0002] Voice noise reduction is a technical problem that headphone products have been committed to solving and optimizing. Existing call noise reduction headphones mainly use the following two voice noise reduction processing solutions:

[0003] 1. The method of combining external array signal processing with feedback microphone for speech noise reduction uses an external sensor array to enhance the speaker's voice and integrates the feedback microphone, using the high signal-to-noise ratio of the feedback microphone to improve the call quality under ambient noise and wind noise.

[0004] 2. Perform voice noise reduction based on auxiliary sensor methods, such as using vibration sensors or acceleration sensors (such as bone conduction sensors) to extract the wearer's voice. Summary of the invention

[0005] The present disclosure provides a call voice noise reduction method, device, equipment and storage medium for improving the call voice noise reduction effect.

[0006] According to one aspect of the present disclosure, a method for reducing call voice noise is provided, comprising:

[0007] Acquire call voice signals and reference signals, where the call voice signals include external dual-microphone front-end signals and built-in microphone signals;

[0008] Performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal;

[0009] Performing second pre-processing on the built-in microphone signal to obtain a second pre-processed signal;

[0010] Using a deep neural network (DNN) denoising model, the first pre-processed signal is denoised to obtain a first denoised signal;

[0011] According to the voice activity detection information, performing denoising on the second pre-processed signal to obtain a second denoised signal, wherein the voice activity detection information comes from an output of the DNN denoising model;

[0012] The first noise reduction signal and the second noise reduction signal are spectrally fused to obtain a noise reduction output speech signal.

[0013] According to another aspect of the present disclosure, a call voice noise reduction device is provided, comprising:

[0014] A signal acquisition module is used to acquire a call voice signal and a reference signal. The call voice signal includes an external dual-microphone front-end signal and a built-in microphone signal.

[0015] A first pre-processing module, used for performing a first pre-processing on an external dual-microphone front-end signal to obtain a first pre-processed signal;

[0016] A second pre-processing module, used for performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal;

[0017] A first denoising module, configured to perform denoising on the first pre-processed signal using a deep neural network (DNN) denoising model to obtain a first denoised signal;

[0018] A second noise reduction module, configured to perform noise reduction processing on the second pre-processed signal according to voice activity detection information to obtain a second noise reduction signal, wherein the voice activity detection information comes from an output of the DNN noise reduction model;

[0019] The fusion module is used to perform spectrum fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal.

[0020] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the call voice noise reduction method of the present disclosure.

[0021] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the call voice noise reduction method of the present disclosure.

[0022] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the call voice noise reduction method of the present disclosure is implemented.

[0023] According to another aspect of the present disclosure, a headset is provided, comprising: at least one processor; a memory and a microphone, wherein the processor, the memory and the microphone are communicated and interconnected; the memory stores instructions that can be executed by at least one processor, and the microphone is used to collect call voice signals; the instructions are executed by at least one processor so that the at least one processor can execute the call voice noise reduction method of the present disclosure.

[0024] In the present disclosure, a DNN noise reduction model and feedback microphone fusion are used to perform spectral fusion on the noise reduction signals obtained after the external dual-microphone front-end signals and the built-in microphone signals are subjected to noise reduction processing respectively, so as to obtain the noise reduction output speech, thereby providing a call voice noise reduction method with better noise reduction effect.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0027] Figure 1 The process diagram of the call voice noise reduction method according to the first embodiment of the present disclosure is as follows Figure 1 ;

[0028] Figure 2 The process diagram of the call voice noise reduction method according to the first embodiment of the present disclosure is as follows Figure 2 ;

[0029] Figure 3 It is a schematic diagram of the training process of the disclosed DNN denoising model;

[0030] Figure 4 is a flowchart of a method for reducing call voice noise according to the second embodiment of the present disclosure;

[0031] Figure 5 is a flowchart of a method for reducing call voice noise according to the third embodiment of the present disclosure;

[0032] Figure 6 is a flowchart of a method for reducing call voice noise according to a fourth embodiment of the present disclosure;

[0033] Figure 7 is a structural schematic diagram of a call voice noise reduction device according to a fifth embodiment of the present disclosure;

[0034] Figure 8 is a block diagram of an electronic device for implementing the call voice noise reduction method of the embodiment of the present disclosure;

[0035] Fig. 9 is a block diagram of an earphone used to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] Figure 1 The process diagram of the call voice noise reduction method according to the first embodiment of the present disclosure is as follows Figure 1 ,like Figure 1 As shown, the method adopts a DNN noise reduction model and feedback microphone fusion to perform spectral fusion on the noise reduction signals obtained after the external dual-microphone front-end signals and the built-in microphone signals are denoised respectively, and the noise reduction output speech is obtained, thereby providing a call voice noise reduction method with better noise reduction effect.

[0038] Figure 2 The process diagram of the call voice noise reduction method according to the first embodiment of the present disclosure is as follows Figure 2 ,like Figure 2 As shown, combined Figure 1 , the method mainly includes:

[0039] Step S101, obtaining a call voice signal and a reference signal, where the call voice signal includes an external dual-microphone front-end signal and a built-in microphone signal.

[0040] The True Wireless Stereo (TWS) headset in this embodiment adopts a three-microphone fusion topology architecture, that is, the TWS headset is equipped with three microphones, namely ffmic, talkmic and fbmic. Among them, ffmic and talkmic are set on the outside of the headset. ffmic is also called a single feedforward microphone, which is used to collect noise in the environment and detect and eliminate noise before the main noise signal reaches the human ear; talkmic is a microphone used by ordinary users when talking, which is used to collect human voices; fbmic is also called a single feedback microphone, which is set on the inside of the headset and is used to perform secondary elimination of noise that is not eliminated by ffmic.

[0041] During a call, the TWS headset receives four signals, namely, ffmic, talkmic, fbmic and ref. The external dual-microphone front-end signal includes ffmic signal and talkmic signal. The ffmic signal comes from the ambient noise collected by ffmic, and the talkmic signal comes from the human voice collected by talkmic. The built-in microphone signal is the fbmic signal, which comes from the noise received by fbmic after ffmic processing. The reference signal is the ref signal, which can come from the reference microphone or non-acoustic sensor set in the headset. This embodiment does not limit the source of the reference signal.

[0042] Step S102: performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal.

[0043] In this embodiment, before performing deep neural network (DNN, Deep NeuralNetworks) noise reduction on the external dual-microphone front-end signal, it is necessary to pre-process the external dual-microphone front-end signal received by the headset. Since the TWS headset has a certain degree of isolation, the echo residue of the external dual-microphone front-end signal is relatively small compared to other voice devices such as speakers. Based on the above characteristics, the pre-processing of the external dual-microphone front-end signal mainly includes array processing, wind noise estimation, echo cancellation and other processing steps, which are used to preliminarily purify the ambient noise collected by ffmic and the human voice collected by talkmic.

[0044] Step S103, performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal.

[0045] In this embodiment, before the built-in microphone signal is denoised, it is necessary to perform preliminary processing on the built-in microphone signal received by the headphone device. Since the built-in microphone signal comes from the signal after ffmic processing, there is no need to perform array processing and wind noise estimation on it. However, since the built-in microphone signal has problems such as large echo and poor internal noise listening, it is necessary to perform echo cancellation processing on it to perform preliminary purification of the built-in microphone signal collected by the headphone device.

[0046] Step S104: Use a deep neural network DNN denoising model to perform denoising on the first pre-processed signal to obtain a first denoised signal.

[0047] After the external dual-microphone front-end signal is processed to obtain the first pre-processed signal, DNN noise reduction can be performed on the first pre-processed signal. The first pre-processed signal is input into the trained DNN noise reduction model, and the first noise reduction signal after noise reduction processing is output.

[0048] Specifically, the DNN denoising model can be an LSTM model, a bidirectional LSTM model, a gated recurrent unit (GRU) model, etc. In this embodiment, a double-layer GRU model is adopted, mainly considering that the GRU model has memory ability. When performing speech denoising frame by frame, the GRU model can observe the spectral information for a period of time, so as to better distinguish speech from noise.

[0049] Step S105: performing noise reduction processing on the second pre-processed signal according to the voice activity detection information to obtain a second noise reduction signal, wherein the voice activity detection information comes from the output of the DNN noise reduction model.

[0050] After processing the built-in microphone signal to obtain the second pre-processed signal, the second pre-processed signal can be subjected to noise reduction processing. Here, the noise reduction is not performed directly using the DNN noise reduction model, but the second pre-processed signal is subjected to noise reduction processing based on the voice activity detection (VAD, Voice Activity Detection) information output by the DNN noise reduction model.

[0051] Specifically, the speech signal and the non-speech signal in the second pre-processed signal can be detected according to the VAD information, and then different noise reduction processes are performed on the speech signal segment and the non-speech signal segment, so as to finally achieve a better speech noise reduction effect.

[0052] Step S106: Spectrum fusion is performed on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal.

[0053] After the above steps S104 and S105, the first noise reduction signal and the second noise reduction signal are obtained, and then the two signals need to be fused to obtain the noise reduction output voice signal, and then the noise reduction output voice signal is output to the human ear.

[0054] Specifically, the first noise reduction signal and the second noise reduction signal may be frequency fused according to a fusion frequency band and a fusion factor. The fusion frequency band and the fusion factor may be determined according to specific signals, and are not limited in this embodiment.

[0055] In this embodiment, a three-microphone fusion noise reduction strategy is adopted. By performing pre-processing and noise reduction processing on the external dual-microphone front-end signal and the built-in microphone signal respectively, a first noise reduction signal and a second noise reduction signal are obtained, and the above two signals are fused to obtain a noise-reduced output voice signal, thereby achieving the effect of improving the call noise reduction efficiency and greatly improving the user experience.

[0056] Based on the above embodiment, in order to obtain a DNN noise reduction model with better noise reduction effect, it is necessary to train the DNN noise reduction model through a single-channel DNN training method before receiving the call voice. Figure 3 The description is as follows:

[0057] Figure 3 Schematic diagram of the training process of the disclosed DNN denoising model. Figure 3 As shown, specifically including:

[0058] Step S201, obtaining clean speech data and noisy speech data.

[0059] In this embodiment, before training the DNN noise reduction model, it is necessary to collect original corpus, including clean speech data of different age groups, genders and languages, and different types of noisy speech data such as subway, human voice, car, wind noise, cafe, gym, keyboard sound, etc.

[0060] Step S202, performing data enhancement on the clean speech data and the noisy speech data to obtain simulated call speech data.

[0061] Specifically, after obtaining clean speech data and noisy speech data, in order to simulate the actual collected signal as much as possible, reverberation is first added to the clean speech data, and then it is superimposed with the noisy speech data according to different signal-to-noise ratios. Finally, the modulation of the signal by the microphone receiving frequency response is simulated through the EQ equalizer.

[0062] Specifically, the EQ equalizer simulates the modulation of the microphone receiving frequency response to the signal in the anechoic tank through white noise. An artificial head is placed in the anechoic room, and the headphones are worn on the artificial ears. White noise is played through the artificial mouth, and the audio signals received by the headphones ffmic, talkmic, and fbmic are collected, so as to calculate the transfer function from the artificial mouth to ffmic, talkmic, and fbmic. When performing data enhancement, the transfer function is convolved with the clean voice data to obtain training data with similar acoustic characteristics to the headphones, that is, simulated call voice data.

[0063] Step S203: extracting acoustic features of the simulated call voice data.

[0064] After collecting the simulated call voice data, it needs to be converted into acoustic features that can be recognized by DNN, including signal processing algorithms such as pre-emphasis, mean subtraction, and frame windowing. Acoustic features represent the characteristics of speech. The acoustic features of noise and speech are quite different. Extracting appropriate acoustic features can reduce the difficulty of model learning and improve the noise reduction effect.

[0065] Specifically, this embodiment adopts 64-dimensional Fbank features, and obtains 64-dimensional Fbank feature acoustic features of the simulated call voice data after framing and windowing, discrete Fourier transform (DFT, Discrete Fourier Transform), filtering and obtaining logarithmic power of the simulated call voice data.

[0066] Step S204, calibrating the training target of single-channel DNN training.

[0067] In this embodiment, before training the DNN denoising model, the DNN denoising model needs to be calibrated with a label, and the label calibration represents the target of model learning. The label in this embodiment is an ideal ratio mask (IRM) calculated based on the signal-to-noise ratio. Specifically, in this embodiment, IRM_MASK is used to represent the amplitude ratio of clean speech data and noisy speech data, and multiplying it with the noisy speech data can inversely solve the clean speech data.

[0068] Step S205: According to the acoustic features and training objectives, a DNN denoising model is obtained by DNN training.

[0069] In this embodiment, after the acoustic features are extracted from the simulated call voice data, DNN forward propagation is performed on the simulated call voice data according to the acoustic features to obtain the estimated value of IRM, and then the error between the estimated value of IRM and the training target Label is calculated, and the simulated call voice data is back-propagated according to the error to update the weights of each node of the DNN denoising model. Finally, when the error is less than the preset error threshold, the update of the DNN denoising model is stopped to complete the training of the DNN denoising model. Specifically, this embodiment adopts a two-layer GRU model.

[0070] In this embodiment, a deep neural network DNN is used to distinguish the acoustic characteristics of speech and noise to train a DNN noise reduction model. The DNN noise reduction model can separate clean speech data from simulated call voice data, thereby achieving a better effect of distinguishing clean speech data from noisy speech data, and can further improve the call noise reduction efficiency of the TWS headset.

[0071] Figure 4 FIG. 1 is a flow chart of a method for reducing call voice noise according to a second embodiment of the present disclosure. Figure 4 As shown, combined Figure 1 , step S102 specifically includes:

[0072] Step S301: Perform linear array processing according to the external dual-microphone front-end signal to obtain a first processed signal.

[0073] Specifically, the linear array processing of the external dual-microphone front-end signal can adopt beamforming technology, zero-point forming technology, spatial spectrum estimation, signal source estimation and other methods. In this embodiment, the beamforming technology is adopted, mainly to enhance the external dual-microphone front-end signal, and form the processed ffmic signal and talkmic signal into one signal. Linear array processing can enhance the required useful signal, suppress useless interference and noise, and extract useful signal features and information contained in the signal. Linear array processing does not destroy the correlation between the external dual-microphone front-end signal and the reference signal.

[0074] Step S302: perform wind noise estimation and elimination processing according to the external dual-microphone front-end signal and the first processed signal to obtain a second processed signal.

[0075] Specifically, to estimate wind noise, it is first necessary to calculate the signal parameters of the external dual-microphone front-end signal and the first processed signal, including correlation, spectral centroid and energy, and then make a wind noise / non-wind noise judgment based on the correlation and spectral centroid. If it is determined to be wind noise, the wind noise is divided into small wind noise, medium wind noise and large wind noise according to the energy. Finally, the external dual-microphone front-end signal and the first processed signal are subjected to wind noise elimination processing according to the wind noise type to obtain a second processed signal.

[0076] Step S303: Perform echo cancellation processing according to the second processed signal and the reference signal to obtain a third processed signal.

[0077] In this embodiment, the second processed signal after echo cancellation is processed together with the reference signal for echo cancellation. The methods for performing the echo cancellation process mainly include spectral subtraction, least mean square (LMS) algorithm, block LMS algorithm, normalized least mean square (NLMS) algorithm, etc. This embodiment does not limit the echo cancellation method.

[0078] Step S304: performing nonlinear array processing on the third processed signal to obtain a first pre-processed signal.

[0079] Specifically, nonlinear array processing is performed on the third processed signal after the echo cancellation processing, and signal enhancement processing is further performed on the external dual-microphone noise reduction signal.

[0080] In this embodiment, array-enhanced linear processing and nonlinear processing are divided into two parts. First, array-enhanced linear processing is performed on the external dual-microphone front-end signal, so as not to destroy the correlation between the external dual-microphone front-end signal and the reference signal. Then, wind noise estimation and elimination processing and echo cancellation processing are performed on the signal after beamforming. Finally, nonlinear array signal processing is performed on it. The above processing steps can convert the echo cancellation processing from multiple times to one time, which can save the time of echo cancellation calculation, thereby improving the call life of the TWS headset.

[0081] Figure 5 FIG. 1 is a flow chart of a method for reducing call voice noise according to a third embodiment of the present disclosure. Figure 5 As shown, combined Figure 1 , step S103 specifically includes:

[0082] Step S401: perform initial echo cancellation processing on the built-in microphone signal using an echo cancellation filter to obtain a first echo cancellation signal.

[0083] Specifically, when performing the second preliminary processing on the built-in microphone signal, a large amount of echo data is firstly used to generate an echo cancellation filter offline, and the built-in microphone signal is processed with the echo cancellation filter in a stage to reduce the echo component in the built-in microphone signal.

[0084] Step S402: Based on the correlation between the first echo-cancelled signal and the reference signal, perform echo cancellation processing on the first echo-cancelled signal again to obtain the second pre-processed signal.

[0085] In this embodiment, a two-stage echo cancellation algorithm is used to perform echo cancellation processing on the built-in microphone signal. First, an echo cancellation filter is used to perform echo cancellation on the built-in microphone signal, and then the built-in microphone signal is echo cancelled again based on the correlation with the reference signal. Through the above echo cancellation processing, it can be ensured that the built-in microphone signal has no residual echo cancellation in single-talk and the near-segment voice is not damaged in dual-talk, thereby ensuring the call quality of the user.

[0086] Figure 6 FIG. 4 is a flow chart of a method for reducing call voice noise according to a fourth embodiment of the present disclosure. Figure 6 As shown, combined Figure 1 , step S105 specifically includes:

[0087] Step S501: determining whether the second pre-processed signal is in a non-speech area according to voice activity detection information.

[0088] In this embodiment, the value of the voice activity detection (VAD, Voice Activity Detection) information output by the DNN noise reduction model is used to determine whether the second pre-processed signal is in the non-speech area or the speech area. When the VAD value is 0, the second pre-processed signal is in the non-speech area. When the VAD value is 1, the second pre-processed signal is in the speech area.

[0089] Step S502, estimating the speech power spectrum when the built-in microphone processed signal is in the speech area, and the noise power spectrum when the built-in microphone processed signal is in the non-speech area.

[0090] Step S503: Calculate the signal-to-noise ratio of the second pre-processed signal according to the noise power spectrum and the speech power spectrum.

[0091] Specifically, the formula for calculating the signal-to-noise ratio of the signal after the second pre-processing is: Among them, Ps is the speech power spectrum and Pn is the noise power spectrum.

[0092] Step S504: performing noise reduction processing on the second pre-processed signal using the signal-to-noise ratio to obtain a second noise-reduced signal.

[0093] Specifically, after the signal-to-noise ratio of the second pre-processed signal is calculated, the signal-to-noise ratio is used to estimate the noise reduction gain to complete the noise reduction.

[0094] In this embodiment, the value of the VAD information output by the DNN noise reduction model is first used to determine whether the second pre-processed signal is in the non-speech area or the speech area, and then the speech power spectrum in the speech area and the noise power spectrum in the non-speech area are calculated to calculate the signal-to-noise ratio. Finally, the signal-to-noise ratio is used to complete the noise reduction of the second pre-processed signal. Through the above embodiments, the noise reduction efficiency of call voice can be further improved.

[0095] Specifically, after step S104 in the first embodiment of the present disclosure, the following steps may also be performed:

[0096] Step S1040, performing a signal-to-noise ratio estimation based on the first pre-processed signal and the first noise reduction signal to obtain the signal-to-noise ratio of the first noise reduction signal, and determining the noise type of the first noise reduction signal based on the signal-to-noise ratio, wherein the noise types include transient noise, steady-state noise, weak noise and strong noise.

[0097] Specifically, step S106 of the first embodiment of the present disclosure specifically includes:

[0098] According to the wind noise type and the noise type, the first noise reduction signal and the second noise reduction signal are spectrally fused to obtain a noise reduction output speech signal.

[0099] Specifically, transient noise is directly processed by beamforming, steady-state noise is subjected to noise intensity detection, weak noise is directly processed by beamforming, and strong noise and the signal after fbmic echo cancellation are channel fused; wind noise is subjected to wind noise intensity judgment, weak wind noise signals below 2k are channel fused, medium wind noise signals below 4k are signal fused, and strong wind noise is replaced with the signal after fbmic echo cancellation.

[0100] By performing different fusion processing on different wind noise types and noise types, a better fusion effect can be achieved, thereby obtaining a noise-reducing output voice signal with less noise, thereby improving the call voice noise reduction efficiency of the TWS headset.

[0101] Figure 7 Schematic diagram of the structure of the call voice noise reduction device of the fifth embodiment of the present disclosure. Figure 7 As shown, the device mainly includes:

[0102] The signal acquisition module 10 is used to acquire a call voice signal and a reference signal, wherein the call voice signal includes an external dual-microphone front-end signal and a built-in microphone signal; the first pre-processing module 20 is used to perform a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal; the second pre-processing module 30 is used to perform a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal; the first noise reduction module 40 is used to perform noise reduction on the first pre-processed signal using a deep neural network DNN noise reduction model to obtain a first noise reduction signal; the second noise reduction module 50 is used to perform noise reduction on the second pre-processed signal according to voice activity detection information to obtain a second noise reduction signal, wherein the voice activity detection information comes from the output of the DNN noise reduction model; the fusion module 60 is used to perform spectral fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output voice signal.

[0103] In one possible implementation manner, the device further includes: a DNN denoising model training module, for training the DNN denoising model; further, the DNN denoising model training module further includes:

[0104] The speech data acquisition submodule is used to acquire clean speech data and noisy speech data; the data enhancement submodule is used to perform data enhancement on clean speech data and noisy speech data to obtain simulated call speech data; the acoustic feature extraction submodule is used to extract the acoustic features of simulated call speech data; the calibration submodule is used to calibrate the training target of single-channel DNN training; the training submodule is used to obtain the DNN denoising model through DNN training based on the acoustic features and training targets.

[0105] Furthermore, the training submodule includes: an estimated value calculation unit, which is used to perform DNN forward propagation on the simulated call voice data according to the acoustic features to obtain an estimated value of the ideal ratio masked IRM; an error calculation unit, which is used to calculate the error between the estimated value of IRM and the training target; a weight updating unit, which is used to perform back propagation on the simulated call voice data according to the error to update the weights of each node of the DNN noise reduction model; and a judgment unit, which is used to stop updating the DNN noise reduction model when the error is less than a preset error threshold.

[0106] In one possible implementation manner, the first pre-processing module 20 includes: a linear array processing submodule, which is used to perform linear array processing according to an external dual-microphone front-end signal to obtain a first processed signal; a wind noise estimation and elimination submodule, which is used to perform wind noise estimation and elimination processing according to the external dual-microphone front-end signal and the first processed signal to obtain a second processed signal; an echo cancellation submodule, which is used to perform echo cancellation processing according to the second processed signal and a reference signal to obtain a third processed signal; and a nonlinear array processing submodule, which is used to perform nonlinear array processing on the third processed signal to obtain a first pre-processed signal.

[0107] Furthermore, the wind noise estimation and elimination module includes: a parameter calculation submodule, which is used to calculate the signal parameters of the external dual-microphone front-end signal and the first processed signal, and the signal parameters include correlation, spectral centroid and energy; a wind noise type judgment submodule, which is used to determine the wind noise type according to the signal parameters, and the wind noise types include non-wind noise, small wind noise, medium wind noise and large wind noise; a wind noise elimination submodule, which is used to perform wind noise elimination processing on the external dual-microphone front-end signal and the first processed signal according to the wind noise type to obtain a second processed signal.

[0108] In one embodiment, the first noise reduction module 40 further includes: a first signal-to-noise ratio calculation submodule, used to estimate the signal-to-noise ratio based on the first pre-processed signal and the first noise reduction signal to obtain the signal-to-noise ratio of the first noise reduction signal; and a noise type determination submodule, used to determine the noise type of the first noise reduction signal based on the signal-to-noise ratio.

[0109] In one possible implementation, the fusion module 60 is further configured to perform spectrum fusion on the first noise reduction signal and the second noise reduction signal according to the wind noise type and the noise type to obtain a noise reduction output speech signal.

[0110] In one possible implementation manner, the second pre-processing module 30 also includes: a primary echo cancellation submodule, which is used to perform primary echo cancellation processing on the built-in microphone signal using an echo cancellation filter to obtain a first echo cancellation signal; and a secondary echo cancellation submodule, which is used to perform secondary echo cancellation processing on the first echo cancellation signal based on the correlation between the first echo cancellation signal and a reference signal to obtain a second pre-processed signal.

[0111] In one possible implementation mode, the second noise reduction module 50 also includes: a speech area judgment submodule, which is used to judge whether the second pre-processed signal is in a non-speech area based on speech activity detection information; a power spectrum estimation submodule, which is used to estimate the speech power spectrum when the built-in microphone processed signal is in the speech area, and the noise power spectrum when the built-in microphone processed signal is in the non-speech area; a second signal-to-noise ratio calculation submodule, which is used to calculate the signal-to-noise ratio of the second pre-processed signal based on the noise power spectrum and the speech power spectrum; and a noise reduction processing submodule, which is used to perform noise reduction processing on the second pre-processed signal using the signal-to-noise ratio to obtain a second noise-reduced signal.

[0112] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, a computer program product and a headset.

[0114] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0115] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0116] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the call voice noise reduction method. For example, in some embodiments, the call voice noise reduction method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the call voice noise reduction method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the call voice noise reduction method in any other appropriate manner (e.g., by means of firmware).

[0118] Fig. 9 A block diagram of a headset provided by an embodiment of the present disclosure is shown, Fig. 9 As shown, the headset includes: at least one processor 900; and a memory 901 and a microphone 902, wherein the processor 900, the memory 901 and the microphone 902 are in communication and interconnected; the memory 901 stores instructions that can be executed by at least one processor 900, and the microphone 902 is used to collect call voice signals; the instructions are executed by at least one processor 900 so that at least one processor 900 can execute the call voice noise reduction method.

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for reducing noise in a call, comprising: Acquire a call voice signal and a reference signal, wherein the call voice signal includes an external dual-microphone front-end signal and a built-in microphone signal; Performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal; Performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal; Using a deep neural network (DNN) denoising model, the first pre-processed signal is denoised to obtain a first denoised signal; According to the voice activity detection information, performing noise reduction processing on the second pre-processed signal to obtain a second noise reduction signal, wherein the voice activity detection information comes from the output of the DNN noise reduction model; Performing spectrum fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal; The step of performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal includes: Performing linear array processing on the external dual-microphone front-end signal to obtain a first processed signal; performing wind noise estimation and elimination processing on the external dual-microphone front-end signal and the first processed signal to obtain a second processed signal; performing echo elimination processing on the second processed signal and the reference signal to obtain a third processed signal; performing nonlinear array processing on the third processed signal to obtain the first pre-processed signal; The step of performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal includes: The built-in microphone signal is subjected to a primary echo cancellation process using an echo cancellation filter to obtain a first echo cancellation signal; and the first echo cancellation signal is subjected to a second echo cancellation process based on a correlation between the first echo cancellation signal and the reference signal to obtain the second pre-processed signal.

2. The method according to claim 1, wherein: The DNN denoising model is obtained by the following single-channel DNN training method; Obtain clean speech data and noisy speech data; Performing data enhancement on the clean speech data and the noisy speech data to obtain simulated call speech data; Extracting acoustic features of the simulated call voice data; Calibrate the training target for single-channel DNN training; The DNN denoising model is obtained through DNN training according to the acoustic features and the training objectives.

3. The method according to claim 2, wherein: The DNN denoising model is obtained by DNN training according to the acoustic features and the training objectives, including: According to the acoustic features, DNN forward propagation is performed on the simulated call speech data to obtain an estimated value of the ideal ratio masking IRM; Calculating the error between the estimated value of the IRM and the training target; Back-propagating the simulated call voice data according to the error, and updating the weights of each node of the DNN noise reduction model; When the error is less than a preset error threshold, the updating of the DNN denoising model is stopped.

4. The method according to claim 1, wherein: The step of performing wind noise estimation and elimination processing according to the external dual-microphone front-end signal and the first processed signal to obtain a second processed signal includes: Calculating signal parameters of the external dual-microphone front-end signal and the first processed signal, the signal parameters including correlation, spectral centroid and energy; determining a wind noise type according to the signal parameter, the wind noise type including no wind noise, small wind noise, medium wind noise and large wind noise; The external dual-microphone front-end signal and the first processed signal are subjected to wind noise elimination processing according to the wind noise type to obtain the second processed signal.

5. The method according to claim 4, wherein: After performing denoising processing on the first pre-processed signal by using the DNN denoising model to obtain a first denoised signal, the method further includes: Estimating a signal-to-noise ratio according to the first pre-processed signal and the first noise reduction signal to obtain a signal-to-noise ratio of the first noise reduction signal; A noise type of the first noise reduction signal is determined according to the signal-to-noise ratio.

6. The method according to claim 5, wherein: The performing spectrum fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal includes: According to the wind noise type and the noise type, spectrum fusion is performed on the first noise reduction signal and the second noise reduction signal to obtain the noise reduction output voice signal.

7. The method according to claim 1, wherein: The step of performing noise reduction processing on the second pre-processed signal according to the voice activity detection information to obtain a second noise reduction signal includes: determining, according to the voice activity detection information, whether the second pre-processed signal is in a non-speech area; estimating a speech power spectrum when the built-in microphone processing signal is in a speech area, and a noise power spectrum when the built-in microphone processing signal is in a non-speech area; Calculating a signal-to-noise ratio of the second pre-processed signal according to the noise power spectrum and the speech power spectrum; The second pre-processed signal is subjected to noise reduction processing by using the signal-to-noise ratio to obtain the second noise-reduced signal.

8. A call voice noise reduction device, comprising: A signal acquisition module, used to acquire a call voice signal and a reference signal, wherein the call voice signal includes an external dual-microphone front-end signal and a built-in microphone signal; A first pre-processing module, used for performing a first pre-processing on the external dual-microphone front-end signal to obtain a first pre-processed signal; A second pre-processing module, used for performing a second pre-processing on the built-in microphone signal to obtain a second pre-processed signal; A first denoising module, configured to perform denoising on the first pre-processed signal using a deep neural network (DNN) denoising model to obtain a first denoised signal; A second noise reduction module, configured to perform noise reduction processing on the second pre-processed signal according to voice activity detection information to obtain a second noise reduction signal, wherein the voice activity detection information comes from the output of the DNN noise reduction model; A fusion module, configured to perform spectrum fusion on the first noise reduction signal and the second noise reduction signal to obtain a noise reduction output speech signal; Wherein, the first pre-processing module includes: A linear array processing submodule, configured to perform linear array processing according to the external dual-microphone front-end signal to obtain a first processed signal; A wind noise estimation and elimination submodule, configured to perform wind noise estimation and elimination processing according to the external dual-microphone front-end signal and the first processed signal to obtain a second processed signal; an echo cancellation submodule, configured to perform echo cancellation processing according to the second processed signal and the reference signal to obtain a third processed signal; A nonlinear array processing submodule, configured to perform nonlinear array processing on the third processed signal to obtain the first pre-processed signal; Wherein, the second pre-processing module includes: A primary echo cancellation submodule, configured to perform primary echo cancellation processing on the built-in microphone signal using an echo cancellation filter to obtain a first echo cancellation signal; The re-echo cancellation submodule is used to perform re-echo cancellation processing on the first echo cancellation signal based on the correlation between the first echo cancellation signal and the reference signal to obtain the second pre-processed signal.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

12. A headset, comprising: at least one processor; and a memory and a microphone, wherein the processor, the memory and the microphone are communicatively interconnected; The memory stores instructions executable by the at least one processor, and the microphone is used to collect call voice signals; The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

Images