Call noise reduction method and related device

Abstract

The invention discloses a call noise reduction method and a related device, and relates to the technical field of audio noise reduction, and the call noise reduction method comprises the steps: obtaining a first audio signal collected by a first microphone array disposed at a first side of an earphone, and a second audio signal collected by a second microphone array disposed at a second side of the earphone; performing near-field beam forming processing on the first audio signal by using a first near-field beam former arranged on the first side of the earphone to obtain a first beam forming signal, and performing near-field beam forming processing on the second audio signal by using a second near-field beam former arranged on the second side of the earphone to obtain a second beam forming signal; determining a target beam forming signal according to the first beam forming signal and the second beam forming signal; and performing noise reduction on the target beam forming signal by using an audio noise reduction model obtained by pre-training to obtain a noise-reduced audio signal. The call noise reduction method disclosed by the invention has a relatively good noise reduction effect and relatively good user experience.

Description

Technical Field

[0001] The present application relates to the technical field of audio noise reduction, and in particular to a call noise reduction method and related devices. Background Art

[0002] Harsh noise environments can reduce the quality of voice communications. Call noise reduction is an important function on headphones and plays an important role in improving voice quality.

[0003] There are many ways to achieve call noise reduction in headphones. One of them is to install a low-sensitivity stem on the left side of the headset, and the user holds the stem close to the stem for calls. However, this type of stem-type headset is very inconvenient to use. In order to solve the problem of the inconvenience of using stem-type headsets, another type of headset has emerged, namely a single-microphone headset. This headset uses a single microphone set on the earphone shell to pick up sound and reduce noise. However, when the user uses the headset, the mouth is 15 cm to 25 cm away from the microphone, which results in a greatly reduced signal-to-noise ratio of sound pickup compared to the stem-type headset. That is, the call noise reduction effect of the single-microphone headset is very limited. Summary of the invention

[0004] In view of this, the present application provides a call noise reduction method and related device to solve the problem that the call noise reduction effect of the solution of setting a single microphone on the earphone shell for sound pickup and noise reduction is very limited. The technical solution is as follows:

[0005] The first aspect of the present application provides a call noise reduction method, comprising:

[0006] Acquire a first audio signal collected by a first microphone array disposed on a first side of the headset and a second audio signal collected by a second microphone array disposed on a second side of the headset;

[0007] Performing near-field beamforming processing on the first audio signal using a first near-field beamformer disposed on a first side of the earphone to obtain a first beamformed signal, and performing near-field beamforming processing on the second audio signal using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal;

[0008] Determining a target beamforming signal according to the first beamforming signal and the second beamforming signal;

[0009] The target beamforming signal is denoised using a pre-trained audio denoising model to obtain a denoised audio signal.

[0010] In a possible implementation, the call noise reduction method further includes:

[0011] The target beamforming signal is fused with the noise-reduced audio signal, and the fused signal is used as the final noise-reduced audio signal.

[0012] In a possible implementation manner, determining a target beamforming signal according to the first beamforming signal and the second beamforming signal includes:

[0013] For corresponding signal frames of the first beamforming signal and the second beamforming signal, a minimum value at the same frequency point is taken to obtain a target beamforming signal.

[0014] In a possible implementation, performing near-field beamforming processing on the first audio signal using a first near-field beamformer to obtain a first beamformed signal, and performing near-field beamforming processing on the second audio signal using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal includes:

[0015] Using a first near-field super-directional beamformer, according to a first near-field super-directional beam coefficient, near-field beamforming processing is performed on the first audio signal to obtain a first near-field super-directional beamforming signal, and using multiple first near-field null-steer matching beamformers in different directions, according to the first near-field null-steer matching beam coefficients in corresponding directions, near-field beamforming processing is performed on the first audio signal respectively to obtain multiple first near-field null-steer matching beamforming signals;

[0016] A second near-field super-directional beamformer is used to perform near-field beamforming processing on the second audio signal according to a second near-field super-directional beam coefficient to obtain a second near-field super-directional beamforming signal, and a plurality of second near-field null-steer matching beamformers in different directions are used to perform near-field beamforming processing on the second audio signal according to the second near-field null-steer matching beam coefficients in corresponding directions to obtain a plurality of second near-field null-steer matching beamforming signals.

[0017] In a possible implementation manner, the process of determining the first near-field super-directive beam coefficient and the second near-field super-directive beam coefficient includes:

[0018] In an anechoic room environment, obtaining multi-channel voice signals collected by the first microphone array and the second microphone array for a headphone wearer, wherein the number of channels of the multi-channel voice signals is the total number of microphones of the first microphone array and the second microphone array;

[0019] Acquire, according to the multi-channel speech signals, a relative transfer function of the first microphone array on speech and a relative transfer function of the second microphone array on speech;

[0020] The first near-field super-directional beam coefficient is determined based on the coherence function matrix of the first microphone array and the relative transmission function of the first microphone array on speech, and the second near-field super-directional beam coefficient is determined based on the coherence function matrix of the second microphone array and the relative transmission function of the second microphone array on speech.

[0021] In a possible implementation, acquiring, according to the multi-channel speech signals, a relative transfer function of the first microphone array on speech and a relative transfer function of the second microphone array on speech includes:

[0022] Determine, according to the multi-channel speech signals, a relative transmission function of a total microphone array consisting of the first microphone array and the second microphone array for speech, wherein the relative transmission function of the total microphone array for speech can characterize the difference between speech signals collected by each non-reference microphone in the total microphone array and speech signals collected by a reference microphone;

[0023] According to the relative transmission function of the total microphone array on speech, the relative transmission function of the first microphone array on speech and the relative transmission function of the second microphone array on speech are obtained.

[0024] In a possible implementation, a process of determining a first near-field null-steering matching beam coefficient and a second near-field null-steering matching beam coefficient in one direction includes:

[0025] In an anechoic room environment, obtaining a multi-path noise signal collected by the first microphone array and the second microphone array for the noise played by the speaker in the direction, wherein the number of paths of the multi-path noise signal is the total number of microphones of the first microphone array and the second microphone array;

[0026] According to the multipath noise signal, acquiring a relative transmission function of the first microphone array on the directional noise and a relative transmission function of the second microphone array on the directional noise;

[0027] A first near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the first microphone array on speech and the relative transmission function of the noise in the direction, and a second near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the second microphone array on speech and the relative transmission function of the noise in the direction.

[0028] In a possible implementation, acquiring, according to the multipath noise signal, a relative transfer function of the first microphone array on the directional noise and a relative transfer function of the second microphone array on the directional noise includes:

[0029] Determine, according to the multipath noise signal, a relative transmission function of a total microphone array consisting of the first microphone array and the second microphone array on the directional noise, wherein the relative transmission function of the total microphone array on the directional noise can characterize a difference between noise signals collected by each non-reference microphone in the total microphone array and noise signals collected by a reference microphone;

[0030] According to the relative transmission function of the total microphone array on the directional noise, the relative transmission function of the first microphone array on the directional noise and the relative transmission function of the second microphone array on the directional noise are obtained.

[0031] In a possible implementation manner, fusing the target beamforming signal with the noise-reduced audio signal includes:

[0032] Weighting the target beamforming signal according to a set first weight, and weighting the noise-reduced audio signal according to a set second weight, to obtain two weighted signals, wherein the sum of the first weight and the second weight is 1;

[0033] The two weighted signals are summed.

[0034] In a possible implementation manner, the number of microphones in the first microphone array is the same as the number of microphones in the second microphone array;

[0035] The first microphone array and the second microphone array are symmetrically arranged on a first side and a second side of the earphone.

[0036] A second aspect of the present application provides a call noise reduction device, comprising: an audio signal acquisition module, a near-field beamforming processing module, a target beamforming signal determination module, and a post-filtering module;

[0037] The audio signal acquisition module is used to acquire a first audio signal collected by a first microphone array arranged on a first side of the earphone and a second audio signal collected by a second microphone array arranged on a second side of the earphone;

[0038] The near-field beamforming processing module is configured to perform near-field beamforming processing on the first audio signal using a first near-field beamformer disposed on a first side of the earphone to obtain a first beamformed signal, and to perform near-field beamforming processing on the second audio signal using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal;

[0039] The target beamforming signal determination module is used to determine a target beamforming signal according to the first beamforming signal and the second beamforming signal;

[0040] The post-filtering module is used to perform noise reduction on the target beamforming signal based on a pre-trained audio noise reduction model to obtain a noise-reduced audio signal.

[0041] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned call noise reduction methods.

[0044] A fourth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned call noise reduction methods.

[0045] A fifth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of any one of the above-mentioned call noise reduction methods.

[0046] By means of the above technical scheme, the call noise reduction method provided by the present application utilizes the beamforming technology of the microphone array to realize the sound selection in the spatial direction and enhance the voice of the wearer. Considering that the noise reduction effect of the beamforming scheme of the monaural microphone array is not ideal, the present application proposes a beamforming scheme of the binaural microphone array, that is, microphone arrays are respectively arranged on both sides of the earphone, and the audio signals synchronously collected by the two microphone arrays are respectively subjected to near-field beamforming processing to obtain a first beamforming signal and a second beamforming signal, and then the final beamforming signal, i.e., the target beamforming signal, is determined according to the first beamforming signal and the second beamforming signal. In order to further improve the noise reduction effect, after obtaining the target beamforming signal, the present application utilizes the pre-trained audio noise reduction model to further perform noise reduction on the target beamforming signal to suppress residual noise. The call noise reduction method provided by the present application has a good noise reduction effect and a good user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0048] Figure 1 A flowchart of a call noise reduction method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of a process for determining a first near-field super-directional beam coefficient and a second near-field super-directional beam coefficient provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of a process for determining a first near-field null-steering matching beam coefficient and a second near-field null-steering matching beam coefficient in any direction provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of a call noise reduction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] In view of the fact that the call noise reduction effect of the solution of setting a single microphone on the earphone shell for sound pickup and noise reduction is very limited, the inventor of this case conducted research. The initial idea was to set a microphone array (such as 2 to 4 microphones) on one side of the headset, and use the beamforming technology of the microphone array to achieve sound selection in the spatial direction and enhance the wearer's voice. Although the above idea can improve the call noise reduction effect to a certain extent, the inventor of this case studied the above idea and found that although the above idea utilizes the spatial information of multiple microphones, the directionality of the beamforming is limited and the noise reduction effect is still not ideal. In view of this, the inventor of this case continued to conduct research, and through continuous research, finally provided a call noise reduction method with better noise reduction effect.

[0056] Next, the call noise reduction method provided by the present application is introduced through the following embodiments.

[0057] See also Figure 1 , shows a flow chart of a call noise reduction method provided in an embodiment of the present application, and the call noise reduction method may include:

[0058] Step S101: acquiring a first audio signal collected by a first microphone array disposed on a first side of the earphone and a second audio signal collected by a second microphone array disposed on a second side of the earphone.

[0059] The earphones in this embodiment are headphones provided with a first microphone array and a second microphone array. The first microphone array can be arranged in an earmuff shell on a first side (for example, the left side) of the earphone, and the second microphone array is arranged in an earmuff shell on a second side (for example, the right side) of the earphone.

[0060] Preferably, the number of microphones in the first microphone array is the same as the number of microphones in the second microphone array, and the first microphone array and the second microphone array are symmetrically arranged in the earmuff shells on both sides. Exemplarily, the first microphone array includes three microphones, namely L1, L2 and L3, and the second microphone array includes three microphones, namely R1, R2 and R3, wherein L1 is symmetrically arranged with R1, L2 is symmetrically arranged with R2, and L3 is symmetrically arranged with R3. It should be noted that this embodiment does not limit the setting positions of the two microphone arrays. For example, the first microphone array and the second microphone array can both be arranged in a straight line, and the first microphone array and the second microphone array can also be arranged in a ring shape.

[0061] In this embodiment, a first microphone array arranged on a first side of the earphone and a second microphone array arranged on a second side of the earphone synchronously collect audio signals. The first audio signal collected by the first microphone array is a multi-channel audio signal, and the second audio signal collected by the second microphone is also a multi-channel audio signal. Assuming that the number of microphones in the first microphone array and the number of microphones in the second microphone array are both 3, the first audio signal and the second audio signal are both 3-channel audio signals, that is, a total of 6-channel audio signals can be collected based on the first microphone array and the second microphone array.

[0062] Step S102: using a first near-field beamformer disposed on a first side of the earphone to perform near-field beamforming processing on the first audio signal to obtain a first beamformed signal, and using a second near-field beamformer disposed on a second side of the earphone to perform near-field beamforming processing on the second audio signal to obtain a second beamformed signal.

[0063] Considering that the distance between the sound source (human mouth) and the first microphone array and the second microphone array is very close, which belongs to a near-field scene, this embodiment uses a first near-field beamformer disposed on the first side of the headset to perform near-field beamforming processing on the first audio signal to obtain a first beamforming signal, and at the same time, uses a second near-field beamformer disposed on the second side of the headset to perform near-field beamforming processing on the second audio signal to obtain a second beamforming signal. Among them, the first beamforming signal and the second beamforming signal are both directionally enhanced audio signals.

[0064] Step S103: determining a target beamforming signal according to the first beamforming signal and the second beamforming signal.

[0065] Specifically, for corresponding signal frames of the first beamforming signal and the second beamforming signal, a minimum value at the same frequency point is taken to obtain a target beamforming signal.

[0066] Step S104: using the pre-trained audio noise reduction model to perform noise reduction on the target beamforming signal to obtain a noise-reduced audio signal.

[0067] Beamforming processing can realize sound selection in a spatial direction and suppress noise in other directions. In order to further suppress noise, after obtaining the target beamforming signal, this embodiment performs post-filtering on the target beamforming signal to suppress residual noise. Specifically, the target beamforming signal is denoised using a pre-trained audio noise reduction model.

[0068] The audio noise reduction model is trained using the training noisy audio signal and the clean audio signal (noise-free audio signal) corresponding to the training noisy audio signal. The training goal of the audio noise reduction model is to make the audio signal obtained after noise reduction of the training noisy audio signal approach the clean audio signal corresponding to the training noisy audio signal. The audio noise reduction model can be trained using training data in the usage scenario of the headset to obtain an audio noise reduction model suitable for the usage scenario of the headset.

[0069] The audio noise reduction model in this embodiment may be an audio noise reduction model based on FSMN (Feedforward Sequential Memory Networks). Of course, this embodiment is not limited thereto. For example, the audio noise reduction model may also be an audio noise reduction model based on CNN, an audio noise reduction model based on TSN, etc. Considering that the headset is a low-power product, the number of parameters and the computational complexity of the selected network model should be as small as possible.

[0070] The call noise reduction method provided in the embodiment of the present application utilizes the beamforming technology of the microphone array to realize the sound selection in the spatial direction and enhance the voice of the wearer. Considering that the noise reduction effect of the beamforming scheme of the monaural microphone array is not ideal, the embodiment of the present application proposes a beamforming scheme of the binaural microphone array, that is, microphone arrays are respectively arranged on both sides of the earphone, and the audio signals synchronously collected by the two microphone arrays are respectively subjected to near-field beamforming processing to obtain a first beamforming signal and a second beamforming signal, and then the final beamforming signal, i.e., the target beamforming signal, is determined according to the first beamforming signal and the second beamforming signal. In order to further improve the noise reduction effect, after obtaining the target beamforming signal, the embodiment of the present application utilizes the pre-trained audio noise reduction model to further perform noise reduction on the target beamforming signal to effectively suppress the residual noise. The call noise reduction method provided in the embodiment of the present application has a good noise reduction effect and a good user experience.

[0071] Considering that in the case of extremely low signal-to-noise ratio, the audio noise reduction model may have the problem of eliminating speech, especially the beginning of each sentence and the position of consonants. In view of this, in another embodiment of the present application, in addition to the above steps S101 to S104, the call noise reduction method may also include the following steps:

[0072] The target beamforming signal is fused with the denoised audio signal (the denoised audio signal is obtained by denoising the target beamforming signal using the audio denoising model), and the fused signal is used as the final denoised audio signal.

[0073] In a possible implementation, the process of fusing the target beamforming signal with the denoised audio signal may include: weighting the target beamforming signal according to a set first weight, and weighting the denoised audio signal (a denoised audio signal obtained by denoising the target beamforming signal using an audio denoising model) according to a set second weight to obtain two weighted signals; and summing the two weighted signals. The sum of the first weight and the second weight is 1.

[0074] If the target beamforming signal is represented as y BF , the denoised audio signal obtained by denoising the target beamforming signal using the audio denoising model is represented as y net , then the final denoised audio signal y out for:

[0075] y out =αy BF +(1-α)y net (1)

[0076] Among them, α is the first weight, 1-α is the second weight, and the specific value of α can be set according to the actual application scenario. For example, α is 0.1 to 0.3.

[0077] In another embodiment of the present application, the specific implementation process of "Step S102: using a first near-field beamformer disposed on a first side of the earphone to perform near-field beamforming processing on the first audio signal to obtain a first beamformed signal, and using a second near-field beamformer disposed on a second side of the earphone to perform near-field beamforming processing on the second audio signal to obtain a second beamformed signal" in the above embodiment is introduced.

[0078] The process of performing near-field beamforming processing on the first audio signal by using a first near-field beamformer disposed on a first side of the earphone to obtain a first beamformed signal, and performing near-field beamforming processing on the second audio signal by using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal may include:

[0079] Step a: using a first near-field super-directional beamformer, according to a first near-field super-directional beam coefficient, to perform near-field beamforming processing on the first audio signal to obtain a first near-field super-directional beamforming signal; and using a plurality of first near-field null-steer matching beamformers in different directions, according to the first near-field null-steer matching beam coefficients in corresponding directions, to perform near-field beamforming processing on the first audio signal respectively to obtain a plurality of first near-field null-steer matching beamforming signals.

[0080] Among them, multiple first near-field null-notch matching beamformers in different directions may include: a first near-field null-notch matching beamformer in the 0° direction (suppressing noise in the 0° direction), a first near-field null-notch matching beamformer in the 90° direction (suppressing noise in the 90° direction), a first near-field null-notch matching beamformer in the 180° direction (suppressing noise in the 180° direction) and a first near-field null-notch matching beamformer in the 270° direction (suppressing noise in the 270° direction). It should be noted that the above-mentioned 0° direction, 90° direction, 180° direction and 270° direction are directions relative to the headphone wearer. It should also be noted that this embodiment does not limit the multiple first near-field null-steer matching beamformers in different directions to include the first near-field null-steer matching beamformers in the above four directions. For example, the multiple first near-field null-steer matching beamformers in different directions may also include first near-field null-steer matching beamformers in 0° direction, 45° direction, 90° direction, 135° direction, 180° direction, 225° direction, 270° direction, and 315° direction.

[0081] This embodiment uses a first near-field super-directional beamformer and multiple first near-field null-steer matching beamformers to perform beamforming processing on the first audio signal respectively, thereby obtaining multiple first beamforming signals (i.e., one first near-field super-directional beamforming signal and multiple first near-field null-steer matching beamforming signals).

[0082] Step b: using a second near-field super-directional beamformer, according to the second near-field super-directional beam coefficient, near-field beamforming processing is performed on the second audio signal to obtain a second near-field super-directional beamforming signal, and using multiple second near-field null-steer matching beamformers in different directions, according to the second near-field null-steer matching beam coefficients in corresponding directions, near-field beamforming processing is performed on the second audio signal respectively to obtain multiple second near-field null-steer matching beamforming signals.

[0083] Exemplarily, the multiple first near-field null-steer matching beamformers in different directions include first near-field null-steer matching beamformers in the 0° direction, the 90° direction, the 180° direction and the 270° direction, and the multiple second near-field null-steer matching beamformers in different directions may also include second near-field null-steer matching beamformers in the 0° direction, the 90° direction, the 180° direction and the 270° direction.

[0084] This embodiment uses a second near-field super-directional beamformer and multiple second near-field null-steer matching beamformers to perform beamforming processing on the second audio signal respectively, thereby obtaining multiple second beamforming signals (i.e., one second near-field super-directional beamforming signal and multiple second near-field null-steer matching beamforming signals).

[0085] Considering that the null-matched beamformer has a better noise reduction effect on stationary noise such as white noise, while the near-field super-directional beamformer has a better noise reduction effect on diffuse noise, in order to obtain a better noise reduction effect, this embodiment combines the two near-field beamformers to reduce the noise of the audio signal.

[0086] Exemplarily, a first near-field super-directional beamformer and a first near-field null-pitched matching beamformer in four different directions (0°, 90°, 180° and 270°) are used to perform near-field beamforming processing on the first audio signal to obtain five first beamformed signals. A second near-field super-directional beamformer and a second near-field null-pitched matching beamformer in four different directions (0°, 90°, 180° and 270°) are used to perform near-field beamforming processing on the second audio signal to obtain five second beamformed signals, i.e., a total of ten beamformed signals can be obtained.

[0087] After obtaining 10 beamforming signals, the target beamforming signal can be determined according to the 10 beamforming signals. Specifically, for the 1st signal frame of the 1st near-field beamforming signal, the 1st signal frame of the 2nd near-field beamforming signal, ..., the 1st signal frame of the 10th near-field beamforming signal, the minimum value of the same frequency point is taken; for the 2nd signal frame of the 1st near-field beamforming signal, the 2nd signal frame of the 2nd near-field beamforming signal, ..., the 2nd signal frame of the 10th near-field beamforming signal, the minimum value of the same frequency point is taken; for the 3rd signal frame of the 1st near-field beamforming signal, the 3rd signal frame of the 2nd near-field beamforming signal, ..., the 3rd signal frame of the 10th near-field beamforming signal, the minimum value of the same frequency point is taken, and so on, the target beamforming signal can be obtained.

[0088] Next, the determination process of the first near-field super-directional beam coefficient and the second near-field super-directional beam coefficient is introduced.

[0089] like Figure 2 As shown, the process of determining the first near-field super-directional beam coefficient and the second near-field super-directional beam coefficient may include:

[0090] Step S201: in an anechoic room environment, obtaining multi-channel voice signals collected by a first microphone array and a second microphone array for a headphone wearer.

[0091] The number of channels of the multi-channel voice signal is the total number of microphones in the first microphone array and the second microphone array.

[0092] The headphone wearer can be allowed to read the text in an anechoic room environment, and then the voice signal of the headphone wearer is collected based on the first microphone array and the second microphone array, so as to obtain a multi-channel voice signal.

[0093] Exemplarily, the first microphone array and the second microphone array include three microphones respectively, and six channels of voice signals can be collected based on the first microphone array and the second microphone array.

[0094] Step S202: acquiring a relative transfer function of the first microphone array on speech and a relative transfer function of the second microphone array on speech according to the multi-channel speech signals.

[0095] Specifically, according to the multi-channel speech signals, the process of obtaining the relative transmission function of the first microphone array on the speech and the relative transmission function of the second microphone array on the speech may include:

[0096] Step S2021: Determine, based on the multi-channel speech signals, a relative transmission function of the total microphone array consisting of the first microphone array and the second microphone array for speech.

[0097] The relative transfer function of the total microphone array on speech can characterize the difference between the speech signals collected by each non-reference microphone in the total microphone array and the speech signals collected by the reference microphone. The microphone closest to the headphone wearer's mouth in the total microphone array (for example, the microphone closest to the headphone wearer's mouth on the left ear side) can be used as the reference microphone.

[0098] Assuming that the total microphone array includes M microphones (the first microphone array and the second microphone array each include M / 2 microphones), the M-channel speech signals collected from the headphone wearer based on the total microphone array including M microphones can be expressed as s(n)=[s1(n),s2(n),s3(n),…,s M (n)], where n in the brackets represents the nth moment, s1(n) represents the speech signal collected by the reference microphone, and s2(n) to s M (n) represents the speech signals collected by M-1 non-reference microphones respectively. The relative transfer function of the total microphone array on speech can be expressed as:

[0099] H2(ω)~H M (ω) can be calculated as follows:

[0100]

[0101] In the above formula The speech signal s1(n) collected by the reference microphone and the speech signal s collected by the mth microphone m The cross power spectrum of (n)(2≤m≤M), is the autopower spectrum of the speech signal s1(n) collected by the reference microphone.

[0102] Step S2022: Obtain the relative transmission function of the first microphone array on speech and the relative transmission function of the second microphone array on speech according to the relative transmission function of the total microphone array on speech.

[0103] In order to obtain the relative transfer function of the total microphone array on speech After that, you can Extract the relative transfer function of the first microphone array on speech and use The relative transfer function of the second microphone array on speech is extracted.

[0104] Exemplarily, the first microphone array and the second microphone array include 3 microphones respectively, that is, the total microphone array includes 6 microphones, and the 6-channel voice signals collected for the headphone wearer based on the total microphone array including 6 microphones are expressed as s(n)=[s1(n), s2(n), s3(n), ..., s6(n)], where s1(n) is the voice signal collected by the reference microphone, and the relative transfer function of the total microphone array on the voice can be expressed as Assuming that s1(n)~s3(n) are collected by the first microphone array and s4(n)~s6(n) are collected by the second microphone array, the relative transfer function of the total microphone array on speech is Extract 1~H3(ω) to obtain the relative transfer function of the first microphone array on speech [1,H2(ω),H3(ω)], and from Extract H4(ω)~H6(ω) to obtain the relative transfer function of the second microphone array on speech [H4(ω), H5(ω), H6(ω)].

[0105] Step S2023, determine the first near-field super-directional beam coefficient according to the coherence function matrix of the first microphone array and the relative transmission function of the first microphone array on speech, and determine the second near-field super-directional beam coefficient according to the coherence function matrix of the second microphone array and the relative transmission function of the second microphone array on speech.

[0106] The far-field super-directive beam is the beam with the maximum directivity factor, and its beamformer can be expressed as:

[0107]

[0108] Among them, Γ(ω) is the coherence function matrix of the microphone array in the diffuse field. Once the microphone array structure is determined, the coherence function matrix is ​​determined. is the steering vector corresponding to the target direction, θ0 represents the direction of the human mouth relative to the reference microphone. Using the Lagrange multiplier method, the super-directional beam coefficient can be obtained as Considering that the actual earphone cannot obtain the sound source signal, the relative steering vector can be used to replace the steering vector. In addition, the real noise scene is used to simulate the diffuse field environment, and the coherence function matrix Γ of the microphone array is used to approximate the coherence function matrix in the diffuse field. Solving the near-field super-directional beam coefficient is equivalent to the following optimization problem:

[0109]

[0110] Solving equation (4) yields the near-field super-directive beam coefficient:

[0111]

[0112] In formula (5) represents the relative transfer function of the microphone array on speech, and Γ represents the coherence function matrix of the microphone array.

[0113] Substituting the coherence function matrix of the first microphone array and the relative transmission function of the first microphone array for speech obtained in step S2022 into formula (5) for calculation, the first near-field super-directional beam coefficient can be obtained. Similarly, substituting the coherence function matrix of the second microphone array and the relative transmission function of the second microphone array for speech obtained in step S2022 into formula (5) for calculation, the second near-field super-directional beam coefficient can be obtained.

[0114] In order to realize beamforming processing, in addition to determining the first near-field super-directive beam coefficient and the second-field super-directive beam coefficient, it is also necessary to determine the first near-field null-steer matching beam coefficient of multiple first near-field null-steer matching beam formers in different directions in the corresponding directions, and the second near-field null-steer matching beam coefficient of multiple second near-field null-steer matching beam formers in different directions in the corresponding directions. Next, the process of determining the first near-field null-steer matching beam coefficient and the second near-field null-steer matching beam coefficient in any direction is introduced.

[0115] like Figure 3 As shown, the process of determining the first near-field null-steering matching beam coefficient and the second near-field null-steering matching beam coefficient in any direction may include:

[0116] Step S301: in an anechoic room environment, obtaining a multi-path noise signal collected by a first microphone array and a second microphone array with respect to the noise played by a speaker in the direction.

[0117] The number of paths of the multi-path noise signal is the total number of microphones in the first microphone array and the second microphone array.

[0118] A single loudspeaker may be used to play white noise at a distance of 1 m in the direction (1 m from the headphone wearer), and then the multi-path noise signals may be collected based on the first microphone array and the second microphone array.

[0119] Exemplarily, the first microphone array and the second microphone array each include three microphones, the direction is 90°, a single speaker is used to play white noise at 1m in the 90° direction, and six noise signals can be collected based on the first microphone array and the second microphone array.

[0120] Step S302: according to the multi-path noise signal, obtaining a relative transmission function of the first microphone array on the noise in the direction and a relative transmission function of the second microphone array on the noise in the direction.

[0121] Specifically, according to the multipath noise signal, the process of obtaining the relative transmission function of the first microphone array on the directional noise and the relative transmission function of the second microphone array on the directional noise may include:

[0122] Step S3021: Determine, according to the multi-path noise signal, a relative transfer function of the total microphone array consisting of the first microphone array and the second microphone array on the noise in the direction.

[0123] The relative transfer function of the total microphone array on the directional noise can characterize the difference between the noise signal collected by each non-reference microphone in the total microphone array and the noise signal collected by the reference microphone.

[0124] Assuming that the total microphone array includes M microphones (the first microphone array and the second microphone array each include M / 2 microphones), the M-path noise signals collected by the total microphone array including M microphones for the noise played by the speaker in this direction can be expressed as The subscript n of s represents noise, and the n in brackets represents the nth moment. represents the noise signal collected by the reference microphone, represents the noise signals collected by M-1 non-reference microphones respectively. The relative transfer function of the total microphone array on the noise in this direction can be expressed as

[0125] In It can be calculated as follows:

[0126]

[0127] In the above formula (6), Noise signal collected by the reference microphone The noise signal collected by the mth microphone The cross power spectrum of Noise signal collected by the reference microphone The autopower spectrum of .

[0128] Step S3021: Obtain a relative transmission function of the first microphone array on the directional noise and a relative transmission function of the second microphone array on the directional noise according to the relative transmission function of the total microphone array on the directional noise.

[0129] Get the relative transfer function of the total microphone array on the noise in this direction After that, you can Extract the relative transfer function of the first microphone array on the noise in this direction, and use The relative transfer function of the second microphone array on the noise in the direction is extracted.

[0130] Exemplarily, the first microphone array and the second microphone array each include three microphones, that is, the total microphone array includes six microphones, and the six-channel noise signals collected by the total microphone array for the noise played by the speaker in the direction can be expressed as For the noise signal collected by the reference microphone, the relative transfer function of the total microphone array on the noise in this direction can be expressed as Assumptions The first microphone array collects The second microphone array collects the noise, and the relative transfer function of the total microphone array on the noise in this direction is Extract Get the relative transmission function of the first microphone array on the noise in this direction And from the relative transfer function of the total microphone array on the noise in this direction Extract Get the relative transfer function of the second microphone array on the noise in this direction

[0131] Step S303: Determine a first near-field null matching beam coefficient in the direction based on the relative transmission function of the first microphone array on speech and the relative transmission function of the first microphone array on noise in the direction, and determine a second near-field null matching beam coefficient in the direction based on the relative transmission function of the second microphone array on speech and the relative transmission function of the second microphone array on noise in the direction.

[0132] The delay-and-add beamformer is the most widely used fixed beamformer. Its goal is to maximize the white noise gain. The constraint is that the response of the beam pattern in the target direction is 1. Since the microphone array in this application is an irregular array, this application proposes the concept of matching beams based on the principle of delay-and-add beamformer, that is, replacing the steering vector in the delay-and-add beamformer with the relative transfer function of the microphone array. Solving the beam coefficient is equivalent to the following optimization problem:

[0133]

[0134] Where M is the number of microphones, I M×M It is an M×M unit matrix. The beam coefficients can be obtained by Lagrange multiplier method.

[0135] Since the target sound source of the headset's microphone array is near the wearer's lips and its direction is fixed, it is hoped that the matching beamformer can further reduce the noise in other directions while ensuring that the target sound source response remains unchanged. From this point of triggering, several null directions can be added to the matching beamformer, namely in, is the relative transfer function of the microphone array on the noise in the i-th direction. Solving the near-field null matching beam coefficient is equivalent to the following optimization problem:

[0136]

[0137] In the above formula (8), represents the relative transfer function of the microphone array on speech, The above formula (8) indicates that the output of the microphone array is minimized with the constraints that the speech output is distortion-free and the output in the noise direction is 0.

[0138] Substituting the relative transmission function of the first microphone array on speech and the relative transmission function of the first microphone array on the noise in that direction into formula (8) and solving it, the first near-field null matching beam coefficient in that direction can be obtained. Similarly, substituting the relative transmission function of the second microphone array on speech and the relative transmission function of the second microphone array on the noise in that direction into formula (8) and solving it, the second near-field null matching beam coefficient in that direction can be obtained.

[0139] The above introduces the call noise reduction method provided in the embodiment of the present application, and the device corresponding to the above call noise reduction method will be introduced below.

[0140] See also Figure 4 , Figure 4 A structural schematic diagram of a call noise reduction device provided in an embodiment of the present application, the call noise reduction device may include: an audio signal acquisition module 401, a near-field beamforming processing module 402, a target beamforming signal determination module 403 and a post-filtering module 404.

[0141] The audio signal acquisition module 401 is used to acquire a first audio signal collected by a first microphone array arranged on a first side of the earphone and a second audio signal collected by a second microphone array arranged on a second side of the earphone.

[0142] The near-field beamforming processing module 402 is used to perform near-field beamforming processing on the first audio signal using a first near-field beamformer arranged on the first side of the earphone to obtain a first beamformed signal, and to perform near-field beamforming processing on the second audio signal using a second near-field beamformer arranged on the second side of the earphone to obtain a second beamformed signal.

[0143] The target beamforming signal determining module 403 is configured to determine a target beamforming signal according to the first beamforming signal and the second beamforming signal.

[0144] The post-filtering module 404 is used to perform noise reduction on the target beamforming signal based on the pre-trained audio noise reduction model to obtain a noise-reduced audio signal.

[0145] In a possible implementation, the call noise reduction device may further include: a signal fusion module.

[0146] The signal fusion module is used to fuse the target beamforming signal with the noise-reduced audio signal, and use the fused signal as the final noise-reduced audio signal.

[0147] In a possible implementation, when the signal fusion module fuses the target beamforming signal with the noise-reduced audio signal, it is specifically used to:

[0148] Weighting the target beamforming signal according to a set first weight, and weighting the noise-reduced audio signal according to a set second weight, to obtain two weighted signals, wherein the sum of the first weight and the second weight is 1;

[0149] The two weighted signals are summed.

[0150] In a possible implementation manner, when the target beamforming signal determination module 403 determines the target beamforming signal according to the first beamforming signal and the second beamforming signal, it is specifically configured to:

[0151] For corresponding signal frames of the first beamforming signal and the second beamforming signal, a minimum value at the same frequency point is taken to obtain a target beamforming signal.

[0152] In a possible implementation, the near-field beamforming processing module 402 includes: a first near-field beamforming processing module and a second near-field beamforming processing module.

[0153] The first near-field beamforming processing module is used to use the first near-field super-directional beamformer to perform near-field beamforming processing on the first audio signal according to the first near-field super-directional beam coefficient to obtain a first near-field super-directional beamforming signal, and use multiple first near-field null-steer matching beamformers in different directions to perform near-field beamforming processing on the first audio signal according to the first near-field null-steer matching beam coefficients in corresponding directions to obtain multiple first near-field null-steer matching beamforming signals.

[0154] The second near-field beamforming processing module is used to use the second near-field super-directional beamformer to perform near-field beamforming processing on the second audio signal according to the second near-field super-directional beam coefficient to obtain a second near-field super-directional beamforming signal, and use multiple second near-field null-steer matching beamformers in different directions to perform near-field beamforming processing on the second audio signal according to the second near-field null-steer matching beam coefficients in corresponding directions to obtain multiple second near-field null-steer matching beamforming signals.

[0155] In a possible implementation, the call noise reduction device may further include: a first beam coefficient determination module for determining a first near-field super-directional beam coefficient and a second near-field super-directional beam coefficient.

[0156] When determining the first near-field super-directional beam coefficient and the second near-field super-directional beam coefficient, the first beam coefficient determination module is specifically used to:

[0157] In an anechoic room environment, obtaining a multi-channel voice signal collected by the first microphone array and the second microphone array for the headphone wearer, wherein the number of channels of the multi-channel voice signal is the total number of microphones of the first microphone array and the second microphone array;

[0158] According to the multi-channel speech signals, obtaining a relative transfer function of the first microphone array on the speech and a relative transfer function of the second microphone array on the speech;

[0159] A first near-field super-directional beam coefficient is determined based on a coherence function matrix of the first microphone array and a relative transmission function of the first microphone array on speech, and a second near-field super-directional beam coefficient is determined based on a coherence function matrix of the second microphone array and a relative transmission function of the second microphone array on speech.

[0160] In a possible implementation, when acquiring the relative transmission function of the first microphone array on speech and the relative transmission function of the second microphone array on speech, the first beam coefficient determination module is specifically used to:

[0161] Determine, according to the multi-channel speech signals, a relative transmission function of a total microphone array consisting of a first microphone array and a second microphone array for speech, wherein the relative transmission function of the total microphone array for speech can characterize the difference between speech signals collected by each non-reference microphone in the total microphone array and speech signals collected by a reference microphone;

[0162] According to the relative transmission function of the total microphone array on speech, the relative transmission function of the first microphone array on speech and the relative transmission function of the second microphone array on speech are obtained.

[0163] In a possible implementation, the call noise reduction device may further include: a second beam coefficient determination module for determining a first near-field null-steering matching beam coefficient and a second near-field null-steering matching beam coefficient in any direction.

[0164] When determining the first near-field null-steering matching beam coefficient and the second near-field null-steering matching beam coefficient in any direction, the second beam coefficient determination module is specifically used to:

[0165] In an anechoic room environment, obtaining a multi-path noise signal collected by the first microphone array and the second microphone array for the noise played by the speaker in the direction, wherein the number of paths of the multi-path noise signal is the total number of microphones of the first microphone array and the second microphone array;

[0166] According to the multipath noise signal, obtaining a relative transmission function of the first microphone array on the noise in the direction and a relative transmission function of the second microphone array on the noise in the direction;

[0167] A first near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the first microphone array on speech and the relative transmission function of the first microphone array on noise in the direction, and a second near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the second microphone array on speech and the relative transmission function of the second microphone array on noise in the direction.

[0168] In a possible implementation, when the second beam coefficient determination module obtains the relative transmission function of the first microphone array on the directional noise and the relative transmission function of the second microphone array on the directional noise according to the multipath noise signal, it is specifically used to:

[0169] Determine, according to the multipath noise signal, a relative transmission function of a total microphone array consisting of the first microphone array and the second microphone array on the directional noise, wherein the relative transmission function of the total microphone array on the directional noise can characterize the difference between the noise signal collected by each non-reference microphone in the total microphone array and the noise signal collected by the reference microphone;

[0170] According to the relative transmission function of the total microphone array on the directional noise, the relative transmission function of the first microphone array on the directional noise and the relative transmission function of the second microphone array on the directional noise are obtained.

[0171] In a possible implementation, the number of microphones in the first microphone array is the same as the number of microphones in the second microphone array; the first microphone array and the second microphone array are symmetrically arranged on the first side and the second side of the earphone.

[0172] The call noise reduction device provided in the embodiment of the present application utilizes the beamforming technology of the microphone array to realize the sound selection in the spatial direction and enhance the voice of the wearer. Considering that the noise reduction effect of the beamforming scheme of the monaural microphone array is not ideal, the embodiment of the present application proposes a beamforming scheme of the binaural microphone array, that is, microphone arrays are respectively arranged on both sides of the earphone, and the audio signals synchronously collected by the two microphone arrays are respectively subjected to near-field beamforming processing to obtain a first beamforming signal and a second beamforming signal, and then the final beamforming signal, i.e., the target beamforming signal, is determined according to the first beamforming signal and the second beamforming signal. In order to further improve the noise reduction effect, after obtaining the target beamforming signal, the call noise reduction device provided in the embodiment of the present application utilizes the pre-trained audio noise reduction model to further perform noise reduction on the target beamforming signal to effectively suppress the residual noise. The call noise reduction device provided in the embodiment of the present application has a good noise reduction effect and a good user experience.

[0173] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0174] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0175] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0176] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0177] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the call noise reduction method provided in the above embodiment.

[0178] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the call noise reduction method provided in the above embodiment.

[0179] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the call noise reduction method provided in the above embodiment.

[0180] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0181] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0182] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0183] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for reducing call noise, characterized in that: include: Acquire a first audio signal collected by a first microphone array disposed on a first side of the headset and a second audio signal collected by a second microphone array disposed on a second side of the headset; Performing near-field beamforming processing on the first audio signal using a first near-field beamformer disposed on a first side of the earphone to obtain a first beamformed signal, and performing near-field beamforming processing on the second audio signal using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal; Determining a target beamforming signal according to the first beamforming signal and the second beamforming signal; The target beamforming signal is denoised using a pre-trained audio denoising model to obtain a denoised audio signal.

2. The call noise reduction method according to claim 1, characterized in that: Also includes: The target beamforming signal is fused with the noise-reduced audio signal, and the fused signal is used as the final noise-reduced audio signal.

3. The call noise reduction method according to claim 1, characterized in that: The determining a target beamforming signal according to the first beamforming signal and the second beamforming signal includes: For corresponding signal frames of the first beamforming signal and the second beamforming signal, a minimum value at the same frequency point is taken to obtain a target beamforming signal.

4. The call noise reduction method according to claim 1, characterized in that: The using a first near-field beamformer to perform near-field beamforming processing on the first audio signal to obtain a first beamformed signal, and using a second near-field beamformer disposed on a second side of the earphone to perform near-field beamforming processing on the second audio signal to obtain a second beamformed signal, includes: Using a first near-field super-directional beamformer, according to a first near-field super-directional beam coefficient, near-field beamforming processing is performed on the first audio signal to obtain a first near-field super-directional beamforming signal, and using multiple first near-field null-steer matching beamformers in different directions, according to the first near-field null-steer matching beam coefficients in corresponding directions, near-field beamforming processing is performed on the first audio signal respectively to obtain multiple first near-field null-steer matching beamforming signals; A second near-field super-directional beamformer is used to perform near-field beamforming processing on the second audio signal according to a second near-field super-directional beam coefficient to obtain a second near-field super-directional beamforming signal, and a plurality of second near-field null-steer matching beamformers in different directions are used to perform near-field beamforming processing on the second audio signal according to the second near-field null-steer matching beam coefficients in corresponding directions to obtain a plurality of second near-field null-steer matching beamforming signals.

5. The call noise reduction method according to claim 4, characterized in that: The process of determining the first near-field super-directive beam coefficient and the second near-field super-directive beam coefficient includes: In an anechoic room environment, obtaining multi-channel voice signals collected by the first microphone array and the second microphone array for a headphone wearer, where the number of channels of the multi-channel voice signals is the total number of microphones in the first microphone array and the second microphone array; Acquire, according to the multi-channel speech signals, a relative transfer function of the first microphone array on speech and a relative transfer function of the second microphone array on speech; The first near-field super-directional beam coefficient is determined based on the coherence function matrix of the first microphone array and the relative transmission function of the first microphone array on speech, and the second near-field super-directional beam coefficient is determined based on the coherence function matrix of the second microphone array and the relative transmission function of the second microphone array on speech.

6. The call noise reduction method according to claim 5, characterized in that: The acquiring, according to the multi-channel speech signals, a relative transmission function of the first microphone array on speech and a relative transmission function of the second microphone array on speech comprises: Determine, according to the multi-channel speech signals, a relative transmission function of a total microphone array consisting of the first microphone array and the second microphone array for speech, wherein the relative transmission function of the total microphone array for speech can characterize the difference between speech signals collected by each non-reference microphone in the total microphone array and speech signals collected by a reference microphone; According to the relative transmission function of the total microphone array on speech, the relative transmission function of the first microphone array on speech and the relative transmission function of the second microphone array on speech are obtained.

7. The call noise reduction method according to claim 5, characterized in that: The process of determining a first near-field null-steering matching beam coefficient and a second near-field null-steering matching beam coefficient in one direction includes: In an anechoic room environment, obtaining a multi-path noise signal collected by the first microphone array and the second microphone array for the noise played by the speaker in the direction, where the number of paths of the multi-path noise signal is the total number of microphones of the first microphone array and the second microphone array; According to the multipath noise signal, acquiring a relative transmission function of the first microphone array on the directional noise and a relative transmission function of the second microphone array on the directional noise; A first near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the first microphone array on speech and the relative transmission function of the noise in the direction, and a second near-field null matching beam coefficient in the direction is determined based on the relative transmission function of the second microphone array on speech and the relative transmission function of the noise in the direction.

8. The call noise reduction method according to claim 7, characterized in that: The step of acquiring, according to the multipath noise signal, a relative transmission function of the first microphone array on the directional noise and a relative transmission function of the second microphone array on the directional noise comprises: Determine, according to the multipath noise signal, a relative transmission function of a total microphone array consisting of the first microphone array and the second microphone array on the directional noise, wherein the relative transmission function of the total microphone array on the directional noise can characterize a difference between noise signals collected by each non-reference microphone in the total microphone array and noise signals collected by a reference microphone; According to the relative transmission function of the total microphone array on the directional noise, the relative transmission function of the first microphone array on the directional noise and the relative transmission function of the second microphone array on the directional noise are obtained.

9. The call noise reduction method according to claim 2, characterized in that: The fusing the target beamforming signal with the noise-reduced audio signal comprises: Weighting the target beamforming signal according to a set first weight, and weighting the noise-reduced audio signal according to a set second weight, to obtain two weighted signals, wherein the sum of the first weight and the second weight is 1; The two weighted signals are summed.

10. The call noise reduction method according to claim 1, characterized in that: The number of microphones in the first microphone array is the same as the number of microphones in the second microphone array; The first microphone array and the second microphone array are symmetrically arranged on a first side and a second side of the earphone.

11. A call noise reduction device, characterized in that: include: Audio signal acquisition module, near-field beamforming processing module, target beamforming signal determination module and post-filtering module; The audio signal acquisition module is used to acquire a first audio signal collected by a first microphone array arranged on a first side of the earphone and a second audio signal collected by a second microphone array arranged on a second side of the earphone; The near-field beamforming processing module is configured to perform near-field beamforming processing on the first audio signal using a first near-field beamformer disposed on a first side of the earphone to obtain a first beamformed signal, and to perform near-field beamforming processing on the second audio signal using a second near-field beamformer disposed on a second side of the earphone to obtain a second beamformed signal; The target beamforming signal determination module is used to determine a target beamforming signal according to the first beamforming signal and the second beamforming signal; The post-filtering module is used to perform noise reduction on the target beamforming signal based on a pre-trained audio noise reduction model to obtain a noise-reduced audio signal.

12. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the call noise reduction method according to any one of claims 1 to 10.

13. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the call noise reduction method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that The method comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the call noise reduction method according to any one of claims 1 to 10.

Images