Adaptive spatial filtering system

Abstract

The invention relates to a spatial filtering system (4) comprising: - at least one first spatial filter (8), each filter being arranged to attenuate noise in a predefined direction; - a second spatial filter (9), arranged to attenuate noise in an optimized direction defined in real time; - a filter selector (10), arranged to select an optimal spatial filter; - a voice activity detector (12) arranged to detect the target voice; at least one parameter of the second spatial filter (9) being recalculated when an output signal of the voice activity detector is equal to a first value representative of an absence of the target voice.

Description

[0001] Adaptive spatial filtering system

[0002] The invention relates to the field of audio devices equipped or connected to one or more microphones.

[0003] BACKGROUND OF THE INVENTION

[0004] Many audio devices incorporate a microphone array forming a microphone antenna designed to capture a voice, known as the "target voice," and use this target voice to perform one or more specific functions. The target voice is usually the voice of the device's user, or the voice of someone the user is speaking to.

[0005] Such an audio device is, for example, a portable device (headset, earphones, hearing aid, etc.), but not necessarily. It could also be, for example, a video conferencing system, a hands-free car kit, a virtual personal assistant, any device equipped with a voice recognition module, etc.

[0006] The target voice is then returned to the user (in the case of a hearing aid, for example), or transmitted to a remote interlocutor (in the case of a headset, for example) or to a cloud server, or interpreted to command a predetermined action (in the case of a virtual personal assistant), etc.

[0007] Of course, microphones do not only capture the target voice, but all the sound signals present in the user's environment: various noises, other voices, etc. These extraneous sound signals, from various sources, degrade the acquisition of the target voice.

[0008] Numerous noise reduction methods exist to attenuate ambient noise and improve communications via audio devices. One such method involves applying a single-channel frequency noise reduction mask to the audio signal. However, the mask tends to alter the desired signal (target voice), thus reducing the method's effectiveness.

[0009] Spatial filtering, or beamforming, is an efficient and widely used method that does not alter the useful signal.

[0010] The goal is therefore to design a spatial filtering system with maximum performance. This is particularly important because the output of spatial filtering is often the input for other denoising algorithms. Improving the efficiency of spatial filtering thus generally improves the efficiency of the entire audio acquisition chain of the audio device.

[0011] SUBJECT OF THE INVENTION

[0012] The invention aims to design a spatial filtering system with maximum performance.

[0013] SUMMARY OF THE INVENTION

[0014] To achieve this goal, a spatial filtering system is proposed, designed to filter ambient noise present in primary audio signals that may also contain a target voice, and comprising:

[0015] - at least one first spatial filter, each arranged to attenuate ambient noise in at least one predefined direction;

[0016] - a second spatial filter, arranged to attenuate ambient noise in at least one optimized direction which is defined in real time;

[0017] - a filter selector, arranged to receive secondary audio signals produced by at least one first spatial filter and the second spatial filter and to select an optimal spatial filter from among at least one first spatial filter and the second spatial filter;

[0018] - a voice activity detector arranged to detect the target voice;

[0019] the spatial filtering system being arranged to recalculate at least one parameter defining the second spatial filter only when an output signal from the voice activity detector is equal to a first value representative of an absence of the target voice.

[0020] The first spatial filters each filter ambient noise in at least one predefined direction. The second spatial filter, on the other hand, is an adaptive filter that is recalculated only when the speech activity detector does not detect the target voice. This second spatial filter is calculated based on real-time ambient noise measurements and therefore adapts to all types of acoustics without being constrained by the diffuse field assumption or the direct field assumption (or any other assumption). The target voice is not present during the ambient noise estimation and therefore the calculation of the second spatial filter, which improves its reproduction by this filter.

[0021] In one particular embodiment, the speech activity detector is configured to detect the target voice within an optimal secondary audio signal produced by the optimal spatial filter. The interaction between the spatial filter and the speech activity detector recursively improves the system's efficiency. Improving the spatial filter's output allows for a more precise calculation of speech activity detection, which in turn improves the spatial filter. This recursion maximizes the system's performance until the spatial filtering system converges to its optimal solution.

[0022] We also propose a spatial filtering system as previously described, further comprising a computing module having an input to which the optimal secondary audio signal is applied and an output connected to an input of the voice activity detector, the computing module being arranged to perform an inference of a previously trained artificial intelligence model and defining a frequency denoising mask, using the optimal secondary audio signal as the input of said model.

[0023] We also propose a spatial filtering system as previously described, in which the output signal of the voice activity detector is:

[0024]

[0025] VAD(l) = kmax-kmin L ' k=kmin G Al(k< l') >

[0026] where l is a time frame index of input audio signals from which the primary audio signals are derived, k is a frequency index, kmin and kmax are minimum and maximum frequency indices, and G AI (k,r) is the frequency mask for noise reduction.

[0027] We further propose a spatial filtering system as previously described, further comprising an estimator of a direction of arrival of the target voice, into the input of which the primary audio signals are applied, said direction of arrival of the target voice being applied into the input of at least a first spatial filter and of the second spatial filter, the estimator of the direction of arrival of the target voice being activated only when the output signal of the voice activity detector is equal to a second value representative of a presence of the target voice.

[0028] We also propose a spatial filtering system as previously described, in which the estimator of the arrival direction of the target voice uses a GCC-PHAT algorithm.

[0029] We also propose a spatial filtering system as previously described, in which the second spatial filter is defined as a function of a delay L VAD necessary for the output signal of the voice activity detector to change from the first value to a second value representative of the presence of the target voice.

[0030] We further propose a spatial filtering system as previously described, in which at least one parameter of the second spatial filter includes a covariance matrix

[0031]

[0032] , B + 1) of ambient noise, which is such that:

[0033] < P NN Çk, l> B + 1) = ^B+I^NNC.^' I 1, B + 1) + (1 (Zc, l ^

[0034]

[0035] VAD)Y H (k,l — L VAD ), where oc B+1 is a smoothing parameter, K(fc, Z) = [Yi(.k,l Y M (k,l)] are the primary audio signals, k is a frequency index, l is a time frame index of the input audio signals from which the primary audio signals are derived, B + l is an identifier of the second spatial filter.

[0036] We also propose a spatial filtering system as previously described, in which the voice activity detector includes an accelerometer.

[0037] We also propose a spatial filtering system as previously described, in which the primary audio signals, weighted by weighting coefficients, are also applied to the input of the filter selector, the weighting coefficient associated with each microphone aiming to compensate for a difference in amplitude of the primary audio signals due to a difference between:

[0038] - the distance between said microphone and the target voice source, and

[0039] - the distance between the microphone closest to the target voice source and the target voice source.

[0040] We also propose a spatial filtering system as previously described, in which at least one first spatial filter and the second spatial filter are MVDR filters.

[0041] We also propose an audio device comprising a processing unit in which the spatial filtering system as previously described is implemented.

[0042] We also propose a filtering method, designed to filter ambient noise present in primary audio signals that may also contain a target voice, the method being implemented in a processing unit of an audio device as previously described, and comprising the steps of:

[0043] - acquire the primary audio signals;

[0044] - use at least one first spatial filter, each to attenuate ambient noise in at least one predefined direction;

[0045] - use a second spatial filter to attenuate ambient noise in at least one optimized direction that is defined in real time;

[0046] - use a filter selector to receive secondary audio signals produced by at least one first spatial filter and the second spatial filter and to select an optimal spatial filter from among at least one first spatial filter and the second spatial filter;

[0047] - use a voice activity detector to detect the target voice;

[0048] recalculate at least one parameter defining the second spatial filter only when an output signal from the voice activity detector is equal to a first value representative of an absence of the target voice.

[0049] We also propose a computer program comprising instructions which lead the processing unit of the device as previously described to execute the steps of the filtering process as previously described.

[0050] In addition, a computer-readable recording medium is proposed, on which the computer program as previously described is recorded.

[0051] The invention will be better understood in light of the following description of particular, non-limiting embodiments of the invention.

[0052] BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Reference will be made to the attached drawings, among which:

[0054] [Fig. 1] Figure 1 represents the spatial filtering system according to a first embodiment;

[0055] [Fig. 2] Figure 2 represents an estimator of the direction of arrival of the target voice;

[0056] [Fig. 3] Figure 3 represents the spatial filtering system according to a second embodiment.

[0057] DETAILED DESCRIPTION OF THE INVENTION With reference to Figure 1, an audio device 1, which is here for example a headset, comprises a plurality of microphones 2 forming a microphone antenna, and a processing unit 3.

[0058] The processing unit 3 is an electronic and software unit. The processing unit 3 includes one or more processing components, and for example a processor or any general-purpose or specialized microprocessor (for example a DSP, for Digital Signal Processor, or a GPU, for Graphics Processing Unit, or an NPU, for Neural Processing Unit), a microcontroller, or a programmable logic circuit such as an FPGA (for Field Programmable Gate Arrays) or an ASIC (for Application Specific Integrated Circuit).

[0059] The processing unit 3 also includes one or more memories (including one or more non-volatile memories), connected to or integrated into the processing component(s). At least one of these memories forms a computer-readable storage medium on which is stored at least one computer program comprising instructions that lead the processing unit 3 to execute the steps of the filtering process that will be described.

[0060] The audio device 1 incorporates a spatial filtering system 4 which implements said filtering process.

[0061] The microphones 2 capture the sound signals present in the environment of the audio device 1 and produce input audio signals. The sound signals include a useful signal, in this case the target voice Vc, but also ambient noise. By "ambient noise," we mean any non-useful sound signal captured by the microphones 2, originating from any type of "parasitic" source.

[0062] The spatial filtering system 4 aims to improve the acquisition of the target voice Vc. The target voice is, for example, the user's voice, or the voice of someone the user is speaking to. It can be a predefined voice, or a voice identified in real time as the target voice. The method used to identify the target voice Vc is known to those skilled in the art and is not described here.

[0063] The filtering system 4 comprises a number of functional modules, which here are all software and / or hardware modules (digital and / or analog) implemented in the processing unit 3.

[0064] These functional modules include:

[0065] - buffers 5 each connected to the output of a separate microphone 2;

[0066] - 6 computing modules, each associated with a separate microphone 2;

[0067] - an estimator 7 of the direction of arrival of the target voice Vc (or vDoA, for Voice Direction of Arrival); - at least one, in this case several first spatial filters 8;

[0068] - at least one, in this case only one second spatial filter 9;

[0069] - a filter selector 10;

[0070] - a calculation module 11;

[0071] - a voice activity detector 12 (or VAD, for Voice Activity Detector) arranged to detect the target voice Vc;

[0072] - a calculation module 14.

[0073] The audio device includes a number M of 2 microphones, with:

[0074] M > 1.

[0075] The processing unit 3 acquires the input audio signals produced by the microphones 2, n being

[0076]

[0077] the temporal index, the index of the time frames of the input audio signals,

[0078]

[0079] The input audio signals y m (n, Z) are digital signals. The microphones 2 are digital microphones here, but they could be analog microphones whose output would then be digitized by one or more analog-to-digital converters of the processing unit 3. The input audio signal y m(n,l) produced by each microphone 2 is measured in real time, and stored in the associated buffer 5.

[0080] Each buffer 5 has a size of bsize, and therefore:

[0081] n\ {1, bsize}.

[0082] Each calculation module 6 then calculates the Short Time Fourier Transform (STFT) of the input audio signal y m (n) associated on a window of size:

[0083] wsize = 2. bsize.

[0084] The output of each computing module 6 therefore represents the spectrum of each input audio signal and forms a primary audio signal, denoted Y m (k, Z), where k is the frequency index. The primary audio signals here are therefore signals in the frequency domain.

[0085] The primary audio signals Y(k, Z) Y M (k, Z) are applied as input to estimator 7 of the direction of arrival of the target voice Vc.

[0086] Estimator 7 of the arrival direction of the target voice Vc therefore uses the spectrum of each input audio signal. Estimator 7 is activated only when the voice activity detector 12 detects the presence of said target voice Vc. The operation of the voice activity detector 12 is described in detail below.

[0087] Referring to Figure 2, the estimator 7 of the target voice arrival direction includes a calculation module 15, whose output is an input to a GCC-PHAT algorithm 16 (for Generalized Cross Correlation with Phase Transform). The GCC-PHAT algorithm is particularly well-suited for microphone antennas containing few sensors (two, for example). This method consists of calculating the interspectral matrix (or cross spectrum) between each microphone m and the 1 ermicrophone. If the antenna is linear, it only needs to be calculated once between the 1 er microphone 2 and the M ième Microphone 2 (the last one) provides the direction of arrival of the voice relative to the entire microphone array. This is true regardless of the algorithm used to estimate the direction of arrival of the voice.

[0088] Using the GCC-PHAT algorithm, the arrival direction of the target voice is calculated as follows.

[0089] Calculation module 15 first calculates:

[0090] (f>im (k> 0 = a <p lm (k, Z — 1) + (1 — ayY-iXk, l)Y m * (fc, Z),

[0091]

[0092] where Z) is the interspectral matrix between the 1 er and the m ième microphone, 0 < a < 1 a smoothing parameter.

[0093] Then, the calculation module 15 normalizes < >i m(Zc, Z) by its absolute value:

[0094] 0 01Tn(fc> O

[0095]

[0096] where 0 Wlm (fc, Z) is the normalized interspectral matrix.

[0097] Estimator 7 then calculates the inverse Fourier transform of c / ) Wlm (k, Z) to deduce the direction of arrival of the target voice Vc.

[0098] The result of this operation is the cross correlation R lm (n, Z).

[0099] A relationship can be established between the vDoA and the index ZcZx(Z) to which R lm (n, Z) gives the maximum value, namely:

[0100] idx(T),c0

[0101] vDoA (Z) = arccos

[0102] Fs.dmics lm Or:

[0103] - 0 V DOA(1) estl the angle of arrival of the target voice Vc relative to the axis between microphones 1 and m, corresponding to the antenna axis if it is linear. The point on the axis of the microphones, relative to which angle 6 is calculated V DOA r is the point equidistant from the two microphones;

[0104] - c0 = 343 m / s is the speed of sound propagation in air;

[0105] - Fs is the sampling frequency of microphones 2;

[0106] dmics lm is the distance between the microphones 1 and m.

[0107] The output of estimator 7 is therefore an estimate of the arrival angle 0 V D O A(-) of the target voice Vc.

[0108] The angle of arrival, just like the primary audio signals, is applied at the input of each of the first spatial filters 8 and the second spatial filter 9.

[0109] The first spatial filters 8 and the second spatial filter 9 are MVDR filters. In English, this stands for Minimum Variance Distortionless Response beamformer, which can be translated into French as "filtrer spatial à variance minière à réponse sans distortion" (minimal variance spatial filtering with distortion-free response). This type of filter is also known as a Capon spatial filter. These filters minimize noise while preserving the signal from the direction of the target voice.

[0110] The angle of arrival of the target voice 0 V D O A(J-) is the input to a function that updates each first spatial filter 8 to orient it in the correct direction, i.e., that of the target voice Vc. Each first spatial filter 8 updates the orientation vector df,ï) (or steering vector) of said first spatial filter 8:

[0111] d

[0112]

[0113] ( / , Z) = [l,...,a lm e~ jkdlm ^ l\...,has 1M e~ jkd ™V],

[0114] where 0 < a lm < 1 is equal to an estimate of the ratio of the amplitude of the target voice Vc between the 1 er microphone and the m ième microphone, k = < D / C0 is the wavenumber, a) = 2nf is the angular frequency, cZ lm (Z) = 7

[0115]

[0116] (x m - x target Çl')Y + (y m - ytargettl'ïY is the estimated distance between the source of the target voice and the m ième microphone, x m r y m , x target ( ), y ta rget ) are respectively the 2D coordinates of the microphone m and the target voice Vc.

[0117] We use two-dimensional (2D) coordinates because the microphone antenna is linear and there exists a 2D plane that contains the antenna and the target voice Vc. If the antenna were not linear, we would use 3D coordinates.

[0118] The coordinates of the microphone m are chosen to be fixed, while the coordinates of the target voice Vc vary according to ^VDOA.^)

[0119]

[0120] Xtarget(J'') ^■target- COS(@vDoA (0) r

[0121] y targetO-) d-target- lÇO vDoA ( ),

[0122] where target is an estimate of the distance between the source of the target voice Vc and device 1.

[0123] The equation for each first spatial filter 8, applied to the M microphones, is:

[0124] w mvdr Çk,l) d (k,i)) <t> NN (k,iy^d(k,i)) '

[0125]

[0126] where w mvdr (.k>l) contains the complex coefficients of each first spatial filter 8 to be applied to each microphone, 'Ï'NNC^ being the ambient noise covariance matrix. The first B spatial filters 8 are calculated using the orientation vector df,ï), and an ambient noise covariance matrix from the diffuse field assumption, namely:

[0127]

[0128] & NNmp (k,l,b) = sinc^r p - T m )cos(n - e nDoA ^y^ where dnDoA^b is the angle of the noise source to be preferentially attenuated,

[0129]

[0130] T p is the propagation time between the 1 er and the p ième microphone, and T m is the propagation time between the 1 er and the m ième microphone, for an acoustic source whose arrival direction is in line with the microphone antenna.

[0131] It would also be possible to use a covariance matrix of ambient noise derived from the direct field assumption, namely:

[0132]

[0133] 0 ww ( / c, / , h) = e~^^ T v~ Tm ^ cosijt ~ enDoA ^

[0134] Each first spatial filter 8 is therefore associated with an angle of a noise source to be attenuated.

[0135] Each 0 n Do^(^) is defined according to the use case, for example by the manufacturer of the device 1, or by the integrator or designer of the spatial filtering system 4, or even by the user of the device 1. We can choose, for example, to attenuate the noise preferentially at 90° for a first spatial filter, at 120° for another first spatial filter, at 180° for another first spatial filter, etc.

[0136] The processing unit 3 thus generates B first spatial filters 8 capable of attenuating noise sources from predefined directions, while preserving the target voice Vc thanks to the link between the orientation vector and the non-distortion constraint, which in other words allows the spatial filter to apply a gain of 1 to the signal coming from the position indicated by the orientation vector, so that said signal is not modified.

[0137] As we saw earlier, the adaptive spatial filtering system 4 also includes a second spatial filter 9, which is calculated in parallel with the estimator 7 and the first spatial filters 8.

[0138] The first spatial filters 8 and the second spatial filter 9 each produce a secondary audio signal:

[0139] for the first 8 spatial filters, and

[0140]

[0141] BF s+1 (fc, Z) for the second spatial filter 9.

[0142] The filter selector 10 receives the secondary audio signals produced by the first spatial filters 8 and the second spatial filter 9, and selects an optimal spatial filter from among the first spatial filters 8 and the second spatial filter 9.

[0143] The operation of filter selector 10 will be detailed below.

[0144] The output of filter selector 10 is an optimal secondary audio signal BF0(k, l), produced by the optimal spatial filter.

[0145] The Voice Activity Detector 12 detects the target voice in the optimal secondary audio signal.

[0146] The voice activity detector 12 therefore produces an output signal (called here VAD) which is equal either to a first value representing an absence of the target voice Vc (here: VAD = 0), or to a second value representing a presence of the target voice Vc (here: VAD = 1).

[0147] The operation of the voice activity detector 12 will be detailed below.

[0148] The second spatial filter 9 is recalculated, that is to say at least one parameter defining the second spatial filter 9 is recalculated, and this only when the output signal of the voice activity detector 12 is equal to the first value representing an absence of the target voice.

[0149] The second spatial filter 9 is defined according to a delay required for the output signal of the voice activity detector 12 to change from the first value to the second value.

[0150] This delay is equal to L here VAD , which is the number of time frames required for the VAD output signal to go from 0 to 1.

[0151] The output signal VAD(k,l - 1) and the primary audio signals Y1(k,l - L VAD )... Y M (k,l — L VAD ) are applied as input to calculation module 14.

[0152] Calculation module 14 calculates an estimate of a covariance matrix < P NN (k, l, B + 1) of ambient noise, which is such that:

[0153] < P NN (k, l > B + 1) = ^B+I^NNC.^' I 1, 5 + 1) + (1 (k, l ^

[0154]

[0155] VAD)Y H (k,l — L VAD ), where (*B+I es t a smoothing parameter, K(fc, Z) = [Y k, Y M (k,l)] are the primary audio signals, k is a frequency index, l is a time frame index of the input audio signals (from which the primary audio signals are derived), B + l is an identifier of the second spatial filter 9, and L VAD is the number of time frames required for the convergence of the voice activity detector 12 (which depends on its calculation method).

[0156] The parameter of the second spatial filter 9, which is recalculated only in the absence of the target voice Vc, is therefore the covariance matrix < P NN (k, l, B + 1) of ambient noise.

[0157] Calculation of the covariance matrix < P NN (k, l, B + 1) by the calculation module 14 is therefore only carried out when the output signal VAD is equal to 0, therefore only when the voice activity detector 12 does not detect the target voice Vc.

[0158] The advantage of this is that it calculates a spatial filter 9 which is based on the noise signal measured in real time, and therefore can adapt to all kinds of the most complex acoustics, without being constrained by the assumption of a diffuse field (or any other assumption) as is the case for the first spatial filters 8.

[0159] Calculate < P NN (k,l, B + 1 only when the voice activity detector 12 does not detect a voice, ensures that the target voice (or the user's voice, or any other voice) will not be present in the noise estimation, and therefore that the target voice will only be subject to the non-distortion constraint of the MVDR filter, and thus ultimately preserved by the second spatial filter 9.

[0160] The primary audio signals, the angle of arrival of the target voice G V DOA(-) and the ambient noise covariance matrix < P NN (k, l, B + 1) are therefore applied to the input of the second spatial filter 9, whose coefficients depend on these inputs.

[0161] The second spatial filter 9 is therefore defined in real time based in particular on the delay L VAD and of < P NN (k, l, B + 1).

[0162] We will now explain the importance of taking the L delay into account. VAD , and therefore to calculate < P NN (k, l, B + 1) from Z — L VAD Depending on the method used, the VAD output signal of the voice activity detector 12 may require a certain delay (here L VAD time frames) to go from 0 to 1. If < P NN (k, l, B + ) was calculated from Y(k, f), this would mean that at the time of the transition from VAD = 0 to VAD = 1, there would be L VAD frames of false negatives would introduce the target voice into the estimation of l, B + 1, thus creating a conflict with the no-distortion constraint. This would attenuate the target voice Vc, whereas we want to preserve it. Taking into account the delay L VAD helps to overcome this drawback.

[0163] The method used by filter selector 10 to select the optimal spatial filter is close to that described in the article HAFEZI, Sina, MOORE, Alastair H., GUIRAUD, Pierre, et al. Subspace Hybrid Beamforming for Head-Worn Microphone Arrays. In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023. p. 1-5.

[0164] Here, however, the selector 10 performs its calculations from the outputs of so-called "static" spatial filters, whose definition is based on the diffuse field hypothesis (these are the first spatial filters 8), but also, in an innovative way, from the output of an adaptive spatial filter (this is the second spatial filter 9).

[0165] This method works as follows. In the specific case of MVDR spatial filters, the no-distortion constraint ensures that the target voice spectrum Vc is theoretically the same at the output of each spatial filter. This is not the case for the noise spectrum. Consequently, if, for each frequency, we use the output of the spatial filter with the lowest absolute value, we ensure that the noise is minimized while the voice is maximized. This is equivalent to writing the following equation:

[0166]

[0167] where Y bf k, l-) and the output of the selected filter, w b mvd r( > is the b th spatial filter among the B + 1 calculated spatial filters, chosen by this equation: b = argmin{\w^.(k,l) H Y(k,l)\}.

[0168]

[0169] The filter selector 10 therefore selects in real time the optimal spatial filter, from among the first spatial filters 8 and the second spatial filter 9, which maximizes the efficiency of the spatial filtering (minimization of noise and maximization of the target voice).

[0170] Advantageously, in addition to the calculated B + l spatial filters, primary audio signals weighted by weighting coefficients are applied to the input of filter selector 10.

[0171] The primary weighted audio signals are therefore the following signals:

[0172] Y(k,l~) = [Y1k,l').a 11 , Y2(ik,l').a 12 ,..., Y M (k,l).a 1M ],

[0173] with a1 = 1 and,

[0174] for all i such that 1 < i < M:

[0175]

[0176] > 1 •

[0177] Introducing weighted primary audio signals into filter selector 10 offers several advantages.

[0178] It is known that spatial filtering, which generates destructive interference between signals, has the drawback of increasing background noise (for an equivalent target voice spectrum, which is the case with MVDR thanks to the non-distortion constraint). By adding the weighted spectra of the primary audio signals in filter selector 10, it is possible, in silence, to reduce the background noise of the spatial filter output to the same level as the background noise of the microphones.

[0179] When (acoustic) noise is introduced, the background noise is masked by this noise, and therefore the noise present in the microphone spectra will inevitably be louder than in the output of the spatial filters. Selector 10 will therefore select the spatial filters. This operation has no effect on the directivity.

[0180] Furthermore, it is known that spatial filters tend to amplify wind noise, which is not strictly speaking acoustic noise but is caused by the wind blowing on the diaphragm of each microphone. Wind noise may be very present on one microphone but less so on another, since the two microphones are not coincident. By adding the spectrum of signals produced by the two microphones (i.e., the weighted primary audio signals) when selecting the spatial filter, wind noise can be attenuated without altering the target voice (Vc).

[0181] The weighting coefficient associated with each microphone 2 aims to compensate for a difference in amplitude of the primary audio signals due to a difference between:

[0182] - the distance between said microphone 2 and the target voice source Vc, and

[0183] - the distance between the microphone 2 closest to the target voice source and the target voice source Vc.

[0184] For example, consider a microphone array consisting of two microphones 2 spaced 0.021 m apart. The axis of antenna 2 is aligned with the source of the target voice Vc. The source of the target voice Vc is located 0.12 m from the first microphone, the first microphone being the closest to the source of the target voice Vc. This means that the second microphone is located 0.12 + 0.021 = 0.141 m from the source of the target voice Vc. The amplitude of the target voice Vc measured by the second microphone is lower than the amplitude of the target voice Vc measured by the first microphone, the ratio between the two (to compensate for the difference) being: d2Vc = 0.141

[0185] - - - 1,175

[0186] dlVc 0.12

[0187] dlVc and d2Vc being respectively the distance between the first microphone and the target voice source Vc and the distance between the second microphone and the target voice source Vc.

[0188] Therefore, in this case we have:

[0189] <z 12 = 1.175

[0190] If this weighting is not applied, there is a risk of selecting the second microphone at the expense of one of the spatial filters, since its amplitude is lower than the output of the spatial filters (which preserve the signal amplitude due to the non-distortion constraint). If this happens, the directivity can be affected because a microphone signal (therefore omnidirectional) may be mistakenly favored over a directional signal.

[0191] Note that this operation is simpler to implement with MVDR filters. Theoretically, the spectrum of the target voice is the same at the output of the spatial filter as at the input (microphone signal), so we can add the microphone signals to the spatial filter selection with certainty that we will not alter the performance of the spatial filter (which could happen if we mistakenly selected the microphone signals permanently, for example, which cannot happen here).

[0192] The voice activity detector 12, on the other hand, works in the following way.

[0193] Conventional methods of voice activity detection, based on energy variation, fundamental frequency estimation, etc., generally produce a result that can be used for spatial filter calculation. However, in some cases, their accuracy may be insufficient. For example, a voice may be detected when non-stationary or harmonic noise is present, making the estimation of < P NN (k, l, B + 1) imprecise.

[0194] Voice activity detection calculated using artificial intelligence is advantageously used because this detection is more precise and better discriminates a voice from noise, and can even discriminate between two different voices (if the model is customized on the target voice Vc).

[0195] The calculation module 11 has an input to which the optimal secondary audio signal BF0(k,l) is applied (it is therefore connected to the output of the filter selector 10) and an output connected to an input of the voice activity detector 12.

[0196] Calculation module 11 applies a denoising frequency mask G AI (k,l) obtained by artificial intelligence, that is to say, it performs an inference from a previously trained artificial intelligence model defining the frequency mask for denoising. The model is, for example, a recurrent convolutional neural network.

[0197] Then, the voice activity detector 12 calculates the average of the frequency mask G AI (k,l) on a predefined frequency band in order to maximize the accuracy of voice activity detection.

[0198] The output signal of voice activity detector 12 is therefore:

[0199] V

[0200]

[0201] AD(T) = kmax-kmin L ' k=kmin G Al(k,l~), kmin and kmax being respectively the minimum and maximum frequency indices between which the mean is calculated, G A ik,l) the frequency denoising mask.

[0202] Note that voice activity detection 12 could be calculated differently, for example by averaging the AI ​​gains:

[0203] kmax kmax VAD(l) = a.~ - - — - YG Æ ( / cJ - l) + (l - a).- - - — - YG AI (k, l) kmax — kmin kmax — kmin

[0204]

[0205] k=kmin k=kmin The frequency axis can be complete (kmin = 1, kmax = fsize r fsize being the number of frequencies of the STFT) or be decomposed into several sub-bands if this allows better results.

[0206] This arrangement has a recursive impact and improves the system's efficiency in a feedback loop-like fashion. The output of the voice activity detector 12 is an input to the spatial filtering. If the voice activity detection is accurate, this improves the performance of the adaptive spatial filtering. However, the output of the second spatial filter 9 is the input to the artificial intelligence denoising network. Therefore, if the spatial filtering output is improved, the artificial intelligence denoising is also improved, leading to a more accurate calculation of the voice activity detection, which in turn improves the spatial filter, and so on. This recursion maximizes the performance of the spatial filter / artificial intelligence denoising combination, ultimately leading the spatial filter to converge to its optimal solution.

[0207] Note that any type of voice activity detector can be used.

[0208] Thus, with reference to Figure 3, the additional calculation module 11 is not required. In this embodiment, voice activity detection is performed directly on the optimal secondary audio signal.

[0209] To implement voice activity detection, one can, for example, use a frequency-based noise reduction mask whose values ​​tend towards 1 when the target voice is detected, and towards 0 when the target voice is not detected. The mask is applied to the optimal secondary audio signal.

[0210] Of course, the invention is not limited to the embodiments described but encompasses any variant falling within the scope of the invention as defined by the claims.

[0211] The device, which incorporates the adaptive spatial filtering system, does not necessarily have microphones itself, but can be connected to equipment that does have integrated microphones. The device then receives the input or primary audio signals.

[0212] The primary audio signals, applied to the input of the spatial filters, are not necessarily signals in the frequency domain but could be signals in the time domain.

[0213] Implementing the spatial filtering system does not necessarily require an estimator of the target voice's arrival direction. It is indeed possible to use a predefined arrival direction for the target voice. For example, one can assume that the source is located directly in front of the user with the device.

[0214] Spatial filters are not necessarily all identical, and are not necessarily MVDR filters.

[0215] For example, it is possible to generate several spatial filters to attenuate sound coming from different directions. For instance, a filter generating a cardioid pattern attenuates the signal coming from 180 degrees relative to the axis of the microphone antenna (in the case of a linear antenna), while a filter generating an acoustic dipole attenuates the signal coming from 90 degrees relative to the antenna axis. To simulate the constraint of no distortion, in order to preserve the spectrum of the target voice, it is possible to apply either time-domain or frequency-domain equalization to each of these spatial filters to compensate for the change in spectrum (of the target voice) due to interference caused by the spatial filter.

[0216] Spatial filters can include, for example, at least one filter of type LCMV (for Linearly Constrained Minimum Variance), DAS (for Delay-And-Sum), GSLC (for Generalized SideLobe Canceller), etc.

[0217] The voice activity detector, which detects the target voice, does not necessarily detect it in the optimal secondary audio signal produced by the optimal spatial filter.

[0218] The voice activity detector could, for example, include an accelerometer.

[0219] The accelerometer can be integrated into the portable audio device (for example on the surface or inside an earphone, on the headband or in the ear cushion of headphones, etc.).

[0220] The accelerometer is almost insensitive to noise. For example, it would be possible to use an algorithm based on the energy variation measured by the sensor or the estimation of the instability of the measured signal (the voice being instability, the silence / background noise of the sensor being stationary), or on the calculation of the fundamental frequency measured on the sensor (the voice generates a harmonic signal, the silence / background noise of the sensor is not harmonic).< / t>

Claims

DEMANDS 1. Spatial filtering system (4), arranged to filter ambient noise present in primary audio signals that may also contain a target voice (Vc), and comprising: - at least one first spatial filter (8), each arranged to attenuate ambient noise in at least one predefined direction; - a second spatial filter (9), arranged to attenuate ambient noise in at least one optimized direction which is defined in real time; - a filter selector (10), arranged to receive secondary audio signals produced by at least one first spatial filter and the second spatial filter and to select an optimal spatial filter from among at least one first spatial filter and the second spatial filter; - a voice activity detector (12) arranged to detect the target voice, the spatial filtering system being arranged to recalculate at least one parameter defining the second spatial filter (9) only when an output signal from the voice activity detector is equal to a first value representative of an absence of the target voice.

2. Spatial filtering system (4) according to claim 1, the voice activity detector (12) being arranged to detect the target voice in an optimal secondary audio signal produced by the optimal spatial filter.

3. A spatial filtering system according to claim 2, further comprising a computing module (11) having an input to which the optimal secondary audio signal is applied and an output connected to an input of the speech activity detector (12), the computing module (11) being arranged to perform an inference of a pre-trained artificial intelligence model and defining a frequency denoising mask (G AI(k, Ï) ), using the optimal secondary audio signal as the input of said model.

4. Spatial filtering system according to claim 3, wherein the output signal of the voice activity detector (12) is: V AD Çl) = kmax-kmin L ' k=kmin G Al (k, l~), where l is a time frame index of input audio signals from which the primary audio signals are derived, k is a frequency index, kmin and kmax are minimum and maximum frequency indices, and G AI (k, r) is the frequency mask for noise reduction.

5. Spatial filtering system according to any one of claims 2 to 4, further comprising an estimator (7) of a target voice arrival direction (Vc), into the input of which the primary audio signals are applied, said target voice arrival direction being applied into the input of at least a first spatial filter (8) and the second spatial filter (9), the target voice arrival direction estimator being activated only when the output signal of the voice activity detector (12) is equal to a second value representative of a target voice presence (Vc).

6. Spatial filtering system according to claim 5, wherein the estimator (7) of the direction of arrival of the target voice uses a generalized phase-transformed cross-correlation algorithm.

7. A spatial filtering system according to any one of the preceding claims, wherein the second spatial filter (9) is defined as a function of a delay L VAD necessary for the output signal of the voice activity detector (12) to change from the first value to a second value representative of the presence of the target voice.

8. A spatial filtering system according to claim 7, wherein at least one parameter of the second spatial filter comprises a covariance matrix < P NN (k, l, B + 1) of ambient noise, which is such that: < P NN (k, l> B + 1) = ^B+I^NNCJ^' l 1, S + 1) + (1 ^B+I)Y l ^VAD)Y H (k,l — L VAD ), Or ( B +i es t is a smoothing parameter, Y (k, Z) = Z),..., Y M(k, Z)] are the primary audio signals, k is a frequency index, Z is a time frame index of input audio signals from which the primary audio signals are derived, B + l is an identifier of the second spatial filter ( 9).

9. Spatial filtering system according to claim 1, wherein the voice activity detector includes an accelerometer.

10. A spatial filtering system according to any one of the preceding claims, wherein the primary audio signals, weighted by weighting coefficients, are also applied to the input of the filter selector (10), the weighting coefficient associated with each microphone (2) being intended to compensate for a difference in amplitude of the primary audio signals due to a difference between: - the distance between said microphone and the target voice source (Vc), and - the distance between the microphone closest to the target voice source and the target voice source.

11. Spatial filtering system according to any one of the preceding claims, wherein at least a first spatial filter (8) and a second spatial filter (9) are minimum variance spatial filters with distortion-free response.

12. Audio device comprising a processing unit (3) in which the spatial filtering system (4) according to any one of the preceding claims is implemented.

13. A filtering method, intended to filter ambient noise present in primary audio signals likely to also contain a target voice (Vc), the method being implemented in a processing unit (3) of an audio device (1) according to claim 12, and comprising the steps of: - acquiring the primary audio signals; - use at least one first spatial filter (8), each to attenuate ambient noise in at least one predefined direction; - use a second spatial filter (9) to attenuate ambient noise in at least one optimized direction which is defined in real time; - use a filter selector (10) to receive secondary audio signals produced by at least one first spatial filter and the second spatial filter and to select an optimal spatial filter from among at least one first spatial filter and the second spatial filter; - use a voice activity detector (12) to detect the target voice; - recalculate at least one parameter defining the second spatial filter only when an output signal from the voice activity detector is equal to a first value representative of an absence of the target voice.

14. Computer program comprising instructions which cause the processing unit (3) of the device according to claim 12 to perform the steps of the filtering process according to claim 13 when said program is executed on the processing unit.

15. Computer-readable recording medium on which the computer program according to claim 14 is recorded.

Images