Hearing device comprising directional system configured to adaptively optimize sound from multiple target locations

Abstract

The present application discloses a hearing device comprising a directional system configured to adaptively optimize sound from a plurality of target locations, the hearing device comprising: a microphone system comprising a plurality of microphones; a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights configured to be applied to the plurality of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality of electrical input signals; the beamformer weights are adaptively optimized for a plurality of target locations by maximizing a target signal-to-noise ratio of sounds from the plurality of target locations, where the signal-to-noise ratio is determined from a first and a second output variance of the at least one beamformer or a time-averaged version thereof, the first and second output variances are determined when the electrical input signal or the at least one beamformed signal is marked as a target (T) and a noise (V), respectively.

Description

Technical Field

[0001] The present application relates to the field of hearing devices such as hearing aids or headphones, and in particular to noise reduction in hearing devices. Background Art

[0002] In directional noise reduction, the target sound is usually assumed to come from a certain direction or position, such as Figure 1A Assuming that for a given target position, the relative transfer function between the microphones is known, we can generate a directional signal that attenuates noise under the constraint that the target position does not change.

[0003] For frequency band k, given a microphone input signal x(k) (x(k) = [x1(k), ..., x M (k)] T ), we can obtain the output signal y from the linear combination of the input signals by multiplying each microphone signal by a (complex-valued) weight, i.e., y = w H x, where w(k) = [w1(k),…,w M (k)] T , H refers to the Hermitian transpose. In this application, this function is called "beamforming" and is provided by a beamformer.

[0004] For the (inter-microphone) noise covariance matrix R V Given a (single) target position θ in the noise field described by θ Given by:

[0005]

[0006] Among them, d θ is the relative transfer function between the microphones for the signal received from the target position θ, see Figure 1A The normalization factor in the denominator ensures that the weights scale the output signal so that the target signal (provided by the beamformer) is unchanged compared to the target signal (incident at) the reference microphone. The target position θ depends on the target direction and the distance from the target to the microphone array (comprising M microphones, each providing a microphone signal x m ,m=1,…,M). In the frequency domain, d θ is an M×1 vector, which will contain M-1 complex values ​​in addition to the value 1 at the reference microphone position due to normalization with the reference microphone. Similarly, the weight w (at each frequency index k) θ is to include the M electrical input signals x1(k),…,x M (k) is an Mx1 vector of values ​​(usually complex values).

[0007] The covariance matrix of the vector X = (X1,…,X M ) T is a general term of the matrix C, and the element C of the matrix C Xi,Xj Equal to X i and X j The covariance of

[0008] C Xi,Xj =cov[X i ,X j ]=E[(X i -E[X i ])(X j -E[X j ])]

[0009] Where E is the expectation operator.

[0010] The covariance matrix (in the context of audio processing in the time-frequency domain) is defined as R x (k,l)=E[x(k,l)x H (k, l)] (when exemplified for a noisy microphone signal x), where k and l are frequency index and time index respectively, and x is an M-dimensional vector:

[0011] x(k,l)=[X1(k,l),…,X M (k,l)]

[0012] comprises the value (usually complex valued) of each of the M microphone signals at a given frequency and time (k, l).

[0013] When the input microphone signal vector x(k,l) is labeled as target (T) and noise (N), the inter-microphone target covariance matrix R T (k, l) and the inter-microphone noise covariance matrix R V (k, l) may be defined as the corresponding covariance matrix of the input microphone signal vector x(k, l) comprising the values ​​of the M electrical input signals (microphone signals) at frequency k and time l. Labeled as target (T) and noise (N) may be implemented, for example, by a band-level voice activity detector (assuming that the target signal comprises voice (e.g., speech)).

[0014] In general, the distance between a sound source and an input transducer (e.g., a microphone) that picks up sound from the sound source in question has a greater influence on the value of an acoustic transfer function (ATF) that represents the propagation of sound from the sound source to the input transducer, the smaller the distance (between the sound source and the input transducer). In other words, the larger the distance, the smaller the change in the acoustic transfer function per unit length (d(ATF) / dL, L representing the distance) for a given direction from the input transducer to the sound source, so that when the distance between the sound source and the input transducer that picks up sound from the sound source is above a threshold distance L, the distance between the sound source and the input transducer that picks up sound from the sound source is greater than the threshold distance L. th The direction may be a good approximation for defining the acoustic transfer function for a given position of the sound source. The threshold distance may for example be in the range of 1-3 m, for example about 2 m (eg determined according to the distance between the input transducers of the hearing device).

[0015] Under the assumption that the target is transmitted from a single position / direction relative to the user, the optimal beamformer weights w above are θ The expression of is valid (e.g. for MVDR beamformer). This is usually not the case. The target signal may not always be transmitted from a single position / direction. We may simultaneously regard several signals as target signals or several positions / directions as target positions / directions. As described in the present invention, several simultaneous sound sources from different positions / directions may, for example, be regarded as simultaneous targets, or all sound sources (or each speech source) in the front half may, for example, be regarded as target signals. Similarly, in the case where the target position / direction is uncertain, the target position / direction may be advantageously assumed to cover a range of possible target positions / directions.

[0016] For the MVDR beamformer, the target covariance matrix R T can be determined as the steering vector d θ and its Hermitian transpose The outer product of where d θ is the steering vector comprising the relative transfer function between the microphone and the (individual) target position θ. Summary of the invention

[0017] The present invention relates primarily, but not exclusively, to a generalized eigenvector (GEV) beamformer, sometimes also referred to as a "generalized eigenvalue beamformer." The weights w of a GEV beamformer may be determined as a set of weights that maximizes the signal-to-noise ratio (SNR) given by:

[0018]

[0019] Here, the signal-to-noise ratio (SNR) is obtained from the (inter-microphone) target covariance matrix R T and the (inter-microphone) noise covariance matrix R V Sure.

[0020] We can estimate the weights w that maximize the SNR as the eigenvector belonging to the largest generalized eigenvalue (hence GEV). Note that the set of weights (w) that maximize the SNR can only be determined up to a scalar. Therefore, we can choose to scale the weights, for example to meet a unit response towards a predefined target position.

[0021] The present invention also relates to topics related to minimum variance distortionless response (MVDR) beamformers.

[0022] It is also mentioned in the present invention that the MVDR beamformer is a special case of the GEV beamformer, where the inter-microphone target covariance matrix R T is the steering vector d θ and its Hermitian transpose The outer product of gives the singular target covariance matrix.

[0023] In the present invention, the term "view vector" is sometimes used instead of the term "steering vector". The term view vector refers to the case where a single target direction or position is considered, such as in conjunction with an MVDR beamformer, where the target direction is usually the user's viewing direction (e.g., determined by the direction of the user's nose). In addition, instead of the term view vector, the term "relative transfer function" can be used.

[0024] The present invention also relates to topics related to a generalized sidelobe canceller (GSC) beamformer architecture. The GSC converts M input signals into a target preserving path (including a target preserving beamformer) and M-1 independent sidelobe cancellation beamformer paths (including corresponding target cancellation beamformers).

[0025] The MVDR beamformer as well as the GEV beamformer may be implemented as corresponding GSC structures. The MVDR beamformer as well as the GEV beamformer may be limited to implementing the beamformer using the GSC structure. Even though the target may not be limited to a single direction, we may choose to normalize the output of the GEV beamformer so that we obtain, for example, a unit response from (one) preferred position / direction.

[0026] The present invention includes multiple aspects, and the features of the apparatus and methods of different aspects can be combined between different aspects.

[0027] A general goal of the invention is to provide a basis for migrating from a scenario where the target sound source is a (single) localized (point) sound source to a scenario where the target sound originates from multiple differently localized sound sources.

[0028] Aspect 1 (Beamformer weights are determined by optimizing SNR)

[0029] One aspect of the invention relates to a hearing aid comprising a beamformer providing at least one beamformed signal as a linear combination of a plurality of electrical input signals, wherein the weights of the beamformer are determined by maximizing a target signal-to-noise ratio of sounds from a plurality of target locations. The target signal-to-noise ratio is determined, for example, based on first and second output variances of the at least one beamformer determined when the electrical input signals or the at least one beamformed signal (Y) are labeled as target and noise, respectively.

[0030] First hearing aid

[0031] In a first aspect of the present application, a hearing aid suitable for being worn by a user is provided. The hearing aid comprises:

[0032] - a microphone system comprising a plurality (M) of microphones adapted to pick up sounds from a user's environment and to provide a corresponding electrical input signal (x), wherein M is greater than or equal to 2; and

[0033] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing at least one beamformed signal (Y) as a weighted sum (Y=(w)) of the plurality (M) of electrical input signals; H x)).

[0034] The beamformer weights (w) are adaptively optimized for the plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations (θ), wherein the signal-to-noise ratio (SNR) is determined based on the first and second output variances (|Y T | 2 ,|Y V | 2 ) or the time-averaged first and second output variance (<|Y T | 2 >,<|Y V | 2 >) is determined, wherein the first and second output variances (|Y T | 2 ,|Y V | 2 ) are determined when the electrical input signal (x) or the at least one beamforming signal (Y) is marked as a target (T) and noise (V), respectively.

[0035] An improved hearing aid may thereby be provided.

[0036] Instead of the term "output variance" the term "output power" may be used in the hearing aid as defined above.

[0037] The output is defined as a set of complex-valued weights (w) multiplied by the input signal (x) which can represent processing in the time-frequency domain. As is known in the art, this can also be done in the time domain. In the time domain, this would correspond to a convolution.

[0038] The beamforming signal may be a function of time (l) and frequency (k). The labeling of the electrical input signal (or beamforming signal) as a target (T) (e.g., speech) or noise (V) may be performed at a time-frequency level (k, l) (e.g., for each time-frequency unit). The labeling may be provided by a target signal detector (e.g., including a voice activity detector). The target detector may, for example, be configured to provide an indication of whether or with what probability a time-frequency unit (k, l) includes a target signal, such as speech.

[0039] In this specification, the term "noise" may cover signals that are not marked as targets, such as natural or artificial noise, or signals that represent interference or distraction of the user from the target signal.

[0040] The (target) signal-to-noise ratio (SNR) can be expressed as:

[0041]

[0042] in<z*z> T and<z*z> V refers to the signal z in frequency band k averaged across the time frames marked as target and noise respectively. T (k,l) and z V (k,l) are the signals of time-frequency units marked as target and noise respectively.

[0043] The hearing aid may comprise a plurality of analysis filter banks configured to provide an electric input signal (x) in a time-frequency representation (k, l), wherein k is a frequency index and l is a time index. The hearing aid may comprise a plurality (M) of analysis filter banks for converting the time-domain electric input signal (x) from a plurality (M) of microphones into a corresponding electric input signal (X) in the time-frequency representation (k, l). Each time-frequency unit (k', l') of the electric input signal in the time-frequency representation (k, l) comprises a value (usually a complex value) of the electric input signal at a corresponding time (l') and frequency (k').

[0044] The hearing aid may comprise a target signal detector configured to provide an indication of whether or with what probability the time-frequency unit (k, l) comprises a target signal such as speech. The target signal detector may comprise a voice activity detector (e.g. a speech detector). The target signal does not necessarily have to be a speech signal (although it is often labeled as such). It may also be defined as a signal transmitted from certain directions (e.g. the front half of the face). The target detector may, for example, comprise an arrival direction detector.

[0045] The hearing aid may comprise a voice activity detector for estimating whether or with what probability an input signal or a time-frequency unit of the input signal comprises a voice signal at a given point in time and providing a voice activity control signal indicating the estimation result. A labelling of a target or noise may be provided by the voice activity detector (alone or in combination with other detectors). Typically, the labelling will be based on, for example, the input signal x at a reference microphone, but the labelling may also be based on a linear combination of signals from more than one microphone (which may be at least one beamforming signal (Y), but as an alternative, it may be based on any other beamformer). The voice activity detector may be configured to classify the input signal as being speech-dominant or noise-dominant (e.g. non-speech). The voice activity detector may be configured to provide the voice activity control signal in a time-frequency representation, for example such that a value of the voice activity control signal is provided for each time-frequency unit (k, l).

[0046] The voice activity detector may be based on or may include an artificial neural network (see, for example Fig.13 ).

[0047] The voice activity control signal may depend (in addition to the input audio signal, e.g. the electrical input signal or the beamforming signal) on spatial information derived from the electrical input signal (x) or a signal derived therefrom. The spatial information may result from a comparison between a beamformer having maximum sensitivity towards the front half and another beamformer having sensitivity towards the rear half ("front" and "rear" being determined e.g. relative to the user).

[0048] The electrical input signal (x) or at least one beamforming signal (Y) may be labeled as target (T) and noise (V), respectively, according to a voice activity control signal from a voice activity detector.

[0049] The target signal-to-noise ratio can be determined as the variance of the first and second outputs of at least one beamformer (|Y S | 2 ,|Y V | 2 ) or the time-averaged first and second output variance (<|Y T | 2 >,<|Y V | 2 >) between two entities.

[0050] The target signal-to-noise ratio can be calculated based on the beamformer weights (w) and the multiple electrical input signals x and x H The time-averaged inner vector product of <xx H > T and <xx H > V Determine, where <.> refers to the average across time, where <xx H > T and <xxH > V The labels may be provided, for example, by a target signal detector (eg, comprising a voice activity detector and / or a direction of arrival detector).

[0051] The target SNR can be calculated based on the beamformer weights (w) and the corresponding inter-microphone target covariance matrix (R S ) and the inter-microphone noise covariance matrix (R V ) is determined. The signal-to-noise ratio can be determined, for example, as:

[0052]

[0053] where w H R T w approximates the first output variance, w H R V w approximates the second output variance. The target covariance matrix between microphones (R T ) and the inter-microphone noise covariance matrix (R V ) are determined when the electrical input signal (x) or at least one beamforming signal (Y) is labeled as target and noise, respectively. Typically, the labeling is based on the input signal x at, for example, a reference microphone, but the labeling may also be based on a signal generated from a linear combination of more than one of a plurality of electrical input signals from the (M) microphones.

[0054] At least one beamformer may be implemented as a linear combination of two or more predetermined or adaptively determined beamformers. The beamformer weights of the predetermined beamformer may be fixed (e.g. determined during hearing aid production or during fitting of the hearing aid for a specific user's needs). The predetermined beamformer may thus be referred to as a "fixed beamformer". The two or more predetermined or adaptively determined beamformers may include M predetermined or adaptively determined beamformers, where M is greater than or equal to 2.

[0055] The first predetermined or adaptively determined beamformer may be configured to have a unity response towards the target direction. The unity response towards the target direction may be provided by a target preserving beamformer. When adjusting the beam pattern to attenuate noise, the unity gain constraint towards the target position may be applied alone. However, in principle, there may be other directions with greater directional amplification than the unity gain direction.

[0056] The first predetermined or adaptively determined beamformer may be denoted as a target preserving beamformer.The first predetermined or adaptively determined beamformer may be configured to provide a unity response (sometimes referred to as a "distortion-free response") for a selected microphone among the microphones, denoted as the reference microphone.

[0057] One or more second predetermined or adaptively determined beamformers may be configured to have a spatial minimum towards a corresponding target position of the plurality of target positions. The spatial minimum towards the target direction may be provided by a target cancellation beamformer. The second predetermined or adaptively determined beamformer may be denoted as a target cancellation beamformer. The second predetermined or adaptively determined beamformer may be composed of M-1 second beamformers.

[0058] The at least one beamformer may be implemented as a generalized sidelobe canceller (GSC) for providing at least one beamformed signal (Y) according to adaptively determined beamformer weights (w), wherein the generalized sidelobe canceller comprises M fixed beamformers, one of the M fixed beamformers is a target signal preserving beamformer, M-1 fixed beamformers are target cancellation beamformers, and each beamformer is configured to provide at least one beamformed signal (Y) according to an associated fixed beamformer weight (w). F,m , m=1,…,M) generates a beamforming signal, wherein the adaptively determined beamformer weights (w) are determined according to an adaptive parameter β or a parameter vector β.

[0059] The adaptive parameter β or parameter vector β may be determined based on the time average of the target-keeping beamformer and the M-1 target-cancelling beamformers.

[0060] Alternatively, the adaptive parameter β is determined based on the target covariance matrix and the noise covariance matrix.

[0061] In the generalized sidelobe canceller, we can express the output Y in terms of a fixed beamformer and an adaptive parameter vector β, namely:

[0062] Y=C0-β T C

[0063] where β is a (M-1)×1 vector and C is a (M-1)×1 vector of the target cancellation beamformer. It may be mentioned that in some other literature (e.g. in the reference manual cited in [Bitzer & Simmer; 2001]), β is made conjugate compared to the definition we use here. The beamformer output (Y) can also be rewritten as:

[0064] Y=C0-β1C1-β2C2…-β M-1 C M-1

[0065] The fixed, distortion-free target beamformer C0 and the fixed target cancellation beamformer C can be represented in terms of the input signal x and the beamformer weights expressed by the M×1 vector a and the M×(M-1) blocking matrix B, respectively, namely:

[0066] C0=a H x

[0067] and

[0068] C=B H x

[0069] The output can thus be expressed in terms of the input signal x and the parameters a, B and β:

[0070] Y=(a-Bβ * ) H x

[0071] We see that the output can be represented either in terms of a fixed beamformer or in terms of the input signal.

[0072] The average of the squared amplitudes of the outputs can thus be expressed as:

[0073] <|Y| 2 >=<(C0-β T C)(C0-β T C) * >

[0074] It can be rewritten as the sum of the averages of the products across different beamformers. Or it can be rewritten as:

[0075] <|Y| 2 >=<(a-Bβ * ) H xx H (a-Bβ * )>

[0076] =(a-Bβ * ) H <xx H >(a-Bβ * )

[0077] =(a-Bβ * )HR x (a-Bβ * )

[0078] That is, the output is based on the covariance matrix (such as R x ,R T or R V , depending on the input samples chosen for averaging) and the parameters a, B and β.

[0079] We can note that the estimation of the covariance matrix requires averaging across M real-valued terms and averaging across Mx(M-1) / 2 complex-valued terms (considering the covariance matrix to be Hermitian). Similarly, we also see that the expression in terms of the sum of beamformer products results in averaging across M real-valued beamformer product terms and averaging across Mx(M-1) / 2 complex-valued beamformer products (again considering that each complex-valued beamformer product is also represented by its complex conjugate).

[0080] The target covariance matrix (R T ) can be determined before the hearing aid is put into normal use. The target covariance matrix (R T ) can be determined, for example, during production or during fitting. The target covariance matrix (R T ) may be determined, for example, based on user input regarding a currently preferred target location (e.g., a direction, e.g., as indicated via a user interface, such as a graphical user interface, of an APP, such as a smart phone or similar (e.g., portable, such as handheld) processing device). The target covariance matrix (R T ) can be determined, for example, based on prior knowledge of one or more target locations.

[0081] The first output variance may be determined based on a target direction determined before normal use of the hearing aid.

[0082] The second aspect (the target covariance matrix is ​​based on the multiple steering vectors (d θ )Sure)

[0083] In one aspect, the invention relates to a hearing aid comprising a microphone / beamformer configuration representable by a full rank target covariance matrix.A second aspect (primarily but not exclusively) relates to a generalized eigenvector beamformer (GEV) (or an approximation thereof).

[0084] It is proposed to adaptively optimize beamformer weights (w) of a beamformer for multiple target locations (θ), for example, by maximizing a target signal-to-noise ratio (SNR) of sounds from the multiple target locations (θ). The SNR can be calculated based on the beamformer weights (w) and the target covariance matrix R between the microphones. T To express.

[0085] The inter-microphone target covariance matrix R of the beamformer T Can be updated to:

[0086]

[0087] in is the target variance of the target position θ, d θ is the steering vector for a given position θ, H refers to the Hermitian transpose, and the sum is performed across multiple (target sound source) positions θ. The steering vector d θIt is defined as the relative transfer function (in frequency band k) between the reference microphone and other microphones for a given target (sound source) position θ, that is, d θ (k) = h θ (k) / h θ (k,ref), where h θ (k) is the vector of the transfer function from the sound source position θ to each microphone (m=1,…,M), h θ (k, ref) is the transfer function from the sound source position θ to the reference microphone (m=ref). The summation may be performed across separately determined related target sound source positions. The position of the target sound source (relative to the hearing device microphone) may be approximated by the direction of the target sound source (i.e. from the hearing device microphone to the target sound source position). In the present invention, the parameter θ is used to cover both interpretations (position, direction, where the position may be represented, for example, by a combination of direction and distance relative to the hearing device microphone).

[0088] Target variance It can be viewed as direction-dependent weighting of different directions.

[0089] Target covariance matrix R T It can be estimated as a fixed full rank matrix by different means:

[0090] 1) Estimate the most likely target direction and obtain As described below.

[0091] 2) Estimate the most likely target direction and generate a target covariance matrix from a weighted sum of the desired target directions, e.g., the sum of target turning vectors from the front half of the face, the sum of the front target turning vectors obtained from a group of individuals.

[0092] 3) R T "Calibration" can be done in a special calibration mode, where the target sound is played from the target direction (no noise is present). In the MVDR beamformer, we will estimate the steering vector only from the target covariance matrix (assuming the target is a point source), e.g. from the eigenvector belonging to the largest eigenvalue. Here, we assume that the target is not completely described by a single direction, but we retain the full rank target covariance matrix.

[0093] For the MVDR beamformer, the target covariance matrix (a target sound source) has rank 1. In the target covariance matrix (R T ) does not have rank 1 (but rank > 1), the MVDR solution cannot be used, and a GEV (or approximated GEV) beamformer can be used instead. For a GEV beamformer (at different positions θ = θ1, ..., θ NTSThere are multiple target sound sources, where NTS is the (current) number of target sound sources. According to the present invention (as pointed out above), the target covariance matrix (R T ) can be expressed as the steering vector d of each target sound source at the corresponding position θ θ A linear combination of the outer products of (e.g., and),

[0094] Target covariance matrix R T The sum in the expression for may be, for example, a weighted sum of the (outer) products of the most likely target steering vectors (see, for example, Figure 6 ).

[0095]

[0096] where p θ is the (real) weight factor (eg, probability estimator) for a given position θ.

[0097] Parameter p θ and Not necessarily related. θ is the probability, which usually sums to 1, and the variance does not necessarily sum to 1. However, in maximizing the SNR given by the equation below,

[0098]

[0099] If we make R T By multiplying by a scalar, the optimal weight (w) remains unchanged. Therefore, if the sound source from a given direction is very energetic, it will dominate the target covariance matrix. It will also most likely be estimated as the most likely target direction.

[0100] Second hearing aid

[0101] In a second aspect of the present application, a hearing aid suitable for being worn by a user is provided. The hearing aid comprises:

[0102] - comprising a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m A microphone system of a plurality of (M) microphones having (n), m=1, ..., M, where n represents time and M is greater than or equal to 2; and

[0103] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality of (M) electrical input signals, thereby providing at least one beamformed signal as a weighted sum of the plurality of (M) electrical input signals; the hearing aid being adapted such that the beamformer weights (w) are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations, wherein the signal-to-noise ratio is a function of the beamformer weights (w), a target inter-microphone covariance matrix (R T ) and the inter-microphone noise covariance matrix (R V ) is represented by the target covariance matrix (R T ) according to the multiple steering vectors (d θ )Sure.

[0104] An improved hearing aid may thereby be provided.

[0105] The hearing aid is configured to adaptively optimize the beamformer weights to maximize the target signal-to-noise ratio (SNR). The target signal-to-noise ratio (SNR) can be given by:

[0106]

[0107] The superscript H refers to the Hermitian transpose.

[0108] The hearing aid may be configured to adaptively optimize the beamformer weights (w) by estimating the beamformer weights as the eigenvector belonging to the largest generalized eigenvalue.

[0109] The directional noise reduction system (e.g. at least one beamformer) may include (or may consist of) a generalized eigenvector beamformer (GEV). The term GEV is taken from [Warsitz & Haeb-Umbach; 2007]. However, in [Warsitz & Haeb-Umbach; 2007], a slightly different SNR is maximized, namely the ratio between the noisy input mixed signal and the noise estimate. On the other hand, in the present invention, the GEV terminology is used, where we maximize the ratio between the target covariance matrix and the noise covariance matrix. The structure of the equation is similar (i.e. the same equation estimates the maximum), but the covariance matrix in the numerator is defined differently.

[0110] A beamformed signal that may be provided by the at least one beamformer may be formed as a linear combination of at least two of the plurality of electrical input signals by multiplying each electrical input signal by a complex-valued beamformer weight (w). A beamformed signal that may be provided by the at least one beamformer may be formed as a linear combination of the plurality of electrical input signals by multiplying each of the plurality of electrical input signals by a (e.g. complex-valued) beamformer weight (w).

[0111] The beamformer weights of the multiple target positions (eg, directions) (θ) of the currently existing target sound sources may be calculated based on the multiple steering vectors (d θ ) to confirm.

[0112] Inter-microphone target covariance matrix R T It can be determined based on the steering vector of the target sound source currently being considered. The target covariance matrix R between microphones T can be determined as the target sound source currently considered (from position θ = θ1, ..., θ NTS , where NTS is the (current) number of target sound sources) and the (outer) product of the steering vector The (possibly weighted) sum of .

[0113] Target covariance matrix R T The sum in the expression for can be a weighted sum of the outer products of the most likely target steering vectors according to the following expression:

[0114]

[0115] where p θ is the (real) weight factor (eg, probability estimator) for a given position (θ).

[0116] The target covariance matrix (R T ) can be determined adaptively. Multiple target positions (θ) of the target sound source and / or the corresponding target covariance matrix (R T ) can be determined adaptively. Multiple target positions (θ) and / or target covariance matrices (R T ) can be adaptively determined using the maximum likelihood (ML) method, for example, see EP3413589A1.

[0117] The target covariance matrix (R T ) can be determined in advance. The target covariance matrix (R T ) can be determined, for example, in a procedure before the hearing aid is normally operated by a user, for example during production or during hearing aid fitting. The target covariance matrix (R T ) so that it can be fixed during use of the hearing aid.

[0118] The (predetermined) target covariance matrix (RT ) can be determined corresponding to a set of predetermined target positions (θ).

[0119] The (predetermined) target covariance matrix (R T ) can be determined by the calibration procedure in a separate calibration mode.

[0120] The target covariance matrix (R T ) can be determined as the turning vector (d θ,p ) is the sum of the outer products of . The target covariance matrix (R T ) can be determined as the turning vector (d θ,p ), weight (w d,p ) for example depending on age or gender (e.g. relative to the user's age or gender).

[0121] The target covariance matrix (R T ) can be determined as R obtained from different individuals T The target covariance matrix is ​​the sum of the target covariance matrices. The individual target covariance can be estimated by playing sounds from the desired target directions and estimating the target covariance matrix from the hearing aid microphone signals when the hearing instrument is fitted to the individual for the intended use. During the absence of noise (or during high SNR), it is easy to obtain a good target covariance estimate from an individual hearing aid user. By basing the target covariance matrix on an average across many individuals, it can be ensured that the hearing instrument performs well across a group of individual users.

[0122] The target covariance matrix (R T ) may be predetermined as a rank>1 matrix of size MxM, where M is the number of electrical input signals. The target covariance matrix may be, for example, a full rank matrix.

[0123] The number M of currently active microphone signals may, for example, be equal to the number of microphones of the microphone system.

[0124] The hearing aid may include a processor configured to apply one or more processing algorithms to the beamformed signal or the further noise reduced signal and provide a further processed signal. One of the processing algorithms may be a compression amplification algorithm configured to compensate for hearing loss of the user.

[0125] The hearing aid may comprise a transform unit for transforming a time domain signal into a signal in a transform domain, such as a frequency domain or a Laplace domain, a Z transform, a wavelet transform, etc. The transform unit may be or may comprise a time-frequency conversion unit for providing a time-frequency (TF) representation of the input signal.

[0126] The hearing aid may comprise a device for providing a plurality (M) of time-domain electrical input signals x in a time-frequency representation (k, l) mThe hearing aid may include a synthesis filter bank for converting a time-frequency representation of a signal (e.g., a further processed signal) into an output signal in the time domain. The time domain output signal may be fed to an output transducer of the hearing device, such as a speaker, for providing the output signal as a stimulus of a perceptible sound to a user of the hearing device.

[0127] The time-frequency representation may comprise an array or mapping of corresponding complex or real values ​​of the signal concerned in a specific time and frequency range. The TF conversion unit may comprise a filter bank for filtering a (time-varying) input signal and providing a plurality of (time-varying) output signals, each output signal comprising a distinct frequency range of the input signal. The TF conversion unit may comprise a Fourier transform unit (e.g. comprising a discrete Fourier transform (DFT) algorithm, or a short-time Fourier transform (STFT) algorithm, or a similar algorithm) for converting the time-varying input signal into a (time-varying) signal in the (time-)frequency domain. The hearing aid may comprise a corresponding inverse transform unit to convert one or more signals from the transform domain concerned into the time domain, depending on the specific application.

[0128] The hearing aid may comprise a post filter connected to at least one beamformer and adapted to receive at least one beamformed signal or a processed version thereof, wherein the post filter is configured to provide a gain to be applied to the at least one beamformed signal or a processed version thereof according to a post filter target signal to noise ratio (PF-SNR) to provide a further noise reduced signal. The post filter target signal to noise ratio (PF-SNR) is preferably estimated with a smaller time constant than a target signal to noise ratio (SNR) maximized by an optimization weight of the at least one beamformer. The target and noise covariance matrices (R respectively) used in estimating the post filter target signal to noise ratio (PF-SNR) are T and R V ) can be averaged with a time constant smaller than the target signal-to-noise ratio (SNR). Fig.12 Shown in.

[0129] The hearing aid may be configured to provide a target signal-to-noise ratio (SNR) and / or a post-filter target signal-to-noise ratio (PF-SNR) in a time-frequency representation.The directional noise reduction system may include a post-filter.

[0130] At least one beamformer may be configured to exhibit an undistorted response to a target sound source at a specified location. The undistorted response to a target sound source at a specified location may be expressed as w H d θ =1=(w H d θ )(d θ H w), where w is a vector of optimization weights of at least one beamformer, dθ is the steering vector for the specified preferred target position (θ).

[0131] The beamformer weights (w) of the at least one beamformer may be determined such that the at least one beamformer exhibits a distortion-free response to a target sound source at a specified position (θ).

[0132] The beamformer weights may be further adjusted, for example, to optimize the SNR of target signals from a range between + / - 15°, and the beamformer weights may be normalized to obtain a distortion-free response from 0°.

[0133] The weights of at least one beamformer are normalized so that the sound from multiple target locations (θ) exhibits a unit response. The condition for providing a unit response from multiple target locations can be written as w H R s w=1, as a constraint for the optimization weights of at least one beamformer, thereby providing unit energy of sound from multiple target directions.

[0134] The hearing aid may comprise an output transducer configured to provide an output stimulus perceivable as sound to a user based on the beamformed signal or said further processed signal.

[0135] The hearing aid may be composed of an air conduction hearing aid, a bone conduction hearing aid, a cochlear implant hearing aid, or a combination thereof, or may include an air conduction hearing aid, a bone conduction hearing aid, a cochlear implant hearing aid, or a combination thereof.

[0136] Aspect 3 (updating the covariance matrix based on voice activity detection based on DOA, FB ratio, etc.)

[0137] A third aspect relates (primarily but not exclusively) to a generalized eigenvector beamformer (GEV).A third aspect may relate to a configuration that may be represented by a full rank covariance matrix.

[0138] It is proposed to update the inter-microphone target covariance matrix and the inter-microphone noise covariance matrix of the beamformer based on voice activity detection, for example based on a) an estimated direction of arrival of sound from a target sound source, b) a comparison of the signal contents provided by a target-preserving beamformer and a target-canceling beamformer, respectively (e.g. the difference between the two), or c) speech detection.

[0139] Third hearing aid

[0140] In a third aspect of the present application, a hearing aid suitable for being worn by a user is provided. The third hearing aid comprises:

[0141] - comprising a device adapted to pick up sound from the user's environment and provide M corresponding electrical input signals x ma microphone system of M microphones with (n), m=1, ..., M, where n represents time and M is greater than or equal to 2; and

[0142] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to at least two of the M electrical input signals, wherein the beamformer weights are obtained by applying a beamformer weight (w) from a target covariance matrix R between the microphones to the at least two electrical input signals. T and the inter-microphone noise covariance matrix R V The target signal-to-noise ratio (SNR) of the sound of the determined multiple target positions (θ) is maximized and adaptively optimized for the multiple target positions (θ).

[0143] The beamformer may be configured to update the inter-microphone target covariance matrix R based on the detection of voice activity T and the inter-microphone noise covariance matrix R V , wherein the voice activity detection is based on one or more of the following:

[0144] a) the estimated direction of arrival of the sound from the target sound source;

[0145] b) comparison of the signal contents provided by the target preserving beamformer and the target cancellation beamformer, respectively; and

[0146] c) Voice Detection

[0147] The purpose of adapting the beamformer weights to multiple locations can be to enhance multiple target locations or to enhance the target signal, the direction of which is acoustically determined by the full-rank target covariance matrix.

[0148] In this specification, the term "speech detection" includes "voice detection" and may be provided, for example, by a binary speech / no speech detection of a voice activity detector. The speech (voice) detection may also include a pending state when no decision can be made as to whether speech (voice) is present or not. The voice activity detector may be implemented, for example, as a neural network based on training examples labeled as speech or no speech in time and frequency (see, for example, Fig.13 ).

[0149] The beamformer weights (w) of the at least one beamformer may be determined such that the at least one beamformer exhibits a distortion-free response to a target sound source at a specified position (θ).

[0150] The specified position may consist of a fixed position, such as in front of the listener. However, the specified position may be allowed to vary over time, for example based on the principal directions (eigenvectors) of the target covariance matrix, or simply based on selecting a column of the target covariance matrix (assuming the target covariance matrix is ​​close to rank 1). In an embodiment, only specified positions within a range of possible positions are allowed.

[0151] The signal-to-noise ratio (SNR) is given by:

[0152]

[0153] where H refers to the Hermitian transpose.

[0154] The directional noise reduction system may include a generalized eigenvector beamformer (GEV). The GEV beamformer may be configured to update the inter-microphone target covariance matrix R according to voice activity detection as specified in a), b) or c) above. T and the noise covariance matrix R V .

[0155] The hearing aid may comprise a transform unit for transforming a time domain signal into a signal in a transform domain, such as frequency domain or Laplace domain, Z transform, wavelet transform, etc. Depending on the specific application, the hearing aid may comprise a corresponding inverse transform unit to transform one or more signals from the transform domain in question into the time domain.

[0156] The voice activity detection, in particular the comparison of the signal contents provided by the target holding beamformer and the target cancellation beamformer, respectively, may be based on b') a difference or a ratio between the signal contents provided by the target holding beamformer and the target cancellation beamformer.

[0157] The hearing aid may comprise a front-to-back detector based on a comparison between a beamformer pointing forward and a beamformer pointing backward (front and back are defined relative to the user, with front being defined, for example, in the direction of the user's nose or in the direction the user is looking).

[0158] A forward-pointing beamformer may be configured to have minimal sensitivity (e.g., zero) to sound coming from behind (see Figure 2 ). A rearward-pointing beamformer can be configured to have minimal sensitivity (e.g., zero) to sound coming from the front (see Figure 2 ). In a preferred embodiment, the front beamformer has a unity response towards the front and the rear beamformer has a unity response towards the rear. Alternatively, the norms of the front and rear weight vectors may be similar.

[0159] The forward-pointing beamformer may be a supercardioid at its maximum sensitivity, configured to be most sensitive to sounds coming from in front of the user (see Figure 3 ). The rearward-pointing beamformer may be a hypercardioid at its maximum sensitivity, configured to be most sensitive to sound coming from behind the user (see Figure 3 ).

[0160] The front and rear beamformers may be based on two microphones (i.e., first order beam patterns), preferably microphones located in a horizontal plane (e.g., relative to what the user hears, on a line parallel to or close to the front-to-back axis (e.g., as determined by the user's nose)). However, the front and rear beamformers may be based on all available microphone signals.

[0161] The target directions of the beamformers towards the front and towards the rear can be fixed, for example limited to determining a predetermined steering vector d θ The user's viewing direction (θ).

[0162] However, the target direction may deviate from the user's viewing direction.

[0163] A fixed beam pattern may be selected based on any desired target angle of interest, such as a fixed beam pattern that optimizes:

[0164]

[0165] Among them, g θ is the possible weight of the signal from direction θ, d θ is the relative acoustic transfer function from the direction θ known in advance. Sum and The sum across specific (eg, pre-recorded) steering vectors (d) when assigned as belonging to the target direction and when assigned as belonging to the noise direction, respectively, is indicated.

[0166] Inter-microphone target covariance matrix R T and the inter-microphone noise covariance matrix R V The updating may be performed based on respective target update criteria and noise update criteria related to a comparison of signal contents provided by the target preserving beamformer and the target cancellation beamformer, respectively.

[0167] The target update criterion and the noise update criterion may be complementary (eg, the noise update criterion is equal to negation (target update criterion)).

[0168] The target update criterion may be, for example:

[0169] If log|C F (t,f)|-log|C B (t,f)|>κ F , then update R T

[0170] Where log is the logarithmic function, |·| refers to the absolute value (amplitude), and C F and C B are the outputs of the target-preserving (e.g., forward-facing) beamformer and the target-cancelling (e.g., backward-facing) beamformer, respectively, and κ Fand κ B is the threshold of the corresponding beamformer. F and κ B exist Figure 4A The parameters t and f represent time and frequency respectively.

[0171] The noise update criterion may be, for example:

[0172] If log|C F (t,f)|-log|C B (t,f)|≤κ B , then update R V

[0173] The parameter κ (kappa) corresponds to the magnitude difference between the two beam patterns at a given angle (θ). For the specific plot, the beam pattern difference as a function of angle is a monotonically decreasing function, so that, in this particular case, a difference of, for example, 3 dB between the front and rear beamformers may correspond to C F -C B =3dB. Thus, for all angles where the difference is greater than 3dB, we update the inter-microphone target covariance matrix R T , for all angles where the difference is less than, for example, -3dB, we update the inter-microphone noise covariance matrix R V From C F and C B When the difference between is neither greater than +3dB nor less than -3dB, we do not update either of the two covariance matrices.

[0174] Threshold κ F and κ B For example, they can be equal, for example, equal to 3dB. If the thresholds are equal, the update is front-to-back symmetric, otherwise R is updated. T , or update R V .

[0175] When C F -C B If the difference (value) is small, the beamformer (C F ,C B ) may be advantageous. The same is true for other thresholds, such as thresholds based on voice activity.

[0176] Threshold κ F and κ B For example, it may be determined adaptively, for example, according to the current acoustic environment.

[0177] When the corresponding target update criterion and noise update criterion are met, the inter-microphone target covariance matrix R T and the inter-microphone noise covariance matrix R VCan be updated recursively. The target and noise covariance matrices can be updated recursively with the same or different time constants. The time constant can be varied based on sensor input, for example, if the head is detected to be moving or turning, the update rate can be increased.

[0178] The front-to-back ratio (FBR) of the optimized beamformer can be based on the estimated target covariance matrix R T and the noise covariance matrix R V , which is determined when the corresponding target and noise update criteria are satisfied.

[0179] Figure 4A The dashed line in the figure shows the FBR of the optimized beamformer based on the estimated target covariance matrix R T and the noise covariance matrix R V , where R T Based on log|C F (t,f)|-log|C B (t,f)|>3dB time frame, R V Based on log|C F (t,f)|-log|C B The time frame when (t,f)|≤-3dB.

[0180] The front-to-back decision (determination) can be implemented using a neural network. The input stage of the FBR including the optimized beamformer is implemented using a neural network (e.g., including a gated recurrent unit (GRU) layer) in Figure 4B , 4C Shown in.

[0181] The target covariance matrix (R T ) can be a fixed (predetermined) full-rank matrix, with only the noise covariance matrix (R V ) can be adapted.

[0182] Direction-based decisions may be combined with voice activity-based decisions. For example, the target covariance may only be updated when a time-frequency unit meets a direction-based criterion and voice (or other sound defined as a sound of interest) is present.

[0183] Combination with a voice activity detector is advantageous. The combination of a voice activity detector with a direction-based decision (eg based on FBR) can ensure that the target covariance matrix is ​​updated only when both the voice-based and direction-based criteria are met.

[0184] example

[0185] Target covariance matrix R T Updates when voice is active and signal is coming from the front.

[0186] Update the noise covariance matrix RV The choice of time may be different from updating the target covariance matrix R T time.

[0187] Option 1

[0188] R V Updated when signal comes from behind.

[0189] Option 2

[0190] R V Updated when the signal is coming from behind and no speech is detected.

[0191] Option 3

[0192] R V Updated when the signal is coming from behind or when no speech is detected.

[0193] An example of a speech detector (eg, a voice activity detector) implemented as a neural network is shown below in conjunction with Fig.13 describe.

[0194] Replacement combination Figure 2 , 3 The (generally) front-to-back detector based voice activity detector described in 4A, 4B, 4C may (along the same lines) be implemented as a self-voice detector based on a comparison between at least one self-voice cancellation beamformer and another linear combination of microphones.

[0195] Instead of a voice activity detector for detecting voices in the user's environment, a self-voice detector can be implemented in a similar manner (by remembering the user's own voice as the "target" and replacing the front-facing and rear-facing beamformers with beamformers pointing towards the user's mouth and in the opposite direction (the beamformer pointing in the opposite direction is, for example, implemented as a self-voice cancellation beamformer)).

[0196] exist Figure 2 (or Figure 3 ) in the frame, for example, towards the front beamformer (C F ) is replaced by the self-voice enhancement (or preservation) beamformer, towards the rear beamformer (C B) is replaced by a self-voice cancellation beamformer. An own voice decision (OVD(k,l)) based on a comparison between the self-voice enhancement beamformer and the self-voice cancellation beamformer can be performed. The comparison can be applied in different channels. The comparison can be based on an amplitude response or a logarithmic amplitude. When the comparison result is greater than a predetermined threshold, the self-voice is detected in a specific channel. For example, if the self-voice is detected in more than X channels (e.g., in most channels (such as a frequency band)), a joint decision (OVD(k,l)) across frequencies can be determined. joint (k,l)).

[0197] OVD joint (k,l)) may be based on a trained neural network. The neural network may be a feed-forward network, a convolutional network or a recurrent network. The input layer may be based on one or more time frames of a comparison between at least one self-voice cancellation beamformer and another linear combination of input microphones, such as a self-voice enhancement beamformer. The training of the neural network may be based on audio samples labeled as self-voice or no self-voice. At the output layer, a non-linear activation function such as a sigmoid function may be applied before applying a threshold.

[0198] In addition to the comparison between beam patterns, the self-speech decision can also be based on a (general) voice activity detector, so that the self-speech is only detected when the comparison results in the detection of the self-speech and the voice. The voice activity detector can also be based on a trained neural network (see, for example, Fig.13 ). In an embodiment, the self-voice detection is based on a network trained jointly using voice features and beamformer based features.

[0199] The self-speech decision may be determined jointly across hearing aids in a binaural setting. For example, "self-speech" may only be detected when both hearing aids detect self-speech. "No self-speech" may only be detected when both hearing instruments do not detect self-speech, meaning that we may also have time frames that may be labeled as "pending".

[0200] The self-voice cancellation beamformer and the self-voice enhancement beamformer may also be applied as separate inputs to the detector (before the comparison, thereby leaving the comparison to the neural network).

[0201] In an embodiment, the neural network is implemented as a simple linear regression unit.

[0202] The target covariance matrix (R) for the (e.g. current) possible target steering vector T ) can be provided based on the likelihood estimator, namely where θ1,…,Θ denotes the (e.g. current) position of interest, estimated, for example, by a method based on maximum likelihood estimation (MLE), where p θis a weight based on the estimated probability (or likelihood) of a given direction θ (at a given point in time), see, for example, EP3413589A1.

[0203] The position range included in the search for the current relevant position of the target sound source can be limited based on a priori knowledge related to the specific listening situation (eg, specific acoustic environment), for example. The position range can be quantified, for example, see EP3413589A1.

[0204] The position range evaluated by the MLE method can be, for example, a complete range around the user that is suitable for a conversation between the user and one or more communication partners (e.g., distance ∈ [0.5m; 3m] and angle ∈ [0°, 360°]). The position range evaluated by the MLE method can be, for example, a sub-range or multiple sub-ranges of the complete range (e.g., a sub-range of the front half (e.g., a sub-range of approximately 0°) and a sub-range of the rear half (e.g., a sub-range of approximately 180°), etc.). The angular range evaluated by the MLE method can be, for example, [0°, 360°] (the complete range around the user) or limited to a sub-range such as [-90°, +90°] (e.g., the front half, e.g., relative to the user's viewing direction) or [0°, +180°] / [0°, -180°] (left or right half) or a sub-range thereof.

[0205] The term "range of locations" may (alternatively or additionally) be understood as a "range of locations across multiple individuals", such as a specific location measured across different people.

[0206] The most likely target position (or target steering vector) may be selected from a dictionary of different candidate positions. Instead of selecting a single target steering vector (such as used in an MVDR beamformer), multiple (N Θ The (currently) possible target steering vectors (given the current (noisy) electrical input signal picked up by the input transducer of the hearing device) may be determined as the vector with the greatest likelihood (e.g. probability, p θ ) of the steering vector (d θ ). The steering vector with the maximum likelihood can be, for example, a vector with a higher likelihood than the threshold (p th ) The large likelihood steering vector or the corresponding N Θ N maximum (current) likelihood values ​​(e.g., probabilities) Θ Steering vector. Probability threshold (p th ) may be greater than or equal to 0.4, for example, in the range of 0.5 to 1, for example, greater than or equal to 0.6. The number of maximum likelihood values ​​(N Θ ) may for example be in the range of 2 to 10, for example between 3 and 5, for example equal to 3 or 4.

[0207] The target covariance matrix (R T) can also be given by the average of the target covariance matrices obtained across different individuals.

[0208] Aspect 4 (GSC-MVDR beamformer (LMS or gradient update))

[0209] The mathematical division in the LMS / NLMS update may be less accurate than the division used in the single-step (direct) estimation. However, in the NLMS case, the division can be simply implemented by a shift operator, which is computationally much cheaper.

[0210] The alternation of LMS, NLMS (and Sign LMS) solutions with single-step (direct) estimation has implementation-related advantages because they only consider second-order terms (multiplication of two numbers) and not fourth-order terms (i.e. multiplication of four numbers), since fourth-order multiplication is much more difficult to implement in fixed-point arithmetic because the bit width is much wider.

[0211] Fourth hearing aid

[0212] In a fourth aspect of the present application, a hearing aid suitable for being worn by a user is provided. The fourth hearing aid comprises:

[0213] - comprising a device adapted to pick up sound from the user's environment and provide M corresponding electrical input signals x m (n), a microphone system of a plurality (M) of microphones with m=1, ..., M, where M is greater than or equal to 3, and n represents time; and

[0214] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising a generalized sidelobe canceller (GSC) structure implemented as a minimum variance distortionless response (MVDR) beamformer for providing at least one beamformed signal, the generalized sidelobe canceller (GSC) structure comprising at least three fixed beamformers, comprising:

[0215] an M×1 target preserving beamformer exhibiting an undistorted response to a target sound source at a specified location, the target preserving beamformer being configured to provide a desired response, such as an omnidirectional response, from the target location, thereby providing a target preserving signal based on beamformer weights applied to the M electrical input signals; and

[0216] at least M-1 targeted cancellation beamformers, each targeted cancellation beamformer for generating a beamforming signal for attenuating sound from a targeted sound source at a specified location according to beamformer weights configured to be applied to at least two of the M electrical input signals;

[0217] -- wherein beamformer weights of a minimum variance distortionless response (MVDR) beamformer are determined as a function of an adaptive parameter vector β; and

[0218] --The adaptive parameter vector β is determined by:

[0219] ---a) using an adaptive update algorithm including the LMS or NLMS update algorithm or the Sign LMS algorithm; or

[0220] ---b) Solve the equation in Refers to the gradient relative to β.

[0221] M x 1 target keeping beamformer refers to M input signals of the target keeping beamformer.

[0222] The NLMS algorithm can be normalized in different ways. The NLMS update term of β(μ<·>) can be normalized using the norm of the fixed target cancellation beamformer C, i.e.

[0223]

[0224] Alternatively, the NLMS update of β can be normalized using the norm of the output signal (Y):

[0225]

[0226] The numerators and denominators in the above two equations can be averaged across time, that is:

[0227]

[0228] and

[0229]

[0230] For example,<C*Y> It can be expressed as:

[0231] <C*Y> =<C*C0> -β1<C*C1> -…-β M-1 <C*C M-1 >

[0232] Where C=[C1 … C M-1 ] T

[0233] However, in this particular case, smoothing is not necessary, since the recursive update of β automatically applies smoothing. For complex-valued signals, the update of β can be expressed as:

[0234] β(n)=β(n-1)+μsign(<C*Y> )

[0235] The real part of β can be updated as:

[0236]

[0237] The imaginary part of β can be updated as:

[0238]

[0239] For example, in the case of M = 2 microphones, we have:

[0240]

[0241] In this particular case, β1 can be updated as:

[0242]

[0243] or

[0244]

[0245] For example, in the case of M>2 microphones, β1 can be updated as:

[0246]

[0247]

[0248] sign() should be understood as updating the real part of β in the direction of the sign of the real part of the gradient and updating the imaginary part of β in the direction of the sign of the imaginary part of the gradient. Since sign LMS is very simple, it is beneficial to update β several times in one frame, that is, to update β at a rate higher than the frame rate, for example 10 times per frame.

[0249] Instead of estimating the size of the update step of the gradient algorithm, we can simply take the step size in the direction of the gradient algorithm, for example:

[0250] β(n)=β(n-1)+μsign(C*Y)

[0251] in

[0252] In the special case of M = 2 microphones, we can omit the expectation and we have:

[0253] β(n)=β(n-1)+μsign(C*Y)

[0254] in

[0255] The preceding determination of β(n) can be simplified even further as shown below:

[0256] Given C = a + ib and Y = c + id, we have C*Y = ac + bd + i (ad - bc)

[0257] Thus, sign(C*Y)=sign(ac+bd)+isign(ad-bc)

[0258] For the real part, we have:

[0259]

[0260] For the imaginary part, we have:

[0261]

[0262] This shows that if sign(a) = sign(c) and sign(b) = sign(d), then if sign(a) = sign(d), we also have sign(b) = sign(c). This shows that either we have one of the following two conditions:

[0263] sign(a)=sign(c)and sign(b)=sign(d)

[0264] sign(a)≠sign(c)and sign(b)≠sign(d)

[0265] Either we have one of the following two conditions:

[0266] sign(a)=sign(d)and sign(b)≠sign(c)

[0267] sign(a)≠sign(d)and sign(b)=sign(c)

[0268] Thus, implementation of the present invention enables updating of the real or imaginary part without computing the product of ad, ac, bd or bc.

[0269] For example, instead of calculating the following equation:

[0270] β(n)=β(n-1)+μΔβ

[0271] in The implementation of the present invention is provided from the following calculation:

[0272]

[0273] and

[0274]

[0275] Thus, implementation of the present invention enables updating either the real part or the imaginary part of β for each iteration, while avoiding the costly computation of the product between two complex numbers.

[0276] The adaptive update algorithm may include, for example, a SIGN LMS algorithm.The updating of the adaptive parameters may be performed using a sign LMS algorithm (eg, a complex sign LMS algorithm), for example for the case of M>2, where the target cancellation beamformers are combined (added together).

[0277] In choosing between using the LMS or NLMS algorithm, it is worth mentioning that the division in the LMS / NLMS update may be less accurate than the division used in the single-step estimate. In the NLMS case, the division can simply be implemented by a shift operator, which is much cheaper (computationally) than the division operation. The shift operator may correspond to rounding the denominator to the nearest 2. N .

[0278] The microphone system may comprise more than two microphones. The microphone system may comprise three microphones.

[0279] Equation Can be set to isolate β i , so that each element of the β vector is given by the beamformer average and the other elements of the β vector, as outlined below.

[0280] Based on the gradients, different update rules can be derived for the three microphones.

[0281] In the case of three microphones, the gradients wrt β1 and β2 are given by:

[0282]

[0283] By setting the gradient to = 0, we obtain two expressions:

[0284]

[0285] In this solution, we do not have to choose a learning rate. We only need to choose a smoothing coefficient for the average operator <··>. Furthermore, if the β term fluctuates during convergence, it may be advantageous to apply low-pass filtering to β1 and β2. It is proposed to iteratively alternate between determining β1 (via the above equation (possibly including low-pass filtering of β1, β2), given a previously estimated value of β2) and determining β2 (via the above equation (possibly including low-pass filtering of β1, β2), given a previously estimated value of β1).

[0286] Solvable equation To isolate β i , so that each element of the β vector is given by the beamformer average only, as outlined below.

[0287] It should be noted that from the above equation, we can also express

[0288]

[0289] Insert into the following equation

[0290]

[0291] Hereby we obtain direct estimates of β1 and β2:

[0292]

[0293] and

[0294]

[0295] As can be seen, the adaptive parameter vector β (e.g. β1, β2 for a microphone system including three microphones) or the adaptive parameter β (for a microphone system including two microphones) can be estimated based on an average across M (e.g. M=2, 3) microphones (instead of estimating the noise covariance matrix (R V )).

[0296] The direct estimation of β1 and β2 is advantageous because it does not depend on previous estimates of β1 and β2, whereby convergence depends only on the beamformer average. On the other hand, the estimation depends on quartic (fourth order) terms, which may be problematic for fixed point implementations.

[0297] Instead, the alternating solution, in which the estimate of β1 depends on β2 (and vice versa), depends only on the quartic term. Both update rules still do not contain any learning rate.

[0298] By determining the specified location of the target sound source, it is ensured that the estimated MVDR beamformer weights are normalized so that the output signal has an undistorted response to the specified target location.

[0299] Equation It can be referred to as the square of the output (Y) of the MVDR beamformer (|·| 2 ) with respect to the adaptive parameter vector β, where Refers to the gradient relative to β.

[0300] The target preserving beamformer and the at least two target cancellation beamformers may be fixed beamformers (ie defined by predetermined (or occasionally (but not frequently) updated) weights).

[0301] The adaptive parameter vector β may be determined from time-averaged values ​​of a (eg fixed) target-preserving beamformer (C0) and at least two target-cancelling beamformers (C1, C2). Time-averaging the beamformers is an alternative to actually estimating the covariance matrix.

[0302] At least M-1 (eg, two) targeted cancellation beamformers may be based on a subset of the M electrical input signals.

[0303] The microphone system may include (e.g., consist of) M=3 microphones. In a first operating mode, the two targeted cancellation beamformers may be based on three electrical input signals. In a second operating mode, the two targeted cancellation beamformers may be based on two electrical input signals (e.g., see Fig. 10C The hearing aid may be adapted to switch (eg gradually) between the first and second operating modes (eg depending on a classification of the acoustic environment, or an estimate of the current remaining capacity of the hearing aid battery, or a current power consumption of the hearing aid).

[0304] For the three-input GSC beamformer, the adaptive parameter vector β can be estimated directly, but direct estimation is difficult to implement because it requires a large dynamic range (due to the quartic terms). As an alternative to direct estimation, a novel adaptive parameter vector β is proposed based on solving Eq. Make an estimate, where Refers to the gradient with respect to β. Thus, for the three-input scheme, the adaptive parameters β1 and β2 can be estimated in an alternating manner (e.g., first estimate β1, then estimate β2, then estimate β1, etc.). The estimate contains only second-order terms, which is more desirable in a fixed-point implementation.

[0305] Aspect 5 (GSC-GEV beamformer (gradient or sign LMS update))

[0306] A generalized eigenvector beamformer (GEV) (or an approximation thereof) implemented as a generalized sidelobe canceller (GSC) structure is determined as a function of the adaptive parameter β (or parameter vector β), for example a) by using an LMS (or NLMS) algorithm (e.g., the Sign-LMS algorithm), or b) by solving the equation ( is the gradient relative to β).

[0307] The directional noise reduction system may, for example, include a generalized eigenvector beamformer (GEV) solution for 2 or 3 or more input transducers (e.g., microphones), and in particular an update rule for estimating adaptive parameters (β) based on averaging across three (or more) fixed beamformers (rather than estimating the noise covariance matrix) (e.g., including gradual changes between beamformer weights for two and three input transducers (e.g., microphones)).

[0308] For two microphones, we still have only two fixed beamformers, but we also determine separate averages when speech (or target) is present, compared to just determining an average when noise is present. For two microphones, we thus estimate multiple averages based on two beamformers, but these averages are split into averages when noise is detected or averages when speech is detected, similar to updating either the speech covariance matrix or the noise covariance matrix (based on the electrical input signal from the input transducer), but in the present invention it is proposed to base the (speech and noise) covariances on the beamformed signals (from the target preserving beamformer and the target cancellation beamformer).

[0309] Fifth hearing aid

[0310] In a fifth aspect of the present application, a hearing aid suitable for being worn by a user is provided. The fifth hearing aid comprises:

[0311] - comprising a device adapted to pick up sound from the user's environment and provide M corresponding electrical input signals x m (n), a microphone system of a plurality (M) of microphones with m=1, ..., M, where M is greater than or equal to 2, and n represents time; and

[0312] - a directional noise reduction system connected to the microphone system and comprising a beamformer implemented as a generalized sidelobe canceller (GSC) for providing a noise reduction signal according to adaptively determined beamformer weights, wherein the beamformer is configured to provide a noise reduction signal to a specific target sound source (S) at a specified position (θ) θ ) exhibits a distortion-free response, wherein the generalized sidelobe canceller is configured to adaptively determine the beamformer weights to maximize a target signal-to-noise ratio (SNR) of the beamformed signal, wherein the target signal may include signals from different locations (θ i ,i=1,…,N TS ) at multiple (N TS The invention relates to a method for transmitting a beamformed signal to a target signal source of a generalized sidelobe canceller, wherein the generalized sidelobe canceller comprises a plurality of (M) fixed beamformers, each fixed beamformer being configured to generate a beam pattern / beamformed signal according to associated fixed beamformer weights, wherein the adaptively determined beamformer weights of the generalized sidelobe canceller beamformer are determined according to an adaptive parameter β or a parameter vector β.

[0313] The adaptively determined beamformer weights of the generalized sidelobe canceller beamformer may be determined according to the adaptive parameter β or parameter vector β in the following manner:

[0314] - using a gradient equation relating the gradient of the target signal-to-noise ratio with respect to the adaptation parameter β or parameter vector β; or

[0315] -Use the sign LMS algorithm.

[0316] sign LMS is a special case of the gradient algorithm where the gradient step size is restricted to move in the direction of the gradient sign (i.e., the sign of the real and imaginary parts of the gradient)

[0317] The target signal-to-noise ratio may be expressed, for example, in terms of a fixed (target-keeping and target-cancelling) beamformer and an adaptive parameter β or parameter vector β.

[0318] When the Sign LMS algorithm is used to update the adaptive parameter β or parameter vector β, the step size may be kept constant. The SignLMS algorithm may be a complex Sign LMS algorithm. The step size may be complex. The real and imaginary parts of the complex step size may be kept constant.

[0319] The advantage of sign LMS is its simplicity. For example, division during LMS can be avoided. Adaptive parameters can be updated more frequently, for example more than once per frame. The lack of accuracy in the sign-gradient step can be compensated by frequent updates of small gradient steps.

[0320] To increase the convergence rate, the step size of adaptive algorithms such as the Sign LMS algorithm can be updated with a momentum term.

[0321] By updating the beamformer weights / adaptive coefficients to optimize the overall SNR rather than the SNR from a specific target source (S θ ), a specific target signal-to-noise ratio at a specific location (θ) is maximized, and the beamformer weights / adaptive parameters of the beamformer are adjusted so that the target signal can be transmitted from more directions than the location that provides an undistorted response to it. The target sound source can be, for example, a sound source including speech. The target sound source can be, for example, a sound source including music.

[0322] The (GSC) beamformer may, for example, comprise a generalized eigenvector beamformer (GEV) (or an approximation thereof) configured to provide a noise-reduced signal determined as a function of an adaptive parameter β or parameter vector β. The generalized eigenvector beamformer (GEV) (or an approximation thereof) may, for example, comprise M fixed beamformers, each of which is configured to generate a beam pattern / beamformed signal according to associated beamformer weights. The adaptively determined beamformer weights of the GSC beamformer may, for example, be determined according to the adaptive parameter β or parameter vector β.

[0323] The (GSC) beamformer may, for example, include a plurality (eg, M) of fixed beamformers, such as one target-holding beamformer and M-1 target-cancelling beamformers.

[0324] The (GSC) beamformer may, for example, include one beamformer configured to have an undistorted response to a target sound source at a specific target location and M-1 beamformers configured as independent target cancellation beamformers for the specific target location (or locations).

[0325] Thereby, it is ensured that the estimated weights of the generalized eigenvector beamformer (GEV) (or its approximation) are normalized so that the output signal of the beamformer has an undistorted response to sounds from a specific target location.

[0326] The distortion-free position can be adaptively changed over time.

[0327] The hearing aid may comprise a post-filter connected to the at least one beamformer and adapted to receive the at least one beamformed signal or a processed version thereof, wherein the post-filter is configured to provide a gain to be applied to the at least one beamformed signal or a processed version thereof to provide a further noise reduced signal according to a post-filter target signal-to-noise ratio.

[0328] The adaptive parameter β or parameter vector β (and hence the adaptively determined beamformer weights of the GSC beamformer) may be determined, for example, by averaging across the (eg M) fixed beamformers.

[0329] The update rule according to the invention is an alternative to estimating the noise covariance matrix.

[0330] A generalized eigenvector beamformer (GEV) (or its approximation) may, for example, include one target preserving beamformer and M-1 target canceling beamformers.

[0331] In the power-saving mode of operation, the M-1 target cancellation beamformers (and optionally the target preserving beamformers) receive as input only a subset of the M electrical input signals (see, for example, Figure 8C or 10C).

[0332] The subset of the M electrical input signals may include two electrical input signals (whereby the M-1 target cancellation beamformers (and optionally the target preserving beamformers) may be first order beamformers).

[0333] Each of the M-1 targeted cancellation beamformers may be configured to take as input only a subset of the M microphone signals, e.g., such that each targeted cancellation beamformer has an input comprising a different subset of the M input signals (e.g., a subset consisting of two input signals such that each targeted cancellation beamformer becomes a first-order beamformer, e.g., a first-order cardioid).

[0334] In combination with any of the preceding aspects or in another separate aspect, the hearing aid is configured such that the first and second target cancellation beamformers of the three-microphone input scheme are based on a subset of the three microphone inputs in at least a second operating mode, wherein the directional noise reduction system is configured to switch (e.g., fade) the beamformer weights of the target-preserving beamformer and the first and second target cancellation beamformers between multiple sets of beamformer weights optimized for three (first mode) and two (second mode) microphones, respectively.

[0335] In combination with any of the preceding aspects or in another separate aspect, a hearing aid comprises a directional noise reduction system comprising a three-microphone input GSC structure, comprising a (e.g., fixed) target preserving beamformer and first and second (e.g., fixed) target cancellation beamformers, wherein the first and second target cancellation beamformers are based on a subset of two of the three microphone inputs, wherein the hearing aid is configured to switch (e.g., fade) between first and second operating modes, wherein in the first mode all three microphone inputs are active, and wherein in the second operating mode only two of the three microphone inputs are active.

[0336] The hearing aid is configured to store in a memory beamformer weights optimized in advance for the first and second operating modes of the directional system.

[0337] Transitioning from the first operating mode to the second operating mode (or vice versa) may be controlled by the user via a user interface, subject to a control signal indicative of the complexity of the current acoustic environment surrounding the user.

[0338] In a three-microphone input system (first operating mode), the GSC structure includes a (e.g., fixed) three-input target preserving beamformer and a first and a second (e.g., fixed) two-input target cancellation beamformer; while in a two-microphone input system, the GSC structure includes a (one, e.g., fixed) two-input target preserving beamformer and a single (e.g., fixed) two-input target cancellation beamformer.

[0339] The main advantage of using a two-microphone (target cancellation) beamformer in a three-microphone system is that switching between a two-microphone system and a three-microphone system becomes easier because the target cancellation beamformer can be reused.

[0340] The trigger for entering a specific operating mode (changing from a three-input to a two-input targeted cancellation beamformer) may for example be related to energy saving. The trigger may for example be based on the input level or another parameter, such as provided by a sound scene detector.

[0341] The complexity of the sound scene can trigger a switch from two to three microphones. The trigger can be a function of level, SNR, remaining battery time, motion, for example.

[0342] In general, it may be argued that battery capacity is not well spent on powering three microphones in all situations. We may apply beamforming based on three microphones only in the most complex environments. Which microphones are switched may depend on the microphone configuration in the hearing device. In the general situation where the user focuses his attention on one sound source in the environment, it may be desirable to switch towards two microphones, the axes of which are most parallel to the front-rear axis. In the self-voice pickup (telephone) operating mode of the hearing aid, different two-microphone configurations may be selected. In the situation where one of the microphones is detected to be inoperative, it may be preferred to select the two microphones that are still functional.

[0343] The hearing aid may comprise a classifier of the user's current acoustic environment. The classifier may provide a classification signal representing the current acoustic environment. The switching of which microphones of the microphone system are to be used for beamforming in a given acoustic situation may be controlled by the classification signal.

[0344] In situations with little or no noise or where the signal comes from a single direction, there is a risk that the target covariance matrix and the noise covariance (or the corresponding fixed beamformer) may converge towards the same value (i.e., R T =R V ). This can be avoided by adding a bias to the covariance matrix, i.e.

[0345]

[0346] and

[0347]

[0348] where σ T and σ V is a small constant and I is the identity matrix (with 1's on the diagonal and 0's elsewhere). Since the microphone signal will always contain microphone noise, it is most important to add a bias to the target covariance matrix, thereby ensuring that the beamformer system will converge towards an MVDR beamformer in low input level situations.

[0349] In the above example, the biases are rank 1 matrices, but the added target covariance biases can also be full rank matrices.

[0350] An SNR estimate, such as a target signal-to-noise ratio (SNR) of the beamformed signal, may be used as an input to the postfilter.

[0351] The LMS or NLMS update terms (μ<·>) of the adaptive parameters or parameter vector β may be normalized in a variety of different ways, see for example the fourth aspect.

[0352] The LMS or NLMS update rate may be further increased, for example, by adding a momentum term.

[0353] Aspect 6 (GSC-GEV beamformer (directly determined (fixed beamformer) or complex sign LMS more new))

[0354] A sixth aspect relates to a two (or three) microphone input directional system implemented as a GSC structure comprising a GEV beamformer including two (or three) fixed beamformers, wherein the update rule for estimating the adaptive parameters (β) of the GSC structure is based on one of the following: a) direct determination of the estimated values ​​of the beamforming signals based on the two (or three) fixed beamformers; and b) the Sign LMS algorithm.

[0355] Sixth hearing aid

[0356] In a sixth aspect of the present application, a hearing aid suitable for being worn by a user is provided. The sixth hearing aid comprises:

[0357] - comprising a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m (n), a microphone system of two or three microphones with m = 1, 2 or m = 1, 2, 3, n representing time; and

[0358] a directional noise reduction system connected to the microphone system and implemented as a generalized sidelobe canceller (GSC) and comprising a generalized eigenvector beamformer (GEV) (or an approximation thereof), the directional noise reduction system being configured to provide a noise reduction signal determined as a function of an adaptive parameter β, the generalized eigenvector beamformer (GEV) comprising at least two fixed beamformers, each fixed beamformer being configured to generate a beamformed signal according to an associated beamformer weight; wherein

[0359] The update rule for estimating the adaptive parameter (β) is based on at least one of the following:

[0360] - direct determination of estimates of the beamformed signals based on two fixed beamformers; and

[0361] -Update the previous value to the current value according to the complex Sign-LMS algorithm.

[0362] The number of microphones M may be equal to 2. The number of microphones M may be equal to 3.

[0363] For the two-microphone case (M=2), the cost function Two solutions (providing maximum and minimum values) for the adaptive parameter β for different equations (e.g. SNR) can be given by the following expressions:

[0364]

[0365] in

[0366]

[0367] Where C0 and C1 are the output signals of the target-preserving beamformer (C0) and the target-cancelling beamformer (C1), respectively. TN and C VN , N = 0, 1 are the values ​​of the target preserving beamformer (C0) and the target canceling beamformer (C1) during the target (T) such as speech and noise (N), respectively, where |·| refers to the amplitude, * refers to the complex conjugate, and <·> refers to the time average.

[0368] The quadratic polynomial yields two solutions for β: one value that maximizes the SNR and another that minimizes the SNR. Using Muller's method, the solution to the quadratic formula can be rewritten as:

[0369]

[0370] From this equation, we can see that for

[0371]

[0372] so, This yields an MVDR solution (ie, A=0), which is the solution that maximizes the SNR.

[0373] In the case of three microphones, a set of complex second-order polynomials of the following type needs to be solved: (and ), where (for the case of β1):

[0374]

[0375]

[0376] and

[0377]

[0378] If the target is only from the direction of the steering vector, include C T1 and C T2 All items will disappear, and <|C T0 | 2 >=1. The polynomial is thus simplified to:

[0379]

[0380] where β1 can be isolated as

[0381]

[0382] Corresponds to the MVDR solution.

[0383] For the two-microphone case (M=2), the update rule of the adaptive parameter (β) based on the complex sign LMS algorithm can be given by the following expressions for the real and imaginary parts of β:

[0384]

[0385] Among them, Y T and Y V is the nearest available estimate of the output signal of the GSC-GEV beamformer when the outputs are labeled as target (T) and (non-target) noise (N), and are the fixed step sizes of the adaptive algorithm in the real and imaginary parts of β, respectively.

[0386] For three microphones (M=3), the update rule of the adaptive parameter vector (β) based on the sign-based LMS can be given by:

[0387]

[0388] and

[0389]

[0390] Among them, C TN and C VN , N = 0, 1, 2 are the values ​​of the target-maintaining beamformer (C0) and the target-cancelling beamformer (C1, C2) during the target (T) such as speech and noise (N), respectively, where Y V and Y T Both depend on the most recent estimates of β1 and β2, and

[0391]

[0392] and

[0393]

[0394] The sign LMS algorithm (ie the adaptive parameters) may be updated more than once per time frame, for example twice per frame or five times per frame.

[0395] A seventh aspect (GSC beamformer, wherein the SNR is expressed in a beamformer optimized according to an electrical input signal Weighted)

[0396] A seventh aspect relates to a hearing aid comprising an adaptive beamformer, implemented as a generalized sidelobe canceller (GSC) comprising M fixed beamformers, wherein beamformer weights (w) of the adaptive beamformer are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations (θ), wherein the beamformer weights (w) are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations (θ). H The time-averaged inner vector product of <xx H > T and <xx H > V Make optimizations.

[0397] Seventh hearing aid

[0398] In a seventh aspect of the present application, a hearing aid suitable for being worn by a user is provided. The seventh hearing aid comprises:

[0399] - comprising a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m A microphone system of a plurality of (M) microphones having (n), m=1, ..., M, wherein M is greater than or equal to 2, and n represents time;

[0400] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality (M) of electrical input signals, wherein the at least one beamformer is implemented as a generalized sidelobe canceller (GSC) for providing the noise reduction signal according to the adaptively determined beamformer weights, wherein the generalized sidelobe canceller comprises M fixed beamformers, one of the M fixed beamformers being a target signal preserving beamformer, M-1 fixed beamformers being different target cancellation beamformers, each beamformer being configured to provide a noise reduction signal according to the associated fixed beamformer weights (w F,m , m=1,…,M) generates a beamforming signal, wherein the adaptively determined beamformer weights are determined according to an adaptive parameter β or a parameter vector β; and

[0401] wherein the beamformer weights (w) are adaptively optimized for the plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of the sound from the plurality of target locations (θ), wherein the signal-to-noise ratio (SNR) is expressed according to the beamformer weights (w), wherein the beamformer weights (w) are based on the plurality of electrical input signals x and x H The time-averaged inner vector product of <xx H > T and <xx H> V Optimize, where <.> refers to the average over time, <xx H > T and <xx H > V are determined when the electrical input signal is estimated to contain target speech (T) and noise (V), respectively.

[0402] x and x H The inner vector product of the electrical input signals from the M microphones is set as the vector x = (x1, ..., x M ) T The superscript T refers to the transpose.

[0403] The hearing aid may comprise a voice activity detector configured to estimate whether or with what probability a given input signal contains voice (eg speech) and to provide a voice activity control signal indicative of the estimation result.

[0404] The voice activity control signal may be used to decide whether to include the estimated time of target speech (T) or noise (V).

[0405] The decision may also rely on other cues such as spatial cues.

[0406] The beamformer weights (w) can be updated iteratively using a gradient-based optimization method. This is possible because the weights are given as a function of an adaptive parameter β (which can be optimized based on a gradient algorithm). This will be further described in the following paragraphs.

[0407] The beamformer weight (w) can be expressed as a function of β, that is, w = a-Bβ * , the beamforming signal Y is expressed as Y = (a-Bβ * ) H x=C0-βC,C0=a H x,C=B H x, where a is the vector containing the weights of the target preserving beamformer C0 and B is the blocking matrix containing the weights of the target canceling beamformer C. We thus have:

[0408]

[0409] Where T and V refer to "target" and "noise" (non-target), respectively.<z*z> T and<z*z> V are the averages of the signal z in frequency band k across the time frames (l) labeled as target and noise, respectively. T and z V Signals of time-frequency units labeled (eg, by a voice activity detector) as target (T) and noise (V), respectively.

[0410] gives the SNR:

[0411]

[0412] and

[0413]

[0414] We can therefore see that the SNR depends on the output signal (Y T and Y V ) is expressed either by weight w (based on β) and the target and noise covariance matrix (R T ,R V ) to express.

[0415] Aspect 8 (beamformer, wherein the beamformer weights are adaptively adjusted based on the steering vectors of multiple target locations Should be optimized)

[0416] An eighth aspect relates to a hearing aid comprising an adaptive beamformer configured to generate at least one beamformed signal according to beamformer weights (w), the beamformer weights being adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations, wherein the signal-to-noise ratio is expressed according to the beamformer weights (w), wherein the beamformer weights are ... (θ). θ ) for optimization.

[0417] Eighth hearing aid

[0418] In an eighth aspect of the present application, a hearing aid suitable for being worn by a user is provided. The eighth hearing aid comprises:

[0419] - comprising a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m A microphone system of a plurality of (M) microphones having (n), m=1, ..., M, wherein M is greater than or equal to 2, and n represents time;

[0420] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality (M) of electrical input signals;

[0421] The beamformer weights (w) are adaptively optimized for the plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of the sound from the plurality of target locations, wherein the SNR is expressed in terms of the beamformer weights (w), wherein the beamformer weights are adaptively optimized in terms of the steering vectors (d θ) for optimization.

[0422] Inter-microphone target covariance matrix R T can be determined as the target sound source currently being considered (from position θ=θ1,…,θ NTS , where NTS is the (current) number of target sound sources) and the (outer) product of the steering vector The (possibly weighted) sum of .

[0423] Target covariance matrix R T The sum in the expression can be the most likely target according to the following expression

[0424]

[0425] where p θ is the (real) weight factor (eg, probability estimator) for a given position θ.

[0426] SNR can be expressed in terms of the steering vector (view vector):

[0427]

[0428] As noted earlier, <|Y| 2 > can be written as:

[0429]

[0430] Due to a H x and B H x is the expression for the beamformer, and we see that the covariance can be replaced by the beamformer mean. Similarly, the numerator <|Y T | 2 >Can be expressed in terms of beamformers:

[0431]

[0432] The numerator can thus also be expressed in terms of a fixed beamformer value given by the weighted sum of the steering vectors multiplied by the fixed beamformer weight given by a and B.

[0433] The target covariance matrix (R T ) can be determined as the turning vector (d θ,p ) is the sum of the outer products of . The target covariance matrix (R T ) can be determined as the turning vector (d θ,p ), weight (w d,p ) for example depending on age or gender (e.g. relative to the user's age or gender).

[0434] In one formulation, the SNR is expressed in terms of the average of the products between the fixed beamformer outputs (Y). More specifically, the expected beamformer output (Y) when the target is dominant is T ) and the expected beamformer output (Y V ).

[0435] The optimized SNR is estimated as follows:

[0436] - Fixed target preserving beamformer (a) (separate estimation in target presence / absence (or based on fixed target covariance));

[0437] - Fixed target cancellation beamformer (B) (estimated separately in target presence / absence (or based on fixed target covariance)).

[0438] At least one beamformer may be implemented as a GEV beamformer, which may be configured based on a target covariance matrix (R T ) and the noise covariance matrix (R V ). However, the SNR can also be expressed in terms of fixed beamformers with different estimates in the GSC structure (where the covariance matrix does not need to be specifically estimated). In other words, instead of estimating the covariance matrix, the mean is estimated as the cross product between the target preserving beamformer and the target cancellation beamformer (estimated during the presence / absence of the target / noise).

[0439] Aspect 9 (GSC beamformer with variable number of active input signals)

[0440] State-of-the-art hearing devices may comprise more than two microphones, for example more than three. In general, it may be argued that the (usually scarce) battery capacity is not well spent on powering all microphones of the hearing aid in all situations. Beamforming based on all (e.g. three) microphones may be applied only in the most complex acoustic environments. Which microphones are switched may depend on the microphone configuration in the hearing device. In the general case where the user focuses his attention on one sound source in the environment, it may be desirable to switch towards two microphones, the axes of which are most parallel to the front-to-back axis. In the self-voice pickup (telephone) operating mode of the hearing aid, a different two-microphone configuration may be selected. In this case, we need a different steering vector d, and therefore a different target-keeping beamformer a and a different target-cancelling beamformer B. In the case where one of the microphones is detected to be inoperative, it may be preferred to select the two microphones that are still functional.

[0441] Ninth hearing aid

[0442] In a ninth aspect of the present application, a hearing aid suitable for being worn by a user is provided. The ninth hearing aid comprises:

[0443] - comprising a plurality of electrical input signals (X) adapted to pick up sound from the user's environment and provide corresponding m , m=1,…,M) a microphone system of a plurality of (M) microphones, wherein M is greater than or equal to 3;

[0444] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising a generalized sidelobe canceller beamformer (GSC) configured to provide at least one beamformed signal (Y), wherein the generalized sidelobe canceller comprises M fixed beamformers, one of the M fixed beamformers is a target signal preserving beamformer (a), M-1 fixed beamformers are target cancellation beamformers (b m-1 , m=2,…,M), each beamformer is configured to be based on an associated fixed beamformer weight (w F,m-1 ,m=1,…,M) generates the intermediate beamforming signal (C m-1 , m=1,…,M), wherein at least one beamforming signal (Y) is based on M intermediate beamforming signals (C m-1 ,m=1,…,M)determine;

[0445] wherein the M fixed beamformers are configured to receive M electrical input signals as inputs in the first operation mode of the directional system, and each fixed beamformer is configured to receive M electrical input signals according to the M electrical input signals (X m ,m=1,…,M) and the corresponding fixed beamformer weights (w F,m-1 , m=1,…,M) provides the corresponding intermediate beamforming signal (C m-1 , m=1,…,M); and

[0446] wherein the M-1 target cancellation beamformers are configured in the second operating mode of the directional system to receive as input a subset of the M electrical input signals, each target cancellation beamformer aligning the M electrical input signals (X m , m=1,…,M) and the corresponding fixed beamformer weights (w' F,m-1 ,m=2,…,M) provides the corresponding intermediate beamforming signal (C m-1 ,m=2,…,M).

[0447] In the case of implementing a switch from two microphones to one microphone, the GSC structure can be omitted. Otherwise, the GSC structure can be used.

[0448] The target cancellation beamformer and the target cancellation beamformer do not have to be fixed. The beamformer weights (a and B) can be adaptively updated to adjust towards the target direction.

[0449] Preferably, the system includes M-1 targeted cancellation beamformers. However, less than M-1 targeted cancellation beamformers may be used (so that we optimize for less than M-1 microphones).

[0450] The number M of microphones may be equal to three.

[0451] There are different options for forming two independent target cancellation beamformers. One option is to use a ref * / d H d)) selects M-1 columns from the M columns of the matrix given by . In this case, each target cancellation beamformer becomes a linear combination of all three microphone signals (for M=3, see e.g. Figure 8C ).

[0452] In another alternative (see, for example, Figure 8C , for M = 3), two independent first-order target cancellation beamformers can be generated. For example:

[0453] - a targeted cancellation beamformer based on microphone #1 and microphone #2;

[0454] - Another targeted cancellation beamformer based on microphone #1 and microphone #3.

[0455] The resulting weights become dependent on all three microphones, and we obtain the same resulting gain from this linear combination when each target cancellation beamformer is based on all three microphones.

[0456] In the second operating mode, the target maintains the electrical input signal (X m ) can, for example, be equal to M-1.

[0457] The advantage of the second approach (for) is that we can for example fade out microphone 3 simply by switching off the targeted cancellation beamformer (C2) based on microphone 1 (M1) and microphone 3 (M3). The targeted cancellation beamformer (C1) based on microphone 1 (M1) and microphone 2 (M2) does not need to be changed depending on whether the resulting beamformer relies on 2 or 3 microphones.

[0458] However, the target cancellation beamformer C0 needs to be scaled differently when it is based on a different number of microphones. In the case of three microphones (M=3), the weight a of the target preserving beamformer is given by dd1* / (|d1| 2 +|d2| 2 +|d3| 2 ), where microphone 1 (M1) is the reference microphone. In the case of two microphones (M=2), it is given by d d1* / (|d1| 2 +|d2|2 ). The difference in ratio between three microphones and two microphones is thus (|d1| 2 +|d2| 2 +|d3| 2 ) / (|d1| 2 +|d2| 2 ), which is about 1.5. However, according to the fading direction, the correct proportional difference should be (|d1| 2 +|d2| 2 +|d3| 2 ) / (|d1| 2 +|d2| 2 ) or (|d1| 2 +|d2| 2 ) / (|d1| 2 +|d2| 2 +|d3| 2 ).

[0459] In the second operating mode, the target maintains the number M of electrical input signals received by the beamformer (a). sub,0 For M.

[0460] In the second operation mode, the M-1 target elimination beamformers (b m-1 ,m=2,…,M) receives M electrical input signals (X m ,m=1,…,M) subset (SS m-1 , m=2,…,M) may include M electrical input signals (X m ,m=1,…,M) where M-1≥M sub,m-1 ≥2.

[0461] Given a subset (SS m-1 ,m=2,…,M) of the electrical input signal (X m ) sub,m-1 ) For all M-1 targets, eliminate the beamformer (b m-1 ,m=2,…,M) can be equal.

[0462] At least two of the subsets may include different numbers of electrical input signals.

[0463] At least two of the subsets may have at least one electrical input signal that is different from each other.

[0464] For M-1 targets, the beamformer (C m-1 , m=2,…,M), at least one of them, for example most or all of them, the electrical input signal (X m ) sub,m-1 ) can be equal to 2, for example.

[0465] When switching between the first and second operating modes, an immediate change between using two and three microphones may be applied. However, to minimize sudden sound effects that may be annoying to the user, a gradual change between the two modes may be applied.

[0466] The hearing aid may be configured to switch (eg fade) between the first and second operating modes of the directional system in dependence on the mode selection signal.

[0467] In a directional system including three microphones, the target-holding beamformer may apply a weight of approximately 1 / 3 to each microphone signal. In a directional system including two microphones, the target-holding beamformer may apply a weight of approximately 1 / 2 to each microphone signal. The simplest thing to do when fading is to apply the proportion to each weight while fading, i.e., when fading from three microphones to two microphones. An example of this is Figure 8C Shown in.

[0468] The hearing aid may include a classifier of the user's current acoustic environment. The classifier provides a classification signal representing the current acoustic environment. The mode selection signal may be determined by (or influenced by) the classification signal.

[0469] The mode selection signal may be determined (or influenced) by the detected sound scene complexity, for example. Switching (fading) the inputs to the target cancellation beamformer from M-1 to M (e.g. from two to three) may for example be initiated by detecting a more complex acoustic environment, or vice versa.

[0470] The hearing aid may be configured to switch (e.g. gradually) between the first and second operating modes (e.g. the target cancellation beamformer inputs are changed from M to M-1 (e.g. from three to two)) to save energy. The hearing aid may include a detector of the current state of the energy source, for example providing an estimate of the current remaining capacity of the hearing aid's battery or the current power consumption of the hearing aid.

[0471] The mode selection signal may be a function of, for example, the input level of the electrical input signal, SNR, remaining battery time, user's motion, etc.

[0472] Standard features applicable to all aspects of the invention

[0473] A hearing aid may include multiple (M≥2) microphones.

[0474] The hearing aid may include a directional noise reduction system, comprising a generalized sidelobe canceller (GSC) beamformer configured to provide at least one beamformed signal, wherein the generalized sidelobe canceller includes M fixed beamformers, one of the M fixed beamformers is a target signal preserving beamformer (a), and M-1 fixed beamformers are target cancellation beamformers (b). m-1,m=2,…,M).

[0475] At least M-1 (eg, two) targeted cancellation beamformers may be based on a subset of the electrical input signals.

[0476] The microphone system may consist of M=3 microphones. In a first operating mode, the two targeted cancellation beamformers may be based on three electrical input signals. In a second operating mode, the two targeted cancellation beamformers may be based on two electrical input signals. The hearing aid may be adapted to switch (e.g. gradually) between the first and second operating modes (e.g. based on a classifier of the acoustic environment, an estimate of the current remaining capacity of a hearing aid battery, or a current power consumption of the hearing aid).

[0477] The directional noise reduction system may, for example, include a generalized eigenvector beamformer (GEV) scheme for two, three or more input transducers (e.g., microphones), and in particular include an update rule for estimating adaptive parameters (β) based on averaging across three (or more) fixed beamformers (rather than estimating a noise covariance matrix). The directional noise reduction system may, for example, be configured to include a gradual selection between beamformer weights for two and three input transducers (e.g., microphones).

[0478] A generalized eigenvector beamformer (GEV) (or its approximation) may, for example, include one target preserving beamformer and M-1 target canceling beamformers.

[0479] In the power-saving mode of operation, the M-1 target cancellation beamformers may receive as input only a subset of the M electrical input signals (see, for example, Figure 8C , Fig. 10C ).

[0480] The subset of the M electrical input signals may include two electrical input signals (whereby the M-1 target cancellation beamformers are first order beamformers).

[0481] Each of the M-1 targeted cancellation beamformers may take as input only a subset of the M microphone signals, e.g., such that each targeted cancellation beamformer has an input comprising a different subset of the M input signals (e.g., a subset consisting of two input signals, such that each targeted cancellation beamformer becomes a first-order beamformer).

[0482] In combination with any of the preceding aspects or in another separate aspect, the hearing aid may be configured such that the first and second target cancellation beamformers for the three-microphone input scheme are based on a subset of the three microphone inputs, wherein the beamformer is configured, for example in a particular operating mode, to fade the beamformer weights of the first and second target cancellation beamformers between multiple sets of beamformer weights optimized for two and three microphones, respectively.

[0483] The main advantage of using a two-microphone beamformer in a three-microphone system is that switching between a two-microphone system and a three-microphone system becomes easier because the target cancellation beamformer can be reused.

[0484] The trigger for entering a specific operating mode (changing from a three-input to a two-input targeted cancellation beamformer) may for example be related to energy saving. The trigger may for example be based on the input level or another parameter, such as provided by a sound scene detector.

[0485] The complexity of the sound scene can trigger a switch from two to three microphones. The trigger can be a function of level, SNR, remaining battery time, motion, for example.

[0486] In general, it may be claimed that battery capacity is not well spent on powering three microphones in all situations. We may apply beamforming based on three microphones only in the most complex environments. Which microphones are switched may depend on the microphone configuration in the hearing device. In the general situation where the user focuses his attention on (one) sound source in the environment, it may be desirable to switch towards two microphones, the axes of which are most parallel to the front-rear axis. In the self-voice pickup (telephone) operating mode of the hearing aid, different two-microphone configurations may be selected. Furthermore, in the situation where one of the microphones is detected to be inoperative, it may be preferred to select the two microphones that are still functional.

[0487] The hearing aid may be adapted to provide frequency dependent gain and / or level dependent compression and / or frequency transposition of one or more frequency ranges to one or more other frequency ranges (with or without frequency compression) to compensate for the hearing impairment of the user. The hearing aid may include a signal processor for enhancing an input signal and providing a processed output signal.

[0488] The hearing aid may comprise an output unit for providing a stimulus perceived by the user as an acoustic signal based on the processed electrical signal. The output unit may comprise a plurality of electrodes of a cochlear implant (for a CI type hearing aid) or a vibrator of a bone conduction hearing aid. The output unit may comprise an output transducer. The output transducer may comprise a receiver (speaker) for providing the stimulus as an acoustic signal to the user (e.g. in an acoustic (air conduction based) hearing aid). The output transducer may comprise a vibrator for providing the stimulus as a mechanical vibration of the skull to the user (e.g. in a bone attached or bone anchored hearing aid). The output unit may (in addition or as an alternative) comprise a transmitter for transmitting the sound picked up by the hearing aid (e.g. via a network, e.g. in a telephone operating mode, or in a headset configuration) to another device such as a remote communication partner.

[0489] The hearing aid may comprise an input unit for providing an electrical input signal representing a sound. The input unit may comprise an input transducer such as a microphone for converting the input sound into the electrical input signal. The input unit may comprise a wireless receiver for receiving a wireless signal comprising or representing a sound and providing the electrical input signal representing the sound.

[0490] The wireless receiver and / or transmitter may be configured to receive and / or transmit electromagnetic signals in the radio frequency range (3kHz to 300GHz), for example. The wireless receiver and / or transmitter may be configured to receive and / or transmit electromagnetic signals in the optical frequency range (e.g., infrared light 300GHz to 430THz or visible light such as 430THz to 770THz), for example.

[0491] A hearing aid may include a directional microphone system adapted to spatially filter sounds from the environment so as to enhance a target sound source among a plurality of sound sources in the local environment of a user wearing the hearing aid. The directional system may be adapted to detect (e.g., adaptively detect) from which direction a particular portion of the microphone signal originates. This may be implemented in a variety of different ways, such as described in the prior art. In hearing aids, microphone array beamformers are typically used to spatially attenuate background noise sources. The beamformer may include a linear constrained minimum variance (LCMV) beamformer. Many beamformer variants can be found in the literature. Minimum variance distortionless response (MVDR) beamformers are widely used in microphone array signal processing. Ideally, the MVDR beamformer keeps the signal from the target direction (also called the line of sight) unchanged, while maximally attenuating sound signals from other directions. The generalized sidelobe canceler (GSC) structure is an equivalent representation of the MVDR beamformer, which provides computational and digital representation advantages over direct implementation in its original form. The present application also relates to a generalized eigenvector beamformer, also called a "generalized eigenvalue beamformer", abbreviated in both cases as GEV.

[0492] The hearing aid may comprise an antenna and a transceiver circuit which enables a wireless link to be established to an entertainment device (e.g. a television), a communication device (e.g. a telephone), a wireless microphone or another hearing aid or the like. The hearing aid may thus be configured to wirelessly receive a direct electrical input signal from another device. Similarly, the hearing aid may be configured to wirelessly transmit a direct electrical output signal to another device. The direct electrical input or output signal may represent or comprise an audio signal and / or a control signal and / or an information signal.

[0493] In general, the wireless link established by the antenna and the transceiver circuit of the hearing aid may be of any type. The wireless link may be a link based on near field communication, such as an inductive link based on inductive coupling between antenna coils of a transmitter part and a receiver part. The wireless link may be based on far field electromagnetic radiation. Preferably, the frequency used to establish the communication link between the hearing aid and the other device is lower than 70 GHz, such as in the range from 50 MHz to 70 GHz, such as higher than 300 MHz, such as in the ISM range higher than 300 MHz, such as in the 900 MHz range or in the 2.4 GHz range or in the 5.8 GHz range or in the 60 GHz range (ISM = Industrial, Scientific and Medical, such standardized ranges are defined, for example, by the International Telecommunication Union ITU). The wireless link may be based on standardized or dedicated technologies. The wireless link may be based on Bluetooth technology (such as low power Bluetooth technology, such as LE Audio) or ultra-wideband (UWB) technology.

[0494] The hearing aid may be or may form part of a portable (i.e. configured to be wearable) device, such as a device comprising a local energy source such as a battery, such as a rechargeable battery. The hearing aid may for example be a low weight, easily wearable device, such as having a total weight of less than 100g, such as less than 20g, such as less than 5g.

[0495] The hearing aid may include a "forward" (or "signal") path between the input and output of the hearing aid for processing audio signals. A signal processor may be located in the forward path. The signal processor may be adapted to provide a frequency-dependent gain according to the specific needs of the user (e.g. hearing loss). The hearing aid may include an "analysis" path having functional parts for analyzing signals and / or controlling processing of the forward path. Part or all of the signal processing in the analysis path and / or the forward path may be performed in the frequency domain, in which case the hearing aid includes appropriate analysis and synthesis filter banks. Part or all of the signal processing in the analysis path and / or the forward path may be performed in the time domain.

[0496] The hearing aid, such as the input unit and / or the antenna and transceiver circuitry, may comprise a transform unit for transforming a time domain signal into a signal in a transform domain (e.g. frequency domain or Laplace domain, Z transform, wavelet transform, etc.). The transform unit may consist of or include a time-frequency (TF) transform unit for providing a time-frequency representation of the input signal. The time-frequency representation may comprise an array or mapping of corresponding complex or real values ​​of the signal concerned in a specific time and frequency range. The TF transform unit may comprise a filter bank for filtering the (time-varying) input signal and providing a plurality of (time-varying) output signals, each output signal comprising a distinct frequency range of the input signal. The TF transform unit may comprise a Fourier transform unit (e.g. a discrete Fourier transform (DFT) algorithm, a short-time Fourier transform (STFT) algorithm, or a similar algorithm) for transforming the time-varying input signal into a (time-varying) signal in the (time-)frequency domain. The minimum frequency f is taken into account by the hearing aid. min To the maximum frequency f max The frequency range of may include a portion of the typical human hearing range from 20 Hz to 20 kHz, for example a portion of the range from 20 Hz to 12 kHz. Typically, the sampling rate f s Greater than or equal to the maximum frequency f max twice, that is, f s ≥2f max The signals of the forward path and / or analysis path of the hearing aid may be split into NI frequency bands (e.g. of uniform width), wherein NI is, for example, greater than 5, such as greater than 10, such as greater than 50, such as greater than 100, such as greater than 500, at least parts of which are processed separately. The hearing aid may be adapted to process the signals of the forward and / or analysis path in NP different frequency channels (NP≤NI). The frequency channels may be of uniform or non-uniform width (e.g. the width increases with frequency), overlapping or non-overlapping.

[0497] The hearing aid may be configured to operate in different modes, such as a normal mode and one or more specific modes, which may be selected by a user or may be automatically selected, for example. The operating mode may be optimized for a specific acoustic situation or environment, such as a communication mode, e.g., a telephone mode. The operating mode may include a low power mode, in which the functionality of the hearing aid is reduced (e.g., to save energy), such as disabling wireless communication and / or disabling specific features of the hearing aid (e.g., reducing the number of microphones actively used).

[0498] The hearing aid may include a plurality of detectors configured to provide status signals related to the current network environment of the hearing aid (e.g., the current acoustic environment), and / or to the current state of a user wearing the hearing aid, and / or to the current state or operating mode of the hearing aid. Alternatively or additionally, one or more of the detectors may form part of an external device that communicates with the hearing aid (e.g., wirelessly). The external device may, for example, include another hearing aid, a remote control, an audio transmission device, a phone (e.g., a smartphone), an external sensor, etc.

[0499] One or more of the plurality of detectors may operate on a full-band signal (time domain). One or more of the plurality of detectors may operate on a band-split signal ((time-)frequency domain), for example in a limited number of frequency bands.

[0500] The plurality of detectors may include a level detector for estimating the current level of the signal of the forward path. The detector may be configured to determine whether the current level of the signal of the forward path is above or below a given (L-)threshold. The level detector acts on the full-band signal (time domain). The level detector acts on the band-split signal ((time-)frequency domain).

[0501] The hearing aid may comprise a voice activity detector (VAD) for estimating whether (or with what probability) an input signal (at a given point in time) comprises a voice signal. In the present specification, a voice signal may be taken to mean a signal comprising speech from a human being. It may also include other forms of vocalizations (such as singing) produced by a human speech system. The voice activity detector unit may be adapted to classify the user's current acoustic environment as a "voice" or a "no-voice" environment. This has the following advantage: time periods comprising an electric microphone signal of human vocalizations (such as speech) in the user's environment may be identified and thus separated from time periods comprising only (or mainly) other sound sources (such as artificially produced noise). The voice activity detector may be adapted to detect the user's own voice as "voice" as well. As an alternative, the voice activity detector may be adapted to exclude the user's own voice from the detection of "voice".

[0502] The hearing aid may include a self-voice detector for estimating whether (or with what probability) a particular input sound (e.g. voice, such as speech) originates from the voice of a user of the hearing system. The microphone system of the hearing aid may be adapted to be able to distinguish the user's own voice from the voice of another person and possibly from unvoiced sounds.

[0503] The plurality of detectors may include a motion detector, such as an accelerometer. The motion detector may be configured to detect movement of the user's facial muscles and / or bones, such as due to speech or chewing (eg, jaw movement), and provide a detector signal indicative of the movement.

[0504] The hearing aid may comprise a classification unit configured to classify the current situation based on an input signal from (at least part of) the detector and possibly other inputs. In this specification, a "current situation" may be defined by one or more of the following:

[0505] a) the physical environment (e.g. including the current electromagnetic environment, such as the presence of electromagnetic signals (including audio and / or control signals) intended or not intended to be received by the hearing aid, or other properties of the current environment other than acoustics);

[0506] b) Current acoustic conditions (input level, feedback, etc.);

[0507] c) the user’s current mode or state (motion, temperature, cognitive load, etc.);

[0508] d) The current mode or status of the hearing aid and / or another device communicating with the hearing aid (selected program, time elapsed since last user interaction, etc.).

[0509] The classification unit may be based on or may comprise a neural network, such as a recurrent neural network, such as a trained neural network.

[0510] Hearing aids may include acoustic (and / or mechanical) feedback control (e.g., suppression) or an echo cancellation system. Adaptive feedback cancellation has the ability to track changes in the feedback path over time. It is usually based on a linear time-invariant filter to estimate the feedback path, but the filter weights are updated over time. The filter updates can be calculated using a stochastic gradient algorithm, including some form of the least mean square (LMS) or normalized LMS (NLMS) algorithm. They all have the property of minimizing the error signal in terms of mean square, with NLMS additionally normalizing the filter updates to the square of the Euclidean norm of a reference signal.

[0511] The hearing aid may also include other appropriate functions for the application in question, such as compression, noise reduction, etc.

[0512] A hearing aid may comprise a hearing instrument, such as a hearing instrument adapted to be located at the user's ear or fully or partially in the ear canal, a headset, an earphone, an ear protection device or a combination thereof. A hearing system may comprise a loudspeaker amplifier (comprising a plurality of input transducers (e.g. a microphone array) and a plurality of output transducers such as one or more loudspeakers, and one or more audio (and possibly video) transmitters, such as for use in an audio conferencing situation), such as comprising a beamformer filter unit, such as to provide a plurality of beamforming capabilities.

[0513] application

[0514] In one aspect, there is provided an application of a hearing aid as described above, in detail in the "Detailed Description" section and in the claims. The application may be provided in a system including one or more hearing aids (such as hearing instruments), headphones, headsets, active ear protection systems, etc., such as a hands-free telephone system, a teleconferencing system (such as including a speaker amplifier), a broadcasting system, a karaoke system, a classroom amplification system, etc.

[0515] The method according to the first aspect

[0516] In one aspect, the present invention provides a method of operating a hearing aid adapted to be worn by a user. The hearing aid comprises: a microphone system comprising a plurality of (M) microphones adapted to pick up sounds from the user's environment and provide corresponding electrical input signals, wherein M is greater than or equal to 2. The method comprises:

[0517] - generating at least one beamformed signal according to beamformer weights configured to be applied to the plurality of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality of electrical input signals; and

[0518] - adaptively optimizing the beamformer weights for multiple target locations by maximizing a target signal-to-noise ratio of sound from the multiple target locations;

[0519] - determining the signal-to-noise ratio based on first and second output variances of the at least one beamformer or time-averaged versions of the first and second output variances, the first and second output variances being determined when the electrical input signal or the at least one beamformed signal is labeled as target and noise, respectively.

[0520] When appropriately replaced by corresponding processes, part or all of the structural features of the apparatus described above, described in detail in the "Detailed Description" or defined in the claims can be combined with the implementation of the method of the present invention, and vice versa. The implementation of the method has the same advantages as the corresponding apparatus.

[0521] The method according to the second aspect

[0522] In one aspect, a method of operating a hearing aid adapted to be worn by a user is provided. The hearing aid comprises: a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m A microphone system of a plurality of (M) microphones having (n), m=1, ..., M, wherein n represents time and M is greater than or equal to 2. The method comprises:

[0523] - generating at least one beamformed signal based on beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals;

[0524] - adaptively optimizing the beamformer weights (w) for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sound from the plurality of target locations (θ);

[0525] - wherein the signal-to-noise ratio is based on the beamformer weights (w), the inter-microphone target covariance matrix (R T ) and the inter-microphone noise covariance matrix (R V ) is represented by the target covariance matrix (R T ) according to the multiple steering vectors (d θ )Sure.

[0526] When appropriately replaced by corresponding processes, part or all of the structural features of the apparatus described above, described in detail in the "Detailed Description" or defined in the claims can be combined with the implementation of the method of the present invention, and vice versa. The implementation of the method has the same advantages as the corresponding apparatus.

[0527] The beamformer weights of the multiple target positions (θ) of the multiple target sound sources currently existing may be calculated based on the multiple steering vectors (d θ ) to confirm.

[0528] The method according to the third to ninth aspects

[0529] The method according to the third to ninth aspects is provided by replacing structural features of the hearing aid according to the third to ninth aspects with equivalent method features.

[0530] When appropriately replaced by corresponding processes, some or all of the structural features of the hearing aid device according to the third to ninth aspects described above, described in detail in the "Detailed Description of the Invention" or defined in the claims may be combined with the implementation of the method according to the third to ninth aspects (and vice versa). The implementation of the method has the same advantages as the corresponding device.

[0531] Computer Programs

[0532] In addition, the present application provides a computer program (product) comprising instructions, which, when executed by a computer, causes the computer to execute the method (steps) described above (for example, according to any one of the first to ninth aspects), described in detail in the "Specific Implementation Methods" and defined in the claims.

[0533] Hearing System

[0534] In another aspect, there is provided a hearing aid as described above (eg according to any one of the first to ninth aspects), described in detail in the "Detailed Description of the Invention" and defined in the claims, and a hearing system including an auxiliary device.

[0535] The hearing system may be adapted to establish a communication link between the hearing aid and the auxiliary device so that information (eg control and status signals, possibly audio signals) can be exchanged or forwarded from one device to the other.

[0536] The auxiliary device may include or may consist of a remote control, a smart phone, or other portable or wearable electronic device such as a smart watch or the like.

[0537] The auxiliary device may consist of or comprise a remote control for controlling the functions and operation of the hearing aid. The functions of the remote control are implemented in a smartphone, which may run an APP enabling control of the functions of the audio processing device via the smartphone (the hearing aid comprises a suitable wireless interface to the smartphone, e.g. based on Bluetooth or some other standardized or proprietary solution).

[0538] The auxiliary device may be constituted by or include an audio gateway device, which is suitable for receiving multiple audio signals (e.g. from an entertainment device such as a TV or music player, from a telephone device such as a mobile phone, or from a computer such as a PC, a wireless microphone, etc.) and is suitable for selecting and / or combining appropriate signals (or signal combinations) from the received audio signals for transmission to the hearing aid.

[0539] The auxiliary device may consist of or may comprise a further hearing aid.The hearing system may comprise two hearing aids adapted to implement a binaural hearing system, eg a binaural hearing aid system.

[0540] APP

[0541] On the other hand, the present invention also provides a non-transient application called APP. The APP includes executable instructions configured to run on an auxiliary device to implement a user interface for the hearing aid or hearing system described above, described in detail in the "Detailed Description of the Invention" and defined in the claims. The APP can be configured to run on a mobile phone such as a smart phone or another portable device that enables communication with the hearing aid or hearing system.

[0542] Embodiments of the present invention may be used, for example, in applications such as hearing aids, headphones, headsets, or combinations thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0543] Various aspects of the present invention will be best understood from the detailed description below in conjunction with the accompanying drawings. For clarity, the drawings are schematic and simplified, and only the details necessary for understanding the present invention are given, while other details are omitted. Throughout the specification, the same reference numerals are used for the same or corresponding parts. The various features of each aspect may be combined with any or all features of the other aspects. These and other aspects, features and / or technical effects will be apparent from and illustrated in conjunction with the following figures, in which:

[0544] Figure 1A shows that a given sound source at position θ (eg relative to the hearing device or the user's head) has for a given frequency band a signal represented by h m (θ) is the transfer function to the mth microphone;

[0545] Figure 1B Schematically shows a hearing aid including a directional system according to an embodiment of the present invention;

[0546] Figure 1C Schematically shows a hearing device comprising a beamformer (BF) according to an embodiment of the present invention, wherein the beamformer comprises a beamformer configured to determine (optimize or approximate) beamformer weights (w = [w1, ..., w M ] T )’s weight optimization module (w);

[0547] Figure 2 A first embodiment of an input stage of a hearing device according to the invention is shown;

[0548] Figure 3 A second embodiment of an input stage of a hearing device according to the invention is shown;

[0549] Figure 4A Two thresholds κ for determining the time and frequency regions are shown F and κ B , where the target covariance matrix R is updated or not T To update the noise covariance matrix R v ;

[0550] Figure 4B schematically shows an implementation of an input stage of a hearing device according to the invention comprising a FBR optimized beamformer and subsequent comparison of the contents of the front and rear beamformers used as input to a neural network to provide a decision to label a target or noise;

[0551] Figure 4Cschematically shows an implementation of an input stage of a hearing device according to the present invention comprising an FBR optimized beamformer as input to a neural network comprising a comparison function based on the FBR optimized beamformer to provide a decision to label a target or noise;

[0552] Figure 5 The optimization of the beamformer based on different criteria is shown;

[0553] Figure 6 schematically shows how the most likely target direction (or target steering vector) can be selected among a dictionary of different candidate directions;

[0554] Figure 7 The optimization of the beamformer based on different criteria is shown;

[0555] Fig. 8A A schematic diagram of an implementation of a directional noise reduction system according to the present invention, which includes a three-microphone generalized sidelobe canceller structure;

[0556] Figure 8B Shows Fig. 8A Schematic diagram of an implementation of the directional noise reduction system of , but where instead of generating two independent target cancellation beamformers based on all three microphones, two independent first-order target cancellation beamformers may be generated;

[0557] Figure 8C Shows Figure 8B Schematic diagram of an implementation of a directional noise reduction system in which the effect of a gradual change from three to two or from two to three microphones as input to a target cancellation beamformer on the beamformer weights is indicated;

[0558] Fig. 9 Schematically shows the surface of the loss function L as a function of the real and imaginary parts of the adaptation parameter β;

[0559] Fig. 10A A first example of a two-microphone GEV beamformer with gain normalization towards the front direction is shown;

[0560] Fig. 10B A second example of a two-microphone GEV beamformer with gain normalization towards the front direction is shown;

[0561] Fig. 10C An example of a three-microphone GEV beamformer is shown, where the adaptive parameters are updated based on the Sign-LMS algorithm;

[0562] Fig.11 An embodiment of a hearing aid according to the present invention is schematically shown;

[0563] Fig.12An embodiment of a hearing aid according to the present invention is schematically shown;

[0564] Fig.13 An implementation of a speech detector using a neural network for a hearing aid according to the invention is schematically shown.

[0565] By the detailed description given below, the further scope of application of the present invention will be apparent. However, it should be understood that while the detailed description and specific examples show the preferred embodiments of the present invention, they are only provided for illustrative purposes. For those skilled in the art, based on the following detailed description, other embodiments of the present invention will be apparent. DETAILED DESCRIPTION

[0566] The detailed description proposed below in conjunction with the accompanying drawings serves as a description of a variety of different configurations. The detailed description includes specific details for providing a thorough understanding of a number of different concepts. However, it is apparent to those skilled in the art that these concepts can be implemented without these specific details. Several aspects of the apparatus and method are described by a number of different blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). Depending on the specific application, design limitations or other reasons, these elements can be implemented using electronic hardware, computer programs or any combination thereof.

[0567] The electronic hardware may include microelectromechanical systems (MEMS), (e.g., application specific) integrated circuits, microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gating logic, discrete hardware circuits, printed circuit boards (PCBs) (e.g., flexible PCBs), and other suitable hardware configured to perform a number of different functions described in this specification, such as sensors for sensing and / or recording physical properties of the environment, device, user, etc. Computer programs shall be broadly construed as instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, execution threads, programs, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0568] The present application relates to the field of hearing devices such as hearing aids or headphones, and in particular to noise reduction in the aforementioned devices.

[0569] Figure 1A shows that a given sound source at position θ has a given frequency band (k) represented by h m The (absolute) (acoustic) transfer function to the mth microphone is given by (θ). The reference microphone (M1) and the mth microphone (M m ) relative (acoustic) transfer function d m(θ) is thus determined by d m (θ) = h m (θ) / h ref (θ), m = 1, ..., M. The relative transfer function from a given target position θ to the microphone array can thus be given by the vector d(θ) = h(θ) / h ref The vector h(θ) represents the absolute acoustic transfer function (h) of the sound from the sound source at position θ to the M microphones of a microphone array (e.g., a hearing device). m (θ),m=1,…,M). Figure 1A In the microphone (M1,…,M M ) are shown as lying on a common line (microphone axis). However, this need not be the case. The steering (or look) vector d is the same as the relative transfer function between the microphones. It can be calibrated (updated) in the absence of background noise, or alternatively in a low noise environment based on sounds played from the target direction.

[0570] For a minimum variance distortionless response (MVDR) beamformer, a set of beamformer weights that maximizes the SNR at the output can be given by the following ratio:

[0571]

[0572] in, is the target covariance, is the noise covariance, and Among them, Γ V is the normalized correlation matrix; R V / TRACE(R V ), where the TRACE function extracts the matrix (here R V ) and add them together. The SNR at the reference microphone is then given by In the specific case where the target signal is a single point source, the target covariance matrix is ​​given by We note that the target covariance matrix (R T ) is strange because it passes through d θ In this particular case, we have a closed-form solution given by:

[0573]

[0574] This is the well-known MVDR beamformer solution.

[0575] Assuming that the target exists only in a single direction may not always correspond to the direction the listener wants to hear. To address this problem, one strategy is to estimate the most likely target direction, or by determining the most likely turn vector d θ(See, for example, EP3413589A1), or by estimating the most likely direction of a point sound source that the user is currently interested in (see, for example, EP3300078A1).

[0576] We will now consider the more general case where the listener is interested in more than one direction, such as multiple speakers coming from a (separate) single direction, or any sound source in the front half of the image. In this case, we assume that the target covariance matrix has full rank, for example, the target covariance matrix can be the sum of different steering vectors, i.e.

[0577]

[0578] In this case, we need to determine a set of weights w that maximizes the following:

[0579]

[0580] It is well known that the solution w can be estimated as the eigenvector belonging to the largest generalized eigenvalue.

[0581] The weight w may also be estimated, for example, by iteratively updating w using gradient-based optimization.

[0582] The gradient is given by:

[0583]

[0584] Other gradient algorithms can be used. We can find the algorithm that maximizes the SNR and satisfies, for example However, we can only determine up to a certain ratio of w, so we must choose how to normalize the set of weights that maximizes the SNR, for example by having a unit response in a certain direction, that is, satisfying |w H d θ | 2 =1.

[0585] This type of beamforming is often referred to as generalized eigenvector (GEV) beamforming.

[0586] Figure 1B A hearing device (HD) such as a hearing aid suitable for being worn by a user is schematically shown. The hearing aid comprises a sensor adapted to pick up sounds from the user's environment and provide a corresponding electrical input signal (x=[x1, ..., x M ]) of M microphones (M1,…,M M ), where M is greater than or equal to 2. Each microphone path comprises an analysis filter bank FBA, for example comprising a filter for converting a time domain input signal (x m ,m=1,…,M) into a sub-band signal (X m, m=1,…,M) of Fourier transform algorithm (such as DFT or STFT, etc.), where k and l are frequency index and time index respectively. The hearing aid also includes a directional noise reduction system connected to the microphone system.

[0587] The directional noise reduction system comprises at least one beamformer BF for arranging the beamformer to be applied to the M electrical input signals according to (usually complex) beamformer weights (w = [w1, ..., w M ]) generates at least one beamforming signal Y F , so that at least one beamforming signal Y F Provided as a weighted sum of M electrical input signals x, Y F =(w H To implement the weighted sum of the input signals, the beamformer comprises a corresponding combining unit (here a multiplication unit X) for combining the (usually complex) weights (w1, ..., w M ) is applied to the corresponding electrical input signal (X m , m=1,…,M) to provide the corresponding weighted input signal (w1X1,…,w M X M ). The weights can be complex conjugated before multiplying with the input signal, i.e., w H =[w1,…,w M ]. Figure 1B The embodiment of the beamformer further includes M input combining units (herein, summing units “+”) for combining the beamforming signal Y F Provided as a linear combination of M electrical input signals:

[0588] Y F =w1X1+,…,+w M X M

[0589] (See its output is the beamforming signal Y F M multiplication units (X) and summation units (+)).

[0590] The hearing device comprises a device for estimating the beamforming signal Y F The SNR estimator SNR-EST is connected to the beamformer BF or forms a part thereof (e.g. Figure 1B ) and is used in iteratively determining the optimal weights (w) for the beamformer (see Figure 1B The optimization module denoted as “modify w to maximize SNR” in .

[0591] Hearing devices, such as Figure 1BIn the directional system (shown in FIG. 1 ), for example, at least one beamformer BF comprises a voice activity detector VAD (for example, embodied as a speech detector). The voice activity detector VAD is configured to (continuously) estimate whether (binary) or with what probability an input (audio) signal comprises a voice signal such as speech. Figure 1B In the embodiment of the present invention, the voice activity detector VAD converts the beamforming signal Y F and / or optionally one or more electrical input signals (X) of a time-frequency representation (k, l) m , m=1,…,M)(see dashed arrows) are received as input (audio) signals, so that the resulting voice activity control signal VLAB can be based on one or more of the signals. The voice activity control signal VLAB can represent the probability of speech presence for the time-frequency unit (k,l), or can be a binary indication (e.g. "speech is dominant" or "speech is not dominant" (e.g. noise is dominant), or for example marked as target (T) and noise (N) respectively), i.e. in any case, VLAB=VLAB(k,l).

[0592] The hearing device HD may comprise (or have access to) a plurality of stored parameters related to optimizing the beamformer weights (w). The hearing device HD may comprise a memory storing the aforementioned parameters. These parameters may comprise predetermined values ​​of the aforementioned parameters, such as a steering vector (d ) for a plurality of positions (θ) θ )(or the target covariance matrix between microphones (R T )), and / or an inter-microphone noise covariance matrix R for a plurality of current acoustic environments (preferably related to the hearing device user) V .

[0593] The noise reduction system may include additional noise reduction modules, for example, for further reducing the spatially filtered (beamformed) signal Y F A single-channel postfilter for the noise in Figure 1B The module PF with a dashed box in FIG. 1 is obtained by extracting the post-filter from an SNR estimator (e.g., from SNR-EST, or from another SNR estimator such as SNR-PF and a corresponding VAD-PF, such as Figure 1B The received signal-to-noise ratio (indicated by the dotted box in the figure) is used to control the gain of the post-filter, thereby further reducing the spatially filtered (beamforming) signal Y F The noise component of Fig.12 .

[0594] exist Fig.12 (and Figure 1B ), the post filter PF is used to further improve the optimized output (Y) of the beamformer BF to provide a further noise-reduced signal (OUT, Y) by applying the post filter gain PFG to the beamformed signal (Y). NR). Alternatively, the post filter may be included in the beamformer weight optimization process, by optimizing the output of the post filter instead of the beamformer to maximize its SNR. The voice activity control signal may also be used as an input to the post filter, either to estimate the weights to optimize the SNR, or as an additional input including the SNR. Different SNR estimators may be applied to the beamformer and the post filter, respectively.

[0595] The beamformer weights (w) are adaptively optimized for multiple target locations (θ) by maximizing the target signal-to-noise ratio (SNR) of the sound from the multiple target locations (θ). The optimization process is schematically indicated by the module "Modify w to maximize SNR", which is Figure 1B In an embodiment of the invention, the beamformer signal Y is formed as part of at least one beamformer BF. The arrow from the SNR estimator (SNR-EST) (signal SNR) through the module "modify w to maximize SNR" symbolizes an adaptive optimization process, wherein the beamformer weights are determined in an adaptive process comprising iterative changes of the beamformer weights (w) (e.g. using an iterative optimization algorithm such as gradient ascent (e.g. steep ascent) or the like) and monitoring a cost function (e.g. MSE) of the change in SNR for a given change in weight. The process is arranged to optimize the beamformed signal Y F The SNR of the beamformer is optimized (maximized) and the weights are frozen at the time when the SNR is maximum. Examples of different expressions of the signal-to-noise ratio for optimizing the beamformer weights are given below.

[0596] The hearing device (e.g., a beamformer SNR estimation module (SNR-EST)) may be configured to adaptively estimate steering vectors (d ) for a plurality of target locations (θ) based on a voice activity control signal VLAB labeled as target (T) or noise (N). θ )(or the corresponding inter-microphone target covariance matrix R T ) and / or the inter-microphone noise covariance matrix R V , see the arrow from the voice activity detector VAD to the SNR estimation module (SNR-EST). The SNR estimation module (SNR-EST) may include a corresponding level detector and / or a smoothing unit (e.g., a low-pass filter) for smoothing the current estimate of the target and noise (e.g., in the beamforming signal and / or the electrical input signal, etc.) over time.

[0597] The hearing device may further comprise an audio signal processor (eg a hearing aid processor) SPRO for applying one or more processing algorithms to the beamformed (possibly further noise-reduced) signal (Y F ,Y NR ), such as adjusting the dynamic input level to suit a range of audible sound levels of the user and applying a corresponding frequency-dependent gain to the input signal (e.g. Figure 1BThe beamforming signal (Y F ) (or optionally, a further noise-reduced signal Y NR Other audio signal processing algorithms may be applied in the audio signal processor SPRO, such as algorithms related to binaural aspects of hearing, feedback control and / or echo cancellation, etc. The audio signal processor SPRO provides a processed output signal OUT in the time-frequency domain.

[0598] The hearing device may further comprise a synthesis filterbank FBS configured to transform the signal OUT in the time-frequency domain into a signal out in the time domain.

[0599] The hearing device further comprises an output transducer (here a loudspeaker SPK) for converting the processed output signal out into a stimulus perceivable as sound to the user. Alternatively or additionally, the output transducer (e.g. in a "headphone application") may comprise an audio transmitter for transmitting the processed signal out to another device or system. The output transducer may, for example, comprise a vibrator of a bone conduction hearing aid. The output transducer may, for example, comprise an electrode array of a cochlear implant hearing aid (in which case the synthesis filter bank may be omitted).

[0600] The hearing aid may comprise further functional units, such as one or more detectors, e.g. a classifier of the current acoustic environment around the user, which may be used to improve the quality of the stimuli presented to the user, e.g. to increase the listening comfort of the user and / or to increase the intelligibility of speech content in an audio signal picked up or received by the hearing device.

[0601] Different expressions of the signal-to-noise ratio for optimizing the beamformer weights are presented below.

[0602] Example 1

[0603] In an embodiment according to the present invention, the beamformer weights (w) are adaptively optimized for the plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of the sound from the plurality of target locations (θ), wherein the target signal-to-noise ratio (SNR) is based on at least one beamformer (e.g., at least one beamformed signal (Y F ), or the (further) noise-reduced signal from the post-filter PF (Y NR ))'s first and second output variance (|Y T 2 ,|Y V | 2 )(or its time-averaged version <|Y T | 2 >,<|Y V | 2 >) to express.

[0604]

[0605] The first and second output variance (|Y T 2 ,|Y V | 2 )(or its time-averaged version <|Y T | 2 >,<|Y V | 2 >) in the electrical input signal (x) or at least one beamforming signal Y F ( Figure 1B Y NR ) are marked as target (T) and noise (V) respectively. The first and second output variances can be determined using a voice activity detector VAD, see Figure 1B The voice activity control signal VLAB.

[0606] In an embodiment according to the present invention, the beamformer weights (w) are adaptively optimized for the multiple target locations (θ) by maximizing the target signal-to-noise ratio (SNR) of the sound from the multiple target locations (θ), wherein the SNR is based on the beamformer weights (w) and the multiple electrical input signals X and X H The time-averaged inner vector product <XX H > T and <XX H > V Determine, where <.> refers to the average over time.

[0607]

[0608] Time-averaged inner vector product <XX H > T and <XX H > V Determined when the electrical input signal (X) is labeled as target (T) and noise (V). H Equal to X1X1*+…+.X M X M *, where * refers to complex conjugation.

[0609] In an embodiment according to the present invention, the beamformer weights (w) are adaptively optimized for multiple target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of the sound from the multiple target locations (θ), wherein the SNR is determined based on the beamformer weights (w) and the corresponding inter-microphone target covariance matrix (R T ) and the inter-microphone noise covariance matrix (R V )Sure.

[0610]

[0611] The target covariance matrix between microphones (R T ) and the inter-microphone noise covariance matrix (R V ) is predetermined (e.g. selected in a given acoustic situation from among a plurality of predetermined values ​​for known acoustic situations), see Figure 1B The memory MEM in the SNR estimation module and the input d θ / R T ,R V The inter-microphone target covariance matrix (R T ) and the inter-microphone noise covariance matrix (R V ) can be used in the SNR estimation module based on the SNR from the voice activity detector VAD and the microphone (M1, ..., M M ) input VLAB and (X1,…,X M ) Adaptively determined.

[0612] Example 2

[0613] In an embodiment according to the present invention, the beamformer weights (w) are adaptively optimized for multiple target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of the sound from the multiple target locations (θ), wherein the SNR is based on the beamformer weights (w) and the inter-microphone target covariance matrix R T Expressed as follows. The target covariance matrix R between microphones T The corresponding steering vectors (d θ ) is determined. Multiple steering vectors (d θ ) can be determined dynamically during use of the hearing device based on the voice activity control signal VLAB (see Figure 1B The arrows in Figure 2 show the arrows to the SNR estimation module. θ )(or the corresponding target covariance matrix R T (θ)) can be determined in advance for a plurality of most likely target sound source positions (possibly selected according to the current sound environment) and stored in the memory MEM of the hearing device (see Figure 1B The dashed arrow in FIG. 1 provides the input steering vector (d θ ) or the covariance matrix (R T )). The signal-to-noise ratio (SNR) and thus the beamformer weights (w) may (additionally) depend on the inter-microphone noise covariance matrix R V The inter-microphone noise covariance matrix R VThe hearing device may be dynamically determined (e.g. in the case of Figure 1B The inter-microphone noise covariance matrix R V can be stored in advance (e.g. for a single noise sound field or for a plurality of different noise sound fields, e.g. for a plurality of corresponding acoustic situations and stored in the memory MEM of the hearing aid) and in a given situation selected from the memory of the hearing device (see Figure 1B The input R of the SNR-EST module is connected to the memory through the dashed arrow. V )Sure.

[0614] The target covariance matrix (R T ) can be a fixed (predetermined) full rank matrix. The noise covariance matrix (R V ) can be determined adaptively.

[0615] Target and / or noise covariance matrix (R T ,R V ) can be used in the voice activity control signal VLAB to indicate that the input (audio) signal monitored by the voice activity detector is marked as "target" (T=>R T ) and "noise" (N=>R V ) is estimated.

[0616] Figure 1C Schematically illustrated is a hearing device HD comprising a beamformer BF according to an embodiment of the present invention, wherein the beamformer comprises a beamformer configured to determine (optimize or approximate) beamformer weights (w=[w1, ..., w M ] T )’s weight optimization module (w, denoted as “correct w to maximize SNR”). Figure 1C Examples and Figure 1BThe embodiment of the present invention is similar to, but partially different from, that of the present invention, and further comprises a user interface (UI) configured to enable a user of the hearing device to specify a target position or multiple target positions that the beamformer should consider when determining the beamformer weights (w) (e.g. by optimizing the signal-to-noise ratio of the signal from the selected target direction). The user interface may be implemented as an APP of an auxiliary device (e.g. a smartphone or similar (e.g. portable, e.g. handheld) processing device) that communicates with the hearing device. The auxiliary device may, for example, include a graphical (e.g. touch-sensitive) display for supporting the APP. The APP may, for example, enable the user to specify one or more spatial angles (or angular ranges) around the user in which target sound sources that are considered as target signal sources by the beamformer are located. The user interface UI is configured to provide a target position control signal TPOS to a memory of the hearing device HD (e.g. by being transmitted from the auxiliary device to the hearing device (e.g. via a wireless link)). Based on the multiple user-selectable positions (θ) and the electrical input signals (X1, . . . , X M ) of multiple predetermined (and stored in MEM) target covariance matrices (R T (θ)), the target covariance matrix for the user-selected location (R T (θ*)) can be provided to the weight optimization module (w, denoted as "modify w to maximize SNR"). The voice activity detector VAD is configured to convert the input audio signal (here the beamforming signal Y) into a time-frequency unit. F ) is marked as target (T) or noise (N), as indicated by the voice activity control signal VLAB to the weight optimization module (w). The weight optimization module (w) is configured to optimize the sub-band signal (Y) marked as noise (N) by the voice activity control signal VLAB. F The inter-microphone noise covariance matrix R is determined by the time period (k, l) V ,like Figure 1C As an alternative, the voice activity detector VAD can estimate the noise covariance matrix R V The estimated value is forwarded to the weight optimization module (w). The weight optimization module (w, denoted as “modify w to maximize SNR”) is configured to make the beamforming signal (Y F (k,l)) is maximized, given the user-selected target position TPOS. The determined beamformer weights (w=[w1,…,w M ] T ) is applied to the electrical (time-frequency domain) input signal X = [X1, ..., X M ] T , whose outputs are added together in a combining (here summing) unit ('+') to convert the (spatially filtered) beamformed signal (Y F ) is provided as a linear combination of the electrical input signals.

[0617] As combined Figure 1B As shown, Figure 1C The hearing device HD further comprises means for applying one or more processing algorithms to the beamformed (possibly further noise-reduced) signal (Y F ). The audio signal processor SPRO provides a processed output signal OUT in the time-frequency domain, and the synthesis filter bank FBS is configured to convert the signal OUT in the time-frequency domain into a signal out in the time domain. Finally, the hearing device further comprises an output converter (here a loudspeaker SPK) for converting the processed output signal out into a stimulus that the user can perceive as sound, and may include other functional units, such as a functional unit that further improves the quality of the stimulus presented to the user.

[0618] Example 3

[0619] A third example (primarily but not exclusively) relates to a hearing device comprising a generalized eigenvector (GEV) beamformer (abbreviated as GEV beamformer). In conjunction with the GEV beamformer, the target covariance matrix (R T ) is assumed to be a full rank matrix. It is proposed to update the inter-microphone target covariance matrix and the inter-microphone noise covariance matrix of the beamformer based on voice activity detection, such as a) an estimated direction of arrival of sound from a target sound source, b) a comparison of signal contents provided by a target preserving beamformer and a target cancellation beamformer, respectively (e.g., the difference between the two), or c) speech detection.

[0620] Zoom w

[0621] As mentioned, maximizing the following equation only determines a set of weights w up to the scaling factor.

[0622]

[0623] One way to deal with this is to still impose an undistorted direction constraint on the preferred direction θ*, for example by scaling w so that

[0624]

[0625] As a result, only a single (real-valued) scaled value of w satisfies the above constraints. This particular scaling ensures that the response towards the preferred target direction θ* is undistorted. However, the SNR can still be optimized even if the current target direction deviates from the preferred target direction.

[0626] Estimated covariance matrix

[0627] In MVDR beamforming, for a fixed target direction, we only have to estimate the noise covariance matrix. Target covariance matrix Known in advance. The noise covariance matrix (R V ) is usually estimated during the absence of speech. When we maximize the SNR, where the target is assumed to be transmitted not only from the desired direction, we therefore need to estimate the target covariance matrix (R T ) and the noise covariance matrix (R V ).

[0628] Optimize the before-after ratio

[0629] In the "front-to-back" framework, it is assumed that the sound coming from the front is the sound of interest to the listener, and the sound coming from the back is considered noise. In this case, our goal is to optimize the front-to-back ratio, so we must decide when to update the target covariance matrix and the noise covariance matrix.

[0630] To this end, it is proposed to use a front-to-back detector based on a comparison between a beamformer pointing forward and a beamformer pointing backward. Figure 2 and Figure 3 Shown in.

[0631] Figure 2 A first embodiment of an input stage of a hearing device according to the invention is shown. Figure 2 shows a beamformer with a forward-pointing cardioid C. F (with zero towards the rear) and the amplitude response of the other beamformer has a cardioid C pointing towards the rear B (with zero towards the front).

[0632] Figure 2 The input stage of the hearing device comprises a microphone system adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m (n), m = 1, 2 two microphones (M1, M2), n represents time. Figure 1B As in the example, each microphone path comprises an analysis filter bank FBA for transforming a time domain input signal (x) representing the sound in the environment into m , i=1,2) into the corresponding sub-band signal (X m , m=1,2). The input stage of the hearing device further comprises a beamformer unit BF connected to the microphone system. The beamformer unit BF is configured to provide two beam patterns: 1) a forward beamformer (C F ) pointing in front of the user (i.e. the user's viewing direction) and 2) the beamformer (C B) is pointed behind the user (e.g., in the opposite direction of the user's viewing direction). The forward beamformer may be implemented, for example, as a delay and sum beamformer (or an equivalent beamformer). The backward beamformer may be implemented, for example, as a delay and subtraction beamformer (or an equivalent beamformer). The forward beamformer may be calibrated (updated weights) based on a signal from the rear (e.g., from a rear cancellation beamformer). Similarly, the backward beamformer may be calibrated based on a signal from the front (e.g., from a front cancellation beamformer).

[0633] Based on a) forward-pointing beamformer (C F , pointing back with zero (or maximum attenuation) amplitude response and b) pointing back with a beamformer ((back)C B , with zero (or maximum attenuation) in the forward direction), can be used to determine whether the incoming sound (in a given time-frequency (TF) unit) arrived from the front or the back (see Figure 2 The calculation module "Compare" in the obtained signal f(C F ,C B ). In its simplest form, the signal f(C F ,C B ) can be binary, such as 1 and 0 (or T (target) and N (noise)). When the C of a given TF unit F and C B When the difference between is positive and negative respectively. Figure 2 The calculation module "Compare" in the embodiment thus implements a "front-back detector" and provides an output signal (f(C)) indicating the source (or main source) of a given signal component (k, l) in the time-frequency domain. F ,C B )).

[0634] For each time-frequency unit (t,f) (or l,k), we thus update the target covariance matrix R based on the following criterion T and the noise covariance matrix R V :

[0635] If log|C F (t,f)|-log|C B (t,f)|>κ F , update R T

[0636] If log|C F (t,f)|-log|C B (t,f)|<κ B , update R V

[0637] where κF and κ B is the threshold value, such as Figure 4A as shown in .

[0638] The target covariance matrix (R T ) and the noise covariance matrix (R V ) can be recursively updated as:

[0639]

[0640] and

[0641]

[0642] Where n is the time frame index, λ T ,λ V is a coefficient in the interval [0; 1]. T ,λ V is the coefficient of the exponential decay that controls the update. Typically, the coefficient λ∈[0;1] is expressed in terms of a time constant given by:

[0643]

[0644] where F s The sampling rate for covariance matrix updates.

[0645] Figure 3 A second embodiment of an input stage of a hearing device according to the invention is shown. In addition to the two beam patterns (C F ,C B ), Figure 3 Examples and Figure 2 The same as the embodiment of Figure 3 In an embodiment of the invention, the beamformer unit BF provides a hypercardioid pointing in front and behind the user, respectively. Thus, the front-to-back detector ("comparison") is based on a comparison between a hypercardioid beamformer maximizing the front-to-back ratio and a hypercardioid enhancing the back-to-front ratio.

[0646] Use a cardioid pattern for comparison (e.g. Figure 2 The advantage of comparing two hypercardioid beam patterns is that the difference between the cardioid patterns is a monotonic function of the direction. However, it may also be advantageous to compare two hypercardioid beam patterns instead, such as Figure 3 , because these beam patterns, on average, have the best separation between the front and rear halves. A fixed beam pattern can also be selected based on any desired angle of interest, i.e., a fixed beam pattern that optimizes:

[0647]

[0648] Among them, g θis the possible weight of the signal from direction θ, d θ is the relative transfer function from the direction θ known in advance. The terms “relative transfer function” and “steering vector” are used interchangeably and are denoted by d θ A special case of the above equation is the hypercardioid that maximizes the front-to-back ratio, where the target directions are all directions from the front half of the diffuse noise field and the noise directions are all directions from the back half of the diffuse noise field.

[0649] The textbook definition of the front-to-back ratio is given by the ratio of all signals from the front half to the signals from the back half. Implicitly, the FBR assumes that the signals in the front half are the signals of interest (target) and that the signals from the back are of less interest (noise). The FBR is defined, for example, in [Simmer et al.; 2001].

[0650] The above example is explained in the case of two microphones, but it is also applicable to more than two microphones.

[0651] Figure 4A Two thresholds κ for determining the time and frequency regions are shown F and κ B , where the target covariance matrix R is updated or not T (log|C F (t,f)|-log|C B (t,f)|>κ F )To update the noise covariance matrix R v (log|C F (t,f)|-log|C B (t,f)|<κ B ). In the shaded area, the covariance matrix is ​​not updated. Figure 4A Polar plot showing a comparison between two directivity patterns for a forward beamformer and a backward cardioid beamformer. Since the two beampatterns have sensitivities in opposite directions, the forward-backward determination can be based on a comparison between the magnitudes of the forward and backward beamformers. The beampatterns shown are first order cardioids (based on two microphones), but the two beamformers can also be based on more microphones. Since the difference between the two beamformers is small, about + / - 90 degrees, we can choose not to update any covariance matrices if the signal is coming from a direction within the shaded area.

[0652] Figure 4B and 4C A slightly different implementation of the input stage of a hearing aid according to the invention is schematically shown, comprising Figure 2 and 3The front-to-back ratio (FBR) optimized beamformer BFU shown in . The front-to-back comparison result (based on the comparison of the corresponding outputs of the front beamformer and the back beamformer) can be used as an input to a neural network (NN), such as a deep neural network (DNN), such as Figure 4B As shown in (see Figure 4B The input module "compares" and the signal ΔCFR(k,l) in , or the before and after comparison can be included in the neural network (NN), such as Figure 4C As shown in (see Figure 4C The NN module "Compare and Determine" and the input signal C of the FBDEC (NN) module F (k,l),C B (k,l)). Figure 4B , 4C The input stage of an embodiment comprises two microphones (M1, M2) for picking up sound from the hearing aid environment. The microphones (M1, M2) provide corresponding (e.g. digitized) electrical input signals (x1, x2) in the time domain. The electrical input signals (x1, x2) in the time domain are converted into the time-frequency domain by means of corresponding analysis filter banks FBA (see signals X1(k, l), X2(k, l), where k and l are frequency and time indices, respectively). The time-frequency domain electrical input signals X1(k, l), X2(k, l) are fed to a front-to-back ratio (FBR-) optimized beamformer (BFU). The FBR beamformer (BFU) provides a) a forward-pointing (cardioid) beamformer (C F , with zero (or maximum attenuation) when pointing backwards) and b) a (cardioid) beamformer pointing backwards ((rear) C B , which has zero (or maximum attenuation) when pointing in the forward direction. Alternatively, you can use Figure 3 A hypercardioid or equivalent, for example, pointing in the opposite direction (assuming the corresponding beam patterns are essentially mirror-symmetric about the axis separating "front" and "rear"). Figure 4B A comparison module ("Compare") in is configured to compare the forward beamformer and the backward beamformer and to provide as output a comparison signal indicating a time-frequency representation of their difference (see ΔCFR(k,l)). The forward-backward difference measure (ΔCFR(k,l)) is input to a forward-backward determination module (FBDEC(NN)) implemented as (or comprising) a neural network (NN).

[0653] Another input to the neural network can be a voice activity control signal (VAD(k,l)), such as Figure 4B and 4C The voice activity control signal may be based on one or more electrical input signals (X1, X2), for example.

[0654] exist Figure 4BIn an embodiment of the invention, the neural network (NN) is instantiated by at least one layer comprising a gated recurrent unit (GRU) followed by one or more feed-forward layers (FF). At the output layer, an activation function ("activation") such as sigmoid is applied, followed by a threshold (threshold) (indicating the "decision" (see output signal FBD(k,l): "before" or "after" (or "none"))). Between the layers of the neural network (NN), non-linear activation functions (such as rectified linear unit (ReLU), sigmoid, tanh, etc.) may be applied.

[0655] Figure 4C A further embodiment of the input stage of a hearing aid according to the invention is schematically shown, comprising: Figure 2 (or Figure 3 ) shown in the front-to-back ratio (FBR-) optimized beamformer (BFU). Figure 4C and Figure 4B Similar, but Figure 4B (and Figure 2 or Figure 3 ) is included in the neural network (NN). Thus, the comparison of the forward and backward beamformers provided by the FBR beamformer (BFU) is included in the neural network calculation (so that the decision is not "biased" by the preceding comparison). The neural network can thus be trained using data where the forward and backward beamformers (or the corresponding input signals) are known for a plurality of input signals (from different directions and) with different signal-to-noise ratios etc.), associated with known forward, backward (or none) decisions (FBD()) and optionally (if the voice activity control signal is used as an additional input to the neural network) also with known values ​​of the VAD input (VAD(k,l)).

[0656] Figure 4A , 4B Embodiments of 4C may be configured to focus on detecting the user's own voice (instead of the front (or front half) sound).

[0657] The advantage of applying neural networks is that cross-channel information can be considered for joint decision making.

[0658] The MVDR beamformer is the best solution for enhancing a single target direction in a noisy field, while the GEV beamformer can consider multiple directions as target directions at the same time. Figure 5 Shown in.

[0659] Figure 5 Optimization of the beamformer based on different criteria is shown. Figure 5Different directivity diagrams (0 to 360°) are shown for a cylindrical diffuse noise field with two microphones located in behind-the-ear hearing aids. Each circle represents a 3 dB change from its nearest neighbor. The directivity diagram is obtained by weighting the clarity index (AI) of the directivity diagrams across frequency bands. The clarity index (e.g., taking a value between 0 and 1) is an estimate of the portion of speech that is audible to a hearing-impaired user with a specific (frequency-dependent) hearing loss. Since we estimate the beamformer weights separately for each frequency band, we maximize the FBR in each frequency band individually. Instead of showing the directivity diagram for each channel, Figure 5 The frequency-weighted average directivity diagram is shown (here illustrated by weighting of the intelligibility index). The dotted directivity diagram shows the MVDR beamformer solution, where the target direction is assumed to be directly in front of the listener (0°, in the upper part of the diagram). This directivity diagram is optimal for attenuating diffuse noise, and we thus obtain the optimal directivity index. However, if we optimize the front-to-back ratio instead, the MVDR beamformer is not optimal. The solid line shows the MVDR beamformer with the true target and noise covariance matrix (R T and R V ). In this case, the DI is lower than that of the MVDR beamformer, but the FBR is higher. The dashed line shows the FBR optimized beamformer (cardioid-based) based on the estimated target and noise covariance matrices, where R T Based on log|C F (t,f)|-log|C B (t,f)|>3dB time frame, R V Based on log|C F (t,f)|-log|C B (t,f)|<3dB time frame, see Figure 4A The FBR-optimized GEV beamformer (solid curve) has a higher DI because it covers more area in the front half and less area in the back half compared to the dashed curve (MVDR beamformer).

[0660] Direction-based decisions may be combined with voice-based decisions, such as one or more of the following: a) R T Update only when the sound comes from the front and there is speech; b) R v Can only be updated when there is no noise; c) R v Can only be updated if there is no noise and / or the sound is coming from the back half.

[0661] The directivity index (DI) is given by the ratio of the response in the target direction θ0 (R) to the response in all other directions:

[0662]

[0663] The front-to-back ratio (FBR) is defined as the ratio between the response of the front (front) half of the surface (R) and the response of the back (back) half of the surface:

[0664]

[0665] Target covariance matrix based on possible target directions

[0666] Figure 6 shows how the most likely target direction (or target steering vector) can be selected among a dictionary of different candidate directions. There are different approaches to estimating the most likely target direction, e.g. for an MVDR beamformer. Instead of selecting a single target steering vector for use in an MVDR beamformer, we can (e.g. Figure 6 ) finds multiple target directions θ in a dictionary of target steering vectors (relative acoustic transfer functions) of the sound source relative to the user's relative position (see, for example, Figure 6 Based on the likelihood estimates, we can generate the target covariance matrix from the weighted sum of the possible target vectors, i.e.

[0667]

[0668] where p θ is the weight of the estimated probability (or likelihood) based on a given direction θ, where p θ It can be estimated as disclosed in, for example, EP3413589A1, or by a trained neural network. The noise covariance matrix (R T ) is estimated during periods when speech is absent (e.g., as indicated by a voice activity detector).

[0669] Figure 7 Different directivity patterns (0 to 360°) resulting from optimization of the beamformer based on different criteria are shown. The dotted line shows the directivity pattern of a well-known MVDR beamformer, where the noise covariance matrix R V Estimated from a cylindrical diffuse noise field based on two hearing aid microphones located behind the left ear. The resulting beamformer is expected to maximize the directivity index (DI). The solid line shows the directivity diagram of the GEV beamformer optimized for estimating target directions from 0 and 180 degrees. The target covariance matrix R T The noise covariance matrix R is obtained by summing the outer product of the steering vector from 0 degrees and the outer product of the steering vector from 180 degrees. VEstimated from a cylindrical diffuse noise field. We note that the directivity pattern has a dipole shape that preserves energy from two target directions. However, with only two microphones, we can only guarantee an undistorted response (0 dB) from a single direction (in this case, forward).

[0670] Figure 7 shows the benefit of generating the target covariance matrix based on the most likely steering vector, i.e.

[0671]

[0672] In this particular case, we assume that the target directions from 0 and 180 degrees are possible target directions, and we thus obtain the full-rank target covariance matrix As from Figure 7 As can be seen, the resulting directivity pattern becomes a dipole (solid line) instead of a The supercardioid directivity diagram (dashed line) obtained in this case. As can be seen from the solid line, the gain in the 180-degree direction (center bottom) is about 10dB compared to the dashed line. Figure 7 In , the beamformer weights are normalized to maintain a distortion-free response (0 dB) from the target direction (0°, top center), i.e. As an alternative, we can normalize the weights so that w H R T w=q, where q is a scalar (scaling parameter), for example q=1.

[0673] Example 4

[0674] In the following, the connection between the minimum variance distortionless response (MVDR) beamformer weights and the generalized sidelobe canceller (GSC) weights is established. It is shown that a GSC structure with a single adaptive parameter can be used even if the target view vector (steering vector) is dynamically updated.

[0675] Given the acoustic transfer function h(k) (in the kth channel), we can calculate the normalized view vector

[0676]

[0677] Such that |d| 2 =1.

[0678] The MVDR beamformer is designed to maximize noise suppression under the constraint that signals from the target direction pass without distortion.

[0679] It can be shown (see, for example, [Bitzer & Simmer; 2001], [Souden et al.; 2010]) that the filter coefficients of the MVDR beamformer can be expressed as follows:

[0680]

[0681] Where R V (k) is the inter-microphone noise covariance matrix, and the vector d(k) is the view vector corresponding to the acoustic transfer function to the target, normalized with respect to the reference microphone.

[0682] The above equation is the same as the following:

[0683]

[0684] in, has been normalized relative to its reference microphone, i.e.

[0685]

[0686] The above equation is valid for any number of microphones M>1. For the special case of M=2 microphones, the MVDR filter output Y F It can be expressed in terms of fixed beamformer outputs C0 and C1 as:

[0687] Y F (k) = C0(k) - β(k)C1(k)

[0688] where the complex scalar β is given by:

[0689]

[0690] in and in

[0691]

[0692] It should be noted that Note that this is just one example of how the beamformer weights can be chosen. C1(k) only has to satisfy that the signal from the desired target direction is cancelled, and C0(k) is chosen to have a given response, such as unity, in the desired target direction. This is actually a special case of a generalized sidelobe canceller, where we have

[0693] w GSC (k) = a-Bβ

[0694] where a is typically an M×1 delay-and-sum beamformer vector that does not change the direction of the target signal, B is an M×(M-1)-sized blocking matrix, and β is an (M-1)×1 adaptive filter matrix with adaptive coefficients, which for the MVDR scheme is given by [Bitzer & Simmer; 2001]

[0695] β=(B H R V B) -1 B H C V a

[0696] Where a and B are orthogonal to each other, that is, a H B=0 1×(M-1) , β is updated in the absence of speech. The beamformer weights are then calculated as:

[0697] w GSC (k) = aB(B H R V B) -1 B H C V a

[0698] Note that in the following we use a slightly different notation, where the adaptation coefficient is conjugate compared to the definition of β from [Bitzer & Simmer; 2001].

[0699] In the special case of two microphones, we have

[0700] w GSC (k) = w0(k) - w1(k)β*(k)

[0701] Where (omitting the frequency index k)

[0702]

[0703] We note that, depending on the design, we can determine β directly from the signals C0 and C1, or from the noise covariance matrix R v Determine β, that is

[0704]

[0705] For example, if the signals C0 and C1 are used elsewhere in the device in question, it may be advantageous to derive β directly from these signals:

[0706]

[0707] But on the other hand, if the viewing direction (steering vector d) (and hence the weights w0 and w1) must be changed, then it is disadvantageous to include the weights in the anticipation operator. In this case, it would be advantageous to derive β directly from the noise covariance matrix.

[0708] The parameter β can be estimated adaptively. In this case, the minimum square error is estimated. Given the output Y of the MVDR (GSC) beamformer

[0709] Y(k)=C0(k)-β(k)C1(k)

[0710] The minimum square error err (omitting the band index k) can be written as:

[0711] err= <YY * >=<|Y| 2 >

[0712] According to C0, C1 and β, we can convert |Y| 2 Rewrite as

[0713]

[0714] Recall that the complex derivative is given by

[0715]

[0716] in and are the real and imaginary parts of the complex parameter x (in this case, the (complex) parameter β). We thus determine

[0717]

[0718] In the two-microphone case, we can remove the expectation since the smoothness is obtained by the recursive update of β. We thus have

[0719]

[0720] Where j is a character with property j 2 = -1 complex unit. In order to make |Y| 2 Minimize, we therefore update β in the direction of the negative gradient, i.e.

[0721]

[0722] Where n is the time index. Instead of using the LMS algorithm, we can use the normalized LMS (NLMS) algorithm given by the following formula to get faster updates:

[0723]

[0724] Other normalization schemes can also be applied. For example, we can normalize the gradient by outputting:

[0725]

[0726] Also, since we apply many small steps, the accuracy of the division does not need to be as high as when estimating β in a single step.

[0727]

[0728] We can therefore use an approximate normalization of the denominator (e.g., by changing |C1| 2 Round to the nearest 2 N , where N is an integer). Thereby, the division may be implemented by a shift operator. To avoid division by zero, a small value may be added to the denominator. In an embodiment, the multiplied beamformers are averaged across the time frame.

[0729] Fig. 8A and Figure 8B Each of them shows a schematic diagram of an implementation of a directional noise reduction system according to the present invention, which includes a three-microphone generalized sidelobe canceller structure. Fig. 8A and Figure 8B A corresponding embodiment of a hearing device HD is shown comprising three microphones (M1, M2, M3), each microphone providing a corresponding electrical (time domain) input signal (x1, x2, x3) representing the sound in the environment of the hearing device HD. Each electrical microphone path comprises an analysis filter bank FBA for transforming the time domain input signal (x1, x2, x3) into a sound signal. i , i=1,2,3) into time-frequency representation (X i , i=1,2,3), for example represented by corresponding time indices and frequency indices (n,k), each time-frequency representation being associated with a (usually complex) value of the signal in question at a given time (n) and frequency (k). The hearing device further comprises a directional noise reduction system comprising a three-input MVDR beamformer implemented as a GSC structure (GSC). The output (Y) of the directional noise reduction system (here the MVDR / GSC-beamformer) is fed to a synthesis filter bank FBS for converting the time-frequency representation of the signal (Y) into an output signal (y) in the time domain. The time-domain output signal is fed to an output transducer of the hearing device HD, here a loudspeaker SPK, for providing the output signal (y) as a stimulus perceivable as sound by a user of the hearing device (here provided as air vibrations towards the eardrum of the user). Further processing of the input signal representing the sound or the output signal of the noise reduction system may be provided, for example by a processor for applying one or more processing algorithms to the corresponding signal of the signal path, such as a compression-amplification algorithm for adapting the level of the input signal to the hearing ability of the user, for example for compensating for a hearing loss of the user. The hearing device HD may for example implement a hearing aid or headphones (or a combination thereof).

[0730] The shown GSC implementation of the MVDR beamformer (GSC, see the dashed box surrounding the beamformers a, b1, b2 and the combining units CU1, CU2, CU3, CU4) comprises a blocking matrix B, where each row of B corresponds to the weights of two independent target cancellation beamformers, i.e.

[0731] b=[b1,b2]

[0732] in

[0733] The two target cancellation beamformers can be written as:

[0734]

[0735] in For the three microphones (in Fig. 8A , 8B In the figure, the noisy input signals are M1, M2, and M3 respectively. Fig. 8A , 8B or through the time domain signal (x i , i=1,2,3) represents or through the time-frequency signal (X i , i = 1, 2, 3). The dimensions of the different elements that define the beamformer (weights (a, b1, b2) and adaptive update coefficients (β)) are Fig. 8A and 8B For the case of M=3, it is specified that: the input signal x is represented by a 3x1 vector. The target preserving beamformer is represented by a 3x1 vector of weights a. The two target canceling beamformers (b1, b2) are represented by a 3x2 dimensional blocking matrix B. The adaptive update coefficient (β) is represented by a 2x1 vector. It should be noted that this can be generalized to M microphones (where M≥3).

[0736] Similarly, we can write the noisy undistorted target signal as

[0737] C0=a H x

[0738] in

[0739] Now we consider the adaptive update coefficient β = [β1, β2] T =(B H R V B) -1 B H R V a, where the superscript T refers to the transpose. We note that this coefficient depends on the noise covariance matrix given by:

[0740] R V = <xx H >

[0741] where β refers to the (time) averaging operator (i.e., averaging when no target is present, VAD = 0). Similar to the two-microphone case, we can avoid estimating the covariance matrix. We rewrite β as

[0742]

[0743] It should be noted that in the two-microphone case (M=2), Simplified to

[0744]

[0745] This is similar (except for the complex conjugation) to the definition of β for a two-microphone GSC beamformer elsewhere in this application, but is similar to the definition provided in [Bitzer & Simmer; 2001].

[0746] For the three-microphone case, we can further rewrite the above equation as

[0747]

[0748] We therefore have

[0749]

[0750] and

[0751]

[0752] because We can add a constant to the denominator to avoid division by zero and thereby limit the size of β.

[0753] It is clear from the above that in order to calculate β, we need to span two real terms (<|C1| 2 > and <|C2| 2 >) and three imaginary terms ( and ). Compared to averaging each element of the covariance matrix (3 real averages and 3 complex averages), this is one less average and thus advantageous when minimizing computational complexity (and thus power consumption) is a priority, such as in hearing aids. In general, for M microphones, we need one less real average than when computing the covariance matrix directly. This can be explained by the fact that the noise covariance matrix is ​​present in both the numerator and the denominator. Therefore, we only need to estimate up to a fraction of the covariance, which saves to a single real average in the beamformer multiplication implementation.

[0754] The target preserving beamformer (C0) is given by the weight a. The weight can be obtained from the target steering vector d as

[0755]

[0756] The weights of the target cancellation beamformer can be determined by selecting M-1 rows from the matrix given below.

[0757]

[0758] For example

[0759]

[0760] There are other choices (other than the above example) for selecting beamformer weights b1 and b2 that cancel signals from the target direction.

[0761] In this case, each target cancellation beamformer becomes a linear combination of all three microphones. We note that a H B=0.

[0762] As an alternative, we can just construct the blocking matrix from two independent two-microphone beamformers (similar to the Griffiths-Jim beamformer).

[0763] We may, for example, select two first-order beamformers in the blocking matrix in the following manner (eg, by removing the input signal X3 of b1 and the input signal X2 of b2).

[0764] We define

[0765]

[0766] The weights of the two first-order beamformers thus become

[0767]

[0768] Thus, the structure of the blocking matrix becomes

[0769]

[0770] We note that a H B=0 is still true.

[0771] A three-microphone generalized sidelobe canceller based on two first-order target cancellation beamformers Figure 8B Shown in. Figure 8B Shown with Fig. 8A Schematic diagram of an implementation of a directional noise reduction system similar to that in , but where instead of generating two independent target cancellation beamformers based on all three microphones, two independent first order target cancellation beamformers may be generated. A two-input or three-input target cancellation beamformer may be enabled in different (two-input and three-input target cancellation beamformer) operating modes of the system. Compared to Fig. 8A (which illustrates a three-input target cancellation beamformer mode), Figure 8B The solid line connection of the two-input target elimination beamformer mode has been removed. The dimensions of the multiple different elements of the beamformer (weights (a, b1, b2) and adaptive update coefficients (β)) are defined as Fig. 8A For the case of M=3, the Figure 8B The difference is that one of the elements of each column of the blocking matrix B is Figure 8B The two-input target cancellation beamformer operation mode of the system is zero.

[0772] In general, it may be argued that battery capacity is not well spent on powering three microphones in all situations. We may apply beamforming based on three microphones only in the most complex environments. Which microphones are switched may depend on the microphone configuration in the hearing device. In the general situation where the user focuses his attention on one sound source in the environment, it may be desirable to switch towards two microphones, the axes of which are most parallel to the front-rear axis. In the self-voice pickup (telephone) operating mode of the hearing aid, different two-microphone configurations may be selected. In the situation where one of the microphones is detected to be inoperative, it may be preferred to select the two microphones that are still functional.

[0773] In a three-microphone input system (first operating mode), the GSC structure includes a (e.g. fixed) three-input target preserving beamformer (a) and a first and a second (e.g. fixed) two-input target cancellation beamformer (b1, b2); while in a two-microphone input system, the GSC structure includes a (one, e.g. fixed) two-input target preserving beamformer (a) and a single (e.g. fixed) two-input target cancellation beamformer (b1).

[0774] Figure 8C It shows that Figure 8B Schematic diagram of an implementation of a directional noise reduction system in which the effect of a gradual change in the input of the target cancellation beamformer from three to two (or from two to three) microphones on the beamformer weights is indicated. Fig. 8A and 8B The corresponding example involves a three-input MVDR beamformer implemented as a GSC structure (GSC) (see the indication of the update rule for the adaptive parameter vector β). It should be noted that although Figure 8C A transition from two to three (or vice versa) microphones at the input to a directional noise reduction system (GSC beamformer) is shown, but is not specifically tied to the MVDR beamformer.

[0775] The starting point is that the system includes 3 inputs to the target-preserving (TM) beamformer (a) and 2 inputs to each of the target-canceling (TC) beamformers (b1, b2).

[0776] Afterwards, we remove the inputs of the target preserving beamformer as well as the target cancellation beamformer (e.g. by disabling the microphones), whereby the TC beamformer that previously received the "removed input" is "cancelled" (the input signal is faded to zero), while the two remaining inputs of the TM beamformer are faded to increase their input levels (to avoid artifacts).

[0777] The fading of the input audio data streams of the (e.g. target-keeping) beamformer from more to fewer (e.g. from three to two) over a certain fading time period may, for example, include the input stage being configured to provide more (e.g. three) data streams as input signals to the directional noise reduction system at a first time point t1 and to provide fewer (e.g. two) data streams as input signals at a second time point t2, wherein the second time t2 is greater than the first time t1. The fading time Δt fad =t2-t1, for example, may be less than a predetermined time range, such as Δt fad <20s, or <10s, such as <5s, such as between 1s and 5s.

[0778] The gradual change process may include determining corresponding gradual change parameters (t1, t2), α(t1), α(t2)) of a gradual change curve that gradually decreases (or increases) the weight of the affected input signal, for example, see Figure 8C Upper and lower gradient curves (α-t).

[0779] In such Figure 8C In a directional system including three microphones in , the target-holding beamformer (C0) may apply a weight (a) of about 1 / 3 to each microphone signal (X1, X2, X3). In a directional system including two microphones, the target-holding beamformer (C0) may apply a weight (a) of about 1 / 2 to each microphone signal. The simplest thing to do when fading is to apply a scaling to each weight while fading, i.e., while fading from three to two microphones (or vice versa). An example of this is in Figure 8C , where the scaling (α) is implemented by a multiplication unit ('X'), see Figure 8C Arrows from the gradient curves in the upper and lower parts of FIG. 8 (scale, α-time t).

[0780] Figure 8C A hearing aid HD comprising a directional noise reduction system is shown, comprising a three-microphone input GSC structure (GSC) comprising a (e.g. fixed) target preserving beamformer (a) and first and second (e.g. fixed) target cancellation beamformers (b1, b2). The first and second target cancellation beamformers are based on subsets of two of the three microphone input signals (X1, X2, X3). The hearing aid is configured to switch (e.g. fade) between first and second operating modes, wherein in the first mode all three microphone inputs (X1, X2, X3) are active and in the second operating mode only two of the three microphone inputs (X1, X2) are active. In other words, one of the three microphones (M3) and one of the beamformers (b2) may be disabled in the second operating mode.

[0781] exist Figure 8CIn the example of FIG. 1 , one of the target cancellation beamformers (the target cancellation beamformer containing the third microphone (M3)) is faded out (a gradient from three to two inputs to the target preserving beamformer (a) is shown). At the same time, the target preserving beamformer weight (a) is increased by 50%, for example by changing the weight ratio of the two remaining coefficients of the input signal (X1, X2) from 1 to 1.5 and fading the third coefficient (of the input signal X3) to zero. After the gradient, the third microphone (M3) can be completely turned off to save computing power. In addition, one of the target cancellation beamformers (the target cancellation beamformer (b2) containing the third microphone signal (X3)) is faded out.

[0782] As an example, consider a target preserving beamformer a=[1 / 3,1 / 3,1 / 3] in a three-input configuration. T When fading out the third microphone, if we do not change the weights, we have

[0783] a H x=[1 / 3,1 / 3,1 / 3]*[x1,x2,0]

[0784] Instead, we propose to change (e.g. gradually) a to [1 / 2,1 / 2,0] before turning off the third microphone. T .

[0785] The hearing aid may be configured to store in a memory the beamformer weights (and fading parameters) optimized in advance for the first and second operating modes of the directional system.

[0786] The main advantage of using a two-microphone (target cancellation) beamformer in a three-microphone system is that transitioning between a two-microphone system and a three-microphone system becomes easier because the target cancellation beamformer can be reused.

[0787] The initiation of the transition from the first to the second operating mode (or vice versa) may be controlled by the user via a user interface or by a control signal indicative of the complexity of the current acoustic environment around the user.

[0788] The trigger for entering a specific operating mode (changing from a three-input to a two-input targeted cancellation beamformer) may for example be related to energy saving. The trigger may for example be based on the input level or another parameter, such as provided by a sound scene detector.

[0789] Complexity of the sound scene can trigger a gradual change from two to three microphones. Triggers can be functions of level, SNR, remaining battery time, motion, for example.

[0790] The MVDR beamformer weights are proportional to

[0791]

[0792] If the estimated beamformer weights w have not been scaled so that w H d=1, we can normalize the weights so that

[0793]

[0794] Thus, we have

[0795]

[0796] Recall that the weights can be written as

[0797]

[0798] We therefore have

[0799]

[0800] For two microphones, we can make β1 and Isolation, that is

[0801]

[0802] For three microphones, with the blocking matrix given by a first-order beamformer, We can easily isolate β1 and β2. In this case, we have

[0803]

[0804] and

[0805]

[0806] In addition to saving some multiplications in each row of the blocking matrix, an advantage of basing the targeted cancellation beamformer on a first order cardioid (e.g., in the two-input targeted cancellation beamformer mode of operation of the system) is that it is possible to independently tune each beamformer (to obtain deeper targeted cancellation nulls (using a three-input targeted cancellation beamformer)) simply by adjusting a single parameter.

[0807] Likewise, it becomes easier to fade from a three-microphone system to a two-microphone system without changing the target cancellation beamformer weights because we have predetermined sets of weights with 3 weights and 2 weights for each target cancellation beamformer. When the target cancellation beamformer weights are based on two microphones, fading to a two-microphone system (or fading from 2 microphones to 3 microphones) becomes easier.

[0808] Similar to the two-microphone case, the update coefficients (β(β1,β2)) can be adaptively estimated for more than two microphones. For the three-microphone case, the least square error (LMS) is estimated. Fig. 8A , 8B In the notation, the output Y of the three-input GSC structure is given by

[0809] Y(k)=C0(k)-β1(k)C1(k)-β2(k)C2(k)

[0810] According to C0, C1, C2β1 and β2, the output amplitude square (<|Y| 2 >) can be rewritten as:

[0811]

[0812] The real part of β1 can be obtained and the imaginary part The derivative of :

[0813]

[0814]

[0815] For convenience, the above equation can be expressed as:

[0816]

[0817] Similarly, the derivative with respect to β2 can be obtained and is given by:

[0818]

[0819] We thus note that the gradient update is similar to the two-microphone case, The LMS update is given by:

[0820] β(n)=β(n-1)+μ<C*Y>

[0821] in

[0822] Instead of using the LMS framework, a faster update can be obtained using the Normalized LMS (NLMS) algorithm given by:

[0823]

[0824] The LMS / NLMS formula for updating the coefficients (β) is valid for any number of microphones >1.

[0825] However, it is worth mentioning that the division operation in the LMS / NLMS update may be less accurate than the division used in the single-step estimation. However, in the NLMS case, the division can be simply implemented by a shift operator, which is computationally cheaper. An alternative estimate of the coefficient (β) is proposed below.

[0826] To estimate β, we have several options. We can estimate β directly as

[0827]

[0828] and

[0829]

[0830] Or we can use the NLMS algorithm to update β1 and β2 as

[0831]

[0832] Since the direct calculation of β1 and β2 involves quartic (fourth order) terms, it is not very easy to implement in fixed point arithmetic. The NLMS algorithm seems (at first glance) more promising since only quadratic terms are involved.

[0833] However, NLMS requires careful selection of learning rate to ensure stability.

[0834] As an alternative to the solution considered above, let us revisit the output magnitude squared |Y| 2 Gradient of the update coefficient β

[0835]

[0836] By setting gradient = 0, we obtain two expressions:

[0837]

[0838] It is proposed to iteratively alternate between determining β1 (by the equation above, given a previously estimated value of β2) and β2 (by the equation above, given a previously estimated value of β1).

[0839] In this scheme, we do not have to choose a learning rate. We only need to choose a smoothing coefficient for the average operator <··>. In addition, if the β term fluctuates during convergence, it may be beneficial to apply a small low-pass filter to β1 and β2.

[0840] Note that from the above equation, we can also Insert into , thus isolating β1. Thus, we again obtain

[0841]

[0842] and

[0843]

[0844] This is the same formula as above for β1 and β2.

[0845] Consider the target covariance matrix R T and the noise covariance matrix R V The signal-to-noise ratio is defined as:

[0846]

[0847] We can estimate the weight w that maximizes the SNR as the eigenvector belonging to the largest generalized eigenvalue (thus providing a GEV beamformer). Note that the weight vector w can be scaled by arbitrary phase and amplitude and still maximize the SNR. If computing the eigenvalues ​​is computationally too expensive, we can instead provide an approximation by maximizing the SNR using gradient ascent. The gradient is given by:

[0848]

[0849] Now, let us consider a more constrained optimization, where we write the weights in terms of a generalized sidelobe canceller.

[0850] In the following, we assume that we can identify specific time-frequency units belonging to the target signal T or the noise signal V. We can thus T = <xx H > T The target covariance matrix is ​​estimated from the input signal portion of R. The subscript T means that we average across the samples belonging to the target signal. Similarly, we can define the noise covariance matrix as R V = <xx H > V , where the subscript V refers to averaging based on samples belonging to noise.

[0851] Recall that the output signal is given by:

[0852] Y(k)=C0(k)-β1(k)C1(k)-β2(k)C2(k)…-β M-1 (k)C M-1 (k)

[0853] where C0(k) is the signal that provides an unchanged response for a given direction, C1(k),…,C M-1 (k) denotes M-1 independent target cancellation beamformers. In the following, we ignore the frequency index k. β refers to the adaptation weight.

[0854] Recall that the output

[0855] Due to C T can be written as <xx H > T , we can write the SNR as

[0856]

[0857] For simplicity, we consider a first-order difference beamformer where

[0858] |Y| 2 =(C0-βC1)(C0-βC1) *

[0859] Thus, we can rewrite the SNR loss function L in terms of β.

[0860]

[0861] In one aspect, a generalized sidelobe canceller (GSC) is provided that includes a specific position / direction for which the target is unchanged. Adaptive coefficients may be updated to maximize the SNR of the beamformed signal (rather than the target direction signal-to-noise ratio), thereby enabling the target to be transmitted from more directions than the specific (unchanged) target direction.

[0862] We know that |Y| 2 The gradient of is given by

[0863]

[0864] To determine the derivative of this fraction, we use the quotient rule: the derivative of the quotient is the derivative of the denominator times the numerator minus the derivative of the numerator times the denominator, all divided by the square of the denominator, or

[0865]

[0866] We thus use a gradient ascent update (generalized here to M microphones, see C T *,C V *)

[0867]

[0868] in

[0869] in

[0870]

[0871] Note that removing the averaging in the above update assuming that averaging can be done over the β learning rate μ is not feasible (because it is not mathematically correct). However, the MVDR LMS update without averaging is feasible because averaging over the learning rate works well when there is only a single averaging term.

[0872] If we look more closely at the gradient

[0873]

[0874] We see that the numerator and denominator are of the same order (fourth order), and the LMS update is therefore normalized. However, instead of passing a real scalar <|Y V | 2 > 2 Normalization, we can also consider other normalization strategies:

[0875]

[0876] (or alternatively (preferably) re-use the mean calculated from the numerator (same as above))

[0877]

[0878] We can also normalize by averaging across the output signals.

[0879]

[0880] Or, averaging with a target-canceling beamformer:

[0881]

[0882] (or alternatively, apply separate averages (same result))

[0883]

[0884] The LMS update rate can be further increased by adding a momentum term. This momentum term is given by:

[0885]

[0886] Where ρ>1, for example, equal to 1.01 or 1.1, and 0<σ<1, for example, 0.5. It should be noted that if is minimized rather than maximized, these intervals will be swapped. We can also apply the maximum and minimum values ​​that μ can take, that is,

[0887]

[0888] Compared with MVDR

[0889] By considering the gradient,

[0890]

[0891] We see that in the case where the target signal is transmitted from the sight direction, we have C T1 = 0, the gradient is simplified to

[0892]

[0893] This differs from the LMS gradient of the MVDR beamformer by only a scalar.

[0894] Sign LMS

[0895] In practice, it can be difficult to ensure a stable step size, since small values ​​in the denominator of the gradient can dramatically increase the step size. A more stable update is achieved by moving in or approximately in the direction of gradient ascent with a more fixed step size (e.g. using a (complex) sign-LMS algorithm). An approximate but cheaper step in the direction of the gradient is to adjust β by fixed values ​​in the real and imaginary directions, i.e.

[0896]

[0897] In the case of two microphones, the expectation can be omitted:

[0898]

[0899] where Y T and Y V is the nearest available output signal estimate, and is a fixed step size.

[0900] We should also note that we can simply set It is easy to convert a GEV gradient-raised beamformer into a MVDR gradient-raised beamformer.

[0901] momentum

[0902] To increase the convergence rate, the step size can be updated using the momentum term, i.e.

[0903]

[0904] and

[0905]

[0906] Or directly rely on the (gradient ascent) value function

[0907]

[0908] and

[0909]

[0910] where ρ is slightly greater than 1, such as 1.01 or 1.1, and 0<σ<, such as 0.5 or 0.2. We also recommend and Set upper and lower limits.

[0911] The update may also be based on an average value (which is a mathematically correct implementation. However, omitting the average operator has only limited consequences for the two microphones, as the fixed step size ensures):

[0912]

[0913] For a three microphone system, the symbol-based LMS is given by:

[0914]

[0915] and

[0916]

[0917] It should be noted that Y V and Y T Both depend on the most recent estimates of β1 and β2.

[0918] Instant Solution

[0919] We can also simply set And determine the optimal β, that is, for the two-microphone case:

[0920]

[0921] By rearranging these items, we have

[0922]

[0923] We thus determine β by solving a complex second-order polynomial, where the two solutions correspond to The β value corresponding to the maximum or minimum value of .

[0924] The two solutions to the polynomial are given by:

[0925]

[0926] in

[0927]

[0928] An example of SNR plotted as a function of the real and imaginary parts of β is given in Fig. 9 For comparison with the MVDR beamformer, we assume that the target is transmitted from a single direction. In this case, C T1 =0, the above equation simplifies to

[0929]

[0930] We see that the solution reduces to the well-known

[0931]

[0932] In summary, given two target-preserving beamformers C T0 and C V0 and two target elimination beamformers C T1 and C V1 , we can estimate β, where C T0 and C T1 Update if target exists, C V0 and C V1 Updated when target is not present. It is not required that the target is transmitted only from the line of sight, and the output signal is still undistorted relative to the selected steering vector. Whether the current signal is determined to be a target or noise can be determined by a voice activity detector, DOA detector or similar detector.

[0933] Fig. 9 The surface of the loss function L is schematically shown as the real part of the adaptive parameter β and the imaginary part The calculated gradient is shown, and the maximum and minimum values ​​are drawn with ○ and × respectively. Loss function L = |Y T | 2 / |Y v | 2 is mapped as a function of β. The arrow shows the gradient of L with respect to β: This is in contrast to the MVDR scheme, where we only have a single extreme value. In this example, there is a maximum and a minimum because the SNR (due to the full rank covariance matrix) cannot become infinitely large or infinitely small.

[0934] Fig. 10AA first (schematic) example of a hearing device HD, such as a hearing aid, comprising a two-microphone GEV beamformer with gain normalization towards the front direction is shown. The upper signal path schematically shows a GSC structure beamformer (see the block enclosed by the dotted rectangle denoted GSC) based on two microphone inputs (X1, X2) and a target preserving beamformer (a) and a target cancellation beamformer (b1), whose outputs (C0 and C1, respectively) are combined according to an adaptive parameter (β1) (using combining elements 'X' and '+') to provide a spatially filtered (beamformed) signal (Y): Y=C0-C1β1. Where appropriate, the upper signal path (forward path) and (optionally) other parts of the hearing device may (e.g. Fig. 10A (as shown in 10B, 10C)) operates in the time-frequency domain by including an analysis filter bank and a synthesis filter bank.

[0935] Based on a voice activity detector VAD, it enables the detection of speech from the front direction (e.g. by comparison between the front and rear cardioids and the voice activity detector, see Fig. 10A (and the lower signal path in 10B, 10C)) updates the target. It is assumed that the sound of interest is mainly speech from the front half of the face. The comparison between the forward beamformer and the backward beamformer is used to distinguish whether the sound is transmitted from the front or the back.

[0936] A voice activity detector VAD (e.g. implemented using a trained neural network, see e.g. Fig.13 ) is used to distinguish between sounds of interest, such as speech (from any direction), and noise. The combination of VAD and direction-based decision making is to update the target-related beamformer (C T0 ,C T1 ) or update the noise-dependent beamformer (C V0 ,C V1 As an alternative to updating the beamformer, one can consider updating the corresponding covariance matrix (R T or R v ). Based on the updated beamformer product, the adaptive beamformer weights may be updated based on the sign-LMS algorithm (e.g., according to β1), such as Fig. 10A shown on the right.

[0937] Fig. 10B A second example of a hearing device HD, such as a hearing aid, comprising a two-microphone GEV beamformer with gain normalization towards the front direction is shown. Fig. 10B Examples and Fig. 10A The embodiment of is similar. Compared with Fig. 10A, the adaptive coefficient β1 is estimated based on the average between (the product of) different beamformer signals. The average value enables the use of updates based on the LMS algorithm and updates based on instantaneous updates. The beamformer product average is usually denoted by E(·) or <·>. Fig. 10B In the example, E[·] is used. Fig. 10B The penultimate module of the lower path refers to the average beamformer signal product determined in this module. Fig. 10A In , averaging is not indicated. This is not necessary, because the sign LMS algorithm still works well even in cases where the decision based on the sign direction is incorrect (due to the lack of averaging).

[0938] Fig. 10C An example of a hearing device HD, such as a hearing aid, comprising a three-microphone GEV beamformer is shown, wherein the adaptation parameters are updated based on the Sign-LMS algorithm. Fig. 10C Examples and Fig. 10A The embodiment of is similar. Compared with Fig. 10A , Fig. 10C The hearing device comprises three microphones (M1, M2, M3), so a GSC structure beamformer (GSC) is based on the three microphone inputs (X1, X2, X3) and a target preserving beamformer (a) and a target cancellation beamformer (b1, b2), the outputs of which (C0 and (C1, C2) respectively) are combined according to adaptive parameters (β1, β2) to provide a spatially filtered (beamformed) signal (Y): Y = C0-(C1β1+C2β2), which is further processed into a signal audible to the user, as combined Fig. 10A (and e.g. Figure 1B ).

[0939] like Fig. 10C As shown in the lower signal branch of , the front-to-back comparison can be based on two first-order beamformers (b2, c, pointing in opposite directions). Preferably, the two first-order beamformers are based on two microphones aligned in a horizontal plane along the front-to-back axis (see, for example Fig.11 M1 and M2 in Fig. 10C In the embodiment of , the two first order cardioids provided by beamformers b2,c are based on the input signals (X1,X3). Alternatively, the decision can be based on all three available microphone signals (X1,X2,X3). The sign LMS update equation is Fig. 10C The middle right portion of is shown without averaging (E(·) or <·>), but averaging between beamformer products may be applied, for example if NLMS updating is used instead of sign LMS updating.

[0940] Fig. 10CThe voice activity detection (VAD) is shown (or labeled by target (T) and noise (N), see Fig. 10C The arrows "speech sign" and "noise sign" in ) can be spatially based on the comparison between the front and rear cardioids. In this case, the rear cardioid C2 is based on two microphones (it could also have been based on three microphones, as indicated by the dashed line). The rear cardioid C1 (based on microphones M1, M2) could also have been used as input for the comparison with the front cardioid C3 instead of C2, in which case C3 should preferably have been based on the same two microphones as C1. Switching (e.g. gradient) between two and three microphones can be considered (see Figure 8C ), for example because the target cancellation beamformer (b1, b2, c) may be based on two or three microphones (as indicated by the dashed lines).

[0941] Post filter gain

[0942] Since our direct goal is to maximize the SNR, our objective function directly produces an estimate of the signal-to-noise ratio. We can then base the post-filter gains on this SNR estimate. Either as a direct mapping of SNR to per-frequency gain, or we can train a neural network to map our SNR estimates across frequencies to gain values ​​(see, for example, EP3694229A1).

[0943] Avoid R t ＝R v

[0944] In situations with little or no noise or where the signal comes from a single direction, there is a risk that the target covariance matrix and the noise covariance (or the corresponding fixed beamformer) may converge towards the same value. This can be avoided by adding a bias to the covariance matrix, i.e.

[0945]

[0946] and

[0947]

[0948] where σ s and σ v is a small constant. Since the microphone signal will always contain microphone noise, it is most important to add a bias to the target covariance matrix, thereby ensuring that the beamformer system will converge towards an MVDR beamformer in low input level situations.

[0949] Bias can be added to the fixed target cancellation beamformer in a similar manner, i.e.

[0950]

[0951] It should be noted that the target covariance matrix may not necessarily be biased towards a single viewing direction. The bias can also be chosen to be a weighted sum across several target directions θ, i.e.

[0952]

[0953] According to the mean value of the covariance matrix

[0954] Recall that the gradient is given by:

[0955]

[0956] The gradient is given by the average value between the different beamformer signals. We can also express this equation in terms of the covariance matrix. Since Y = (a-Bβ * ) H x and = B H x, we have

[0957]

[0958]

[0959] The above modifications are intended to be used in all embodiments including a beamformer where the SNR of the beamformed signal is maximized (eg, in conjunction with a GEV beamformer or a near-GEV beamformer).

[0960] Fig.11 An embodiment of a hearing aid (HD) according to the invention is schematically shown. The hearing aid comprises a BTE part (BTE) adapted to be located at or behind the outer ear (pinna) of a user. The BTE part comprises a housing in which a microphone system comprising a plurality of microphones (here three microphones (M1, M2, M3) are housed. Each of the microphones (M1, M2, M3) provides an electrical input signal representative of the sound in the environment of the hearing aid (HD). Fig.11In an embodiment, the microphones are located in the housing of the BTE part, one (M1) at the top, one (M3) at the bottom and one (M2) in the middle of the housing of the BTE part. The three microphones are thus spaced apart to the greatest extent and arranged in an angled manner. This microphone configuration can be used to increase the options for directional systems based thereon (e.g., beamformers capable of producing good self-speech beamformers, and beamformers that focus on (or eliminate) sound sources outside the (horizontal) plane determined by the user's ears and nose). The hearing aid also includes an ITE part (ITE) adapted to be located in or at the ear canal of the user. The ITE part includes a speaker unit SU, which includes a speaker that presents processed sound based on an electrical input signal provided by the microphones (M1, M2, M3). A (usually soft) dome-shaped structure DO is connected to the speaker unit SU to guide the speaker unit in the user's ear canal (whereby the position of the speaker unit can be controlled when the speaker unit is installed in the ear canal).

[0961] The BTE part and the ITE part are mechanically and electrically connected via an interconnection element IC, which includes a cable for electrically connecting an electronic circuit (eg a processor) in the BTE part to an electronic circuit (eg a speaker) in the ITE part.

[0962] The processor may include a directional noise reduction system according to the present invention. The processor may be connected to a microphone system. The directional noise reduction system includes at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to multiple (M) electrical input signals. The at least one beamformed signal may be provided, for example, as a weighted sum of multiple (M) electrical input signals provided by the microphone system. The processor may be configured, for example, to adaptively optimize the beamformer weights (w) for multiple target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the multiple target locations, as provided by multiple different aspects of the present invention.

[0963] Fig.11 The hearing aid of the embodiment shown in is a so-called receiver-in-the-ear (RITE) type hearing aid. However, other hearing aid types including multiple input transducers may also benefit from the directional system according to the invention, thereby enabling the presentation of sound signals to the user based on the beamformer including multi-target directional optimization according to the invention. These alternative types may include hearing aids, wherein the output transducer is located in the BTE part, and the interconnection element (IC) between the BTE part and the ITE part includes an acoustic tube to guide the loudspeaker to the ITE part (e.g. an ear mold customized for the user's ear (e.g. ear canal)).

[0964] Fig.12An embodiment of a hearing aid (HD) according to the invention is schematically shown. The hearing aid comprises three microphones (M1, M2, M3) providing respective (time domain) electrical input signals (x1, x2, x3). Each microphone path comprises an analysis filter bank FBA providing a respective time-frequency representation (X1, X2, X3) of the electrical input signal. The hearing aid (HD) further comprises a beamformer (BF) for receiving the time-frequency representation (X1, X2, X3) of the electrical input signal and providing a beamformed signal (Y) according to beamformer weights (w). The beamformer weights (w) are for example based on a target and a noise covariance matrix (R , respectively ) controlled by a voice activity control signal (VAD1) from a voice activity detector. T and R V ) is estimated by maximizing the SNR. Since the acoustic properties of the target signal and the noise signal vary rather slowly, the covariance matrix (or beamformer average) is updated slowly (e.g. using a time constant with a duration of 50 ms or more). The estimated weights (w) from the beamformer module (BF) are used as the SNR estimator (see Fig.12 The input of the SNR(w) module in

[0965]

[0966] Since the estimation (PF-SNR) of the SNR estimator (SNR(w)) must be faster due to the fast changes of the speech signal, the estimation of the covariance matrix is ​​based on a much faster average (e.g., on the order of 1-10 ms). T ,R V ) may be controlled by a different voice activity control signal (VAD2) provided by another voice activity detector. The estimated SNR (PF-SNR) is mapped to a post-filter gain PFG (see the output of the post-filter gain module (PF gain)), which is applied to the beamforming signal (Y) by a multiplication unit (CU, 'X') to provide an output signal (OUT). The post-filter gain module (PF gain) may also include smoothing across time. The post-filter gain module may be implemented using a neural network trained based on examples of SNR estimates and required gain patterns (see, for example, EP3694229A1). The (frequency domain) output signal (OUT) from the combination unit is fed to a synthesis filter bank FBS to provide a corresponding time domain output signal (out). In a typical hearing aid application, the output of the combination unit (CU) including the noise reduced input (audio) signal will be fed to a processor for applying further processing algorithms to the signal to enhance its value for the hearing aid user, for example for increasing the intelligibility of speech in the signal, or for increasing the listening comfort of ambient signals (e.g. music), such as Figure 1BThe processor may be adapted to apply a compression algorithm for compensating for the user's hearing loss.

[0967] Fig.13 An example of a voice activity detector (e.g., an own voice detector) implemented as a neural network (NN) is schematically shown. The (time domain) electrical input signal x provided by the input microphone Mic is converted to the frequency domain (X(k,l), k and l are frequency index and time index, respectively) using an analysis filter bank FBA. The amplitude |X(k,l)| of the complex frequency domain signal X(k,l) is converted to the logarithmic domain (in the dB module), and an optional normalization is applied (in the "Normalization" module, the dashed box indicating "optional"). The normalization module (Normalization) is followed by a neural network layer (NN, exemplified by a gated recurrent unit layer (GRU) and a feedforward layer (FF)). The neural network may include one or more (hidden) layers (e.g., between 2 and 5 layers) between the input layer and the output layer. Between the ("hidden") layers (here GRU, FF) of the neural network, a nonlinear activation function (e.g., implemented as a rectified linear unit (ReLU), a sigmoid function, or a tanh function, etc.) may be applied. At the output layer, an activation function (Activation) (e.g., sigmoid) is applied followed by a threshold function (Threshold). The output of the neural network may be a frequency-dependent VAD control signal indicating "speech", "no speech" (or optionally, "don't know"). Different thresholds may be applied based on voice activity decisions or no speech decisions. The neural network may include Fig.13 The neural network is trained on examples of frequency domain signals, where each time and frequency sample is either labeled as speech, or labeled as no speech, or (optionally) don't care. Alternatively, the label is provided as signal-to-noise ratio (where high SNR corresponds to speech and low SNR corresponds to no speech).

[0968] The input signal of the neural network (NN) is shown as a single microphone signal (here an electrical input signal in the time-frequency (logarithmic) domain, optionally a normalized signal But it can be several signals or a combination of several signals. The output signal of the neural network can be provided as a voice activity control signal (VAD(k,l)) varying with time and frequency, indicating the presence of voice (such as speech) or the absence of voice (or ambiguity) in a given time-frequency unit (k,l) of the current input signal.

[0969] The structural features of the apparatus described above, described in detail in the “Detailed Description of the Invention” and defined in the claims may be combined with the steps of the method of the present invention when appropriately replaced by corresponding processes.

[0970] Unless expressly stated, the singular forms "one", "the" used herein include the plural form (i.e., have the meaning of "at least one"). It should be further understood that the terms "having", "including" and / or "comprising" used in the specification indicate the presence of the described features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or combinations thereof. It should be understood that, unless expressly stated, when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate intervening element. As used herein, the term "and / or" includes any and all combinations of one or more listed related items. Unless expressly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed.

[0971] It should be appreciated that reference to "an embodiment" or "embodiment" or "aspect" or features that "may" include in this specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. In addition, the specific features, structures or characteristics may be appropriately combined in one or more embodiments of the present invention. The foregoing description is provided to enable those skilled in the art to implement the various aspects described herein. Various modifications will be apparent to those skilled in the art.

[0972] The claims are not limited to the various aspects shown herein, but rather have the full scope consistent with the claim language wherein, unless expressly stated otherwise, elements referred to in the singular do not mean "one and only one" but rather "one or more." Unless expressly stated otherwise, the term "some" means one or more.

[0973] Examples of methods and products (hearing aids) according to the invention are presented in the following projects:

[0974] Item 1. A hearing aid suitable for being worn by a user, the hearing aid comprising:

[0975] - a microphone system comprising a plurality (M) of microphones adapted to pick up sounds from a user environment and provide a corresponding electrical input signal, wherein M is greater than or equal to 2;

[0976] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights configured to be applied to a plurality of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality of electrical input signals; and

[0977] wherein the beamformer weights are adaptively optimized for multiple target locations by maximizing a target signal-to-noise ratio of sounds from the multiple target locations, wherein the signal-to-noise ratio is determined based on first and second output variances of the at least one beamformer or time-averaged versions of the first and second output variances, wherein the first and second output variances are determined when the electrical input signal or the at least one beamformed signal is labeled as target (T) and noise (V), respectively.

[0978] Item 2. The hearing aid according to item 1, comprising a plurality of analysis filter banks configured to provide the electrical input signal in a time-frequency representation (k, l), where k is a frequency index and l is a time index.

[0979] Item 3. The hearing aid according to Item 1, comprising a target signal detector configured to provide an indication to the timing frequency unit (k, l) whether or with what probability it includes a target signal, such as speech.

[0980] Item 4. The hearing aid according to Item 1, comprising a voice activity detector for estimating whether or with what probability at least one of the electrical input signals includes a voice signal at a given point in time and providing a voice activity control signal indicating the estimation result.

[0981] Item 5. A hearing aid according to Item 4, wherein the voice activity detector is based on or includes an artificial neural network.

[0982] Item 6. A hearing aid according to item 4, wherein the voice activity control signal further depends on spatial information derived from the electrical input signal or a signal derived therefrom.

[0983] Item 7. The hearing aid of Item 6, wherein the spatial information is derived from a comparison between a beamformer having maximum sensitivity toward the front half and another beamformer having sensitivity toward the rear half.

[0984] Item 8. The hearing aid according to Item 4, wherein the electrical input signal or at least one beamforming signal is labeled as target (T) and noise (V) respectively according to the voice activity control signal.

[0985] Item 9. The hearing aid according to Item 1, wherein at least one beamformer is implemented as a linear combination of two or more predetermined or adaptively determined beamformers.

[0986] Item 10. The hearing aid of Item 9, wherein the first predetermined or adaptively determined beamformer is configured to have a spatial maximum toward a specific target position among the plurality of target positions.

[0987] Item 11. The hearing aid of Item 9, wherein the one or more second predetermined or adaptively determined beamformers are configured to have a spatial minimum towards a corresponding target position of the plurality of target positions.

[0988] Item 12. The hearing aid of Item 1, wherein the at least one beamformer is implemented as a generalized sidelobe canceller (GSC) for providing at least one beamformed signal according to adaptively determined beamformer weights, wherein the generalized sidelobe canceller comprises M fixed beamformers, one of the M fixed beamformers is a target signal preserving beamformer, M-1 fixed beamformers are target cancellation beamformers, and each beamformer is configured to provide at least one beamformed signal according to an associated fixed beamformer weight (w F,m , m=1,…,M) generates a beamforming signal, wherein the adaptively determined beamformer weights are determined according to an adaptive parameter β or a parameter vector β.

[0989] Item 13. The hearing aid according to Item 12, wherein the adaptive parameter β or the parameter vector β is determined based on the time average of the target preserving beamformer and M-1 target cancellation beamformers.

[0990] Item 14. The hearing aid of Item 1, wherein the first output variance is determined based on a target direction determined before normal use of the hearing aid.

[0991] Item 15. A hearing aid suitable for being worn by a user, the hearing aid comprising:

[0992] - a microphone system comprising a plurality (M) of microphones adapted to pick up sounds from a user's environment and to provide a corresponding electrical input signal (x), wherein M is greater than or equal to 2; and

[0993] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing at least one beamformed signal (Y) as a weighted sum (Y=(w)) of the plurality (M) of electrical input signals; H x)); and

[0994] wherein the beamformer weights (w) are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sound from the plurality of target locations (θ), wherein the SNR is determined based on the beamformer weights (w) and a plurality of electrical input signals x and x H The time-averaged inner vector product of <xx H > T and <xx H > VDetermine, where <.> refers to the average across time, where <xx H > T and <xx H > V Determined when the electrical input signal (x) is labeled as target (T) and noise (V), respectively.

[0995] Item 16. A hearing aid adapted to be worn by a user, the hearing aid comprising:

[0996] - a microphone system comprising a plurality (M) of microphones adapted to pick up sounds from a user's environment and to provide a corresponding electrical input signal (x), wherein M is greater than or equal to 2; and

[0997] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing at least one beamformed signal (Y) as a weighted sum (Y=(w)) of the plurality (M) of electrical input signals; H x)); and

[0998] wherein the beamformer weights (w) are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations (θ), wherein the SNR is determined based on the beamformer weights (w) and the corresponding inter-microphone target covariance matrix (R S ) and the inter-microphone noise covariance matrix (R V ) to confirm.

[0999] Item 17. A hearing aid adapted to be worn by a user, the hearing aid comprising:

[1000] - comprising a device adapted to pick up sound from the user's environment and provide a corresponding electrical input signal x m A microphone system of a plurality of (M) microphones having (n), m=1, ..., M, where n represents time and M is greater than or equal to 2; and

[1001] a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights (w) configured to be applied to a plurality (M) of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality (M) of electrical input signals;

[1002] wherein the beamformer weights (w) are adaptively optimized for a plurality of target locations (θ) by maximizing a target signal-to-noise ratio (SNR) of sounds from the plurality of target locations, wherein the SNR is determined based on the beamformer weights (w), an inter-microphone target covariance matrix (RT ) and the inter-microphone noise covariance matrix (R V ) is represented by the target covariance matrix (R T ) according to the multiple steering vectors (d θ )Sure.

[1003] Item 18. The hearing aid of Item 17, wherein the target signal-to-noise ratio (SNR) is determined according to the following expression:

[1004]

[1005] where H refers to the Hermitian transpose.

[1006] Item 19. The hearing aid according to Item 17, configured to adaptively optimize the beamformer weights (w) by estimating the beamformer weights as the eigenvectors belonging to the largest generalized eigenvalues.

[1007] Item 20. The hearing aid of Item 17, wherein at least one beamformer comprises a generalized eigenvector beamformer (GEV) or an approximation thereof.

[1008] References

[1009] ·EP3413589A1(Oticon)12.12.2018.

[1010] ·EP3300078A1(Oticon)28.03.2018.

[1011] ·EP3252766A1(Oticon)06.12.2017.

[1012] ·EP3694229A1(Oticon)12.08.2020.

[1013] ·[Bitzer&Simmer; 2001] J.Bitzer and KUSimmer, "SuperdirectiveMicrophone Arrays," in "Microphone Arrays–Signal Processing Techniques," M.Brandstein and D.Wards (Eds.), Springer-Verlag, 2001, Chapter 2.

[1014] ·[Simmer et al.;2001]Simmer,K.U.,Bitzer,J.,&Marro,C.”Post-filteringtechniques”,in“Microphone Arrays–Signal Processing Techniques,”M.Brandsteinand D.Wards(Eds.),Springer-Verlag,2001,Chapter 3.

[1015] ·[Souden et al.;2010]Souden,M.,Benesty,J.,and Affes,S.,"On OptimalFrequency-Domain Multichannel Linear Filtering for Noise Reduction",IEEEAudio Speech and Language Processing vol.18,no.2,2010,pp.260-276.

[1016] ·[Warsitz&Haeb-Umbach;2007]E.Warsitz and R.Haeb-Umbach,“Blindacoustic beamforming based on generalized eigenvalue decomposition”,IEEETransactions on Audio,Speech,and Language Processing,vol.15,no.5,pp.1529–1539,2007.

Claims

1. A hearing aid adapted to be worn by a user, the hearing aid comprising: - a microphone system comprising a plurality, M, of microphones adapted to pick up sounds from a user's environment and provide a corresponding electrical input signal, wherein M is greater than or equal to 2; a directional noise reduction system connected to the microphone system, the directional noise reduction system comprising at least one beamformer for generating at least one beamformed signal according to beamformer weights configured to be applied to a plurality of electrical input signals, thereby providing the at least one beamformed signal as a weighted sum of the plurality of electrical input signals; and wherein the beamformer weights are adaptively optimized for multiple target locations by maximizing a target signal-to-noise ratio of sounds from the multiple target locations, wherein the signal-to-noise ratio is determined based on first and second output variances of the at least one beamformer or time-averaged versions of the first and second output variances, wherein the first and second output variances are determined when the electrical input signal or the at least one beamformed signal is labeled as target (T) and noise (V), respectively.

2. The hearing aid according to claim 1, comprising a plurality of analysis filter banks configured to provide the electrical input signal in a time-frequency representation (k, l), where k is a frequency index and l is a time index.

3. The hearing aid according to claim 1, comprising a target signal detector configured to provide an indication to the timing frequency unit (k, l) whether or with what probability it comprises a target signal.

4. The hearing aid according to claim 1, comprising a voice activity detector for estimating whether or with what probability at least one of the electrical input signals comprises a voice signal at a given point in time and providing a voice activity control signal indicating the estimation result.

5. The hearing aid of claim 4, wherein the voice activity detector is based on or comprises an artificial neural network.

6. A hearing aid according to claim 4, wherein the voice activity control signal further depends on spatial information derived from the electrical input signal or a signal derived therefrom.

7. The hearing aid of claim 6, wherein the spatial information is derived from a comparison between a beamformer having maximum sensitivity towards the front half and another beamformer having sensitivity towards the rear half.

8. The hearing aid according to claim 4, wherein the electric input signal or at least one beamforming signal is labeled as target (T) and noise (V), respectively, according to the voice activity control signal.

9. The hearing aid according to claim 1, wherein at least one beamformer is implemented as a linear combination of two or more predetermined or adaptively determined beamformers.

10. The hearing aid according to claim 9, wherein the first predetermined or adaptively determined beamformer is configured to have a spatial maximum towards a specific target position of the plurality of target positions.

11. The hearing aid according to claim 9, wherein the one or more second predetermined or adaptively determined beamformers are configured to have a spatial minimum towards a corresponding target position of the plurality of target positions.

12. The hearing aid of claim 1 , wherein the at least one beamformer is implemented as a generalized sidelobe canceller (GSC) for providing at least one beamformed signal according to adaptively determined beamformer weights, wherein the generalized sidelobe canceller comprises M fixed beamformers, one of the M fixed beamformers is a target signal preserving beamformer, M-1 fixed beamformers are target cancellation beamformers, each beamformer is configured to provide at least one beamformed signal according to an associated fixed beamformer weight (w F,m , m=1,…,M) generates a beamforming signal, wherein the adaptively determined beamformer weights are determined according to an adaptive parameter β or a parameter vector β.

13. The hearing aid according to claim 12, wherein the adaptive parameter β or the parameter vector β is determined based on the time average of the target preserving beamformer and M-1 target cancellation beamformers.

14. The hearing aid of claim 1, wherein the first output variance is determined based on a target direction determined before normal use of the hearing aid.

Images