Hearing aid with determination of turn-taking

Abstract

A hearing aid for determining a turn-taking is disclosed, comprising: an input unit for receiving an input sound signal from an environment of a hearing aid user and providing at least one electrical input signal representing the input sound signal; an output unit for providing at least one set of stimuli perceivable as sound to the hearing aid user based on a processed version of the at least one electrical input signal; a voice activity detector (VAD) configured to determine speech in the input sound signal; an own voice detector (OVD) configured to determine own voice of the hearing aid user in the input sound signal; a processing unit connected to the input unit and the output unit and comprising signal processing parameters of the hearing aid to provide the processed version of the at least one electrical input signal; a turn-taking determination unit configured to determine a turn-taking behavior of the hearing aid user; wherein the processing unit is configured to adjust the signal processing parameters based on the determined turn-taking behavior of the hearing aid user.

Description

Technical Field

[0001] This application relates to hearing aids suitable for wearing in or at the ear of a hearing aid user and / or suitable for complete or partial implantation in the head of a hearing aid user. This application also relates to hearing systems comprising first and second hearing aids. Background Technology

[0002] Current hearing aids are optimized to help users communicate (conversation) in the best possible way under different acoustic settings. However, there is a lack of information from the user to the hearing aid about how well the hearing aid can support the user: how good is speech intelligibility, and / or how high is the listening effort in the current situation, what is the state of fatigue, and / or when the hearing aid withdraws from or participates in the conversation?

[0003] Attempts have been made to provide this type of information to hearing aids, for example, through physiological measurements such as the use of electroencephalography (EEG) sensors. However, these attempts require additional, specialized hardware not currently available in hearing aids, and there has been no attempt to use direct behavioral measurements of hearing aid users.

[0004] Turn-taking is a term characterizing the dynamic behavior of a conversation or dialogue between two people. When two people turn-taking in a conversation, it is quantified by the pauses or overlaps in their speech. Research indicates that this stimulus-response pattern depends on the hearing loss of the person and the type of noise. When one of the two speakers is hearing-impaired, overlap decreases and pauses increase. Furthermore, speech productivity decreases as the hearing-impaired person speaks more slowly. Pauses further increase and speech rate further decreases as hearing aid users must exert more auditory effort and may become fatigued.

[0005] Therefore, there is a need for a hearing aid and hearing system configured to measure turn-taking and use it as a real-time speech outcome metric to modify hearing aid settings in real time. Summary of the Invention

[0006] In one aspect of this application, a hearing aid suitable for wearing in or at the ear of a hearing aid user and / or suitable for being fully or partially implanted in the head of a hearing aid user is provided.

[0007] The hearing aid may include an input unit for receiving an input sound signal from the environment of the hearing aid user and providing at least one electrical input signal representing the input sound signal.

[0008] The input sound signal may include speech components originating from one or more speech sources. The input sound signal may include sound containing noise signal components. The noise signal components may be generated from one or more noise sources. The electrical input signal may represent the sound in the hearing aid user's environment.

[0009] The input unit may include an input transducer, such as a microphone, for converting an input audio signal into an electrical input signal. The input unit may also include a wireless receiver for receiving wireless signals that include or represent sound and providing an electrical input signal representing said sound. The wireless receiver may, for example, be configured to receive electromagnetic signals in the radio frequency range (3 kHz to 300 GHz). The wireless receiver may, for example, be configured to receive electromagnetic signals in the optical frequency range (e.g., infrared light 300 GHz to 430 THz, or visible light, e.g., 430 THz to 770 THz).

[0010] The hearing aid may include an output unit for providing a hearing aid user with at least one set of stimuli that can be perceived as sound, based on a processed version of at least one electrical input signal.

[0011] Hearing aids may include an output unit for providing stimulation, perceived as an acoustic signal by the hearing aid user, based on processed electrical signals. The output unit may include multiple electrodes of a cochlear implant (for CI-type hearing aids) or a vibrator of a bone conduction hearing aid. The output unit may include an output transducer. The output transducer may include a receiver (speaker) for providing the stimulation as an acoustic signal to the user (e.g., in acoustic (air conduction-based) hearing aids). The output transducer may also include a vibrator for providing the stimulation as mechanical vibrations of the skull to the user (e.g., in bone-attached or bone-anchored hearing aids).

[0012] Hearing aids may include a speech activity detector (VAD) configured to determine speech in an input sound signal.

[0013] VAD can be configured to repeatedly estimate whether or with what probability that at least one electrical input signal or a signal derived therefrom includes speech (voice signal).

[0014] For example, VAD can detect when speech is present in an input audio signal, for example, by looking at the synchronous modulation in multiple harmonic bands.

[0015] Thus, speech included in at least one electrical input signal can be enhanced.

[0016] Hearing aids may include an oral self-voice detector (OVD) configured to determine the user's own voice in the input sound signal.

[0017] OVD can be configured to repeatedly estimate whether or with what probability that at least one electrical input signal or a signal derived therefrom includes the speech of the source self-hearing device user.

[0018] OVD can indicate at least one electrical input signal or the source of the signal from which the self-speaking user's voice originates, or the source of the self-speaking user's voice with a probability higher than the self-voice presence probability (OVPP) threshold.

[0019] For example, OVD can detect self-voice based on the proximity effect that causes a small level difference between the microphones of a hearing aid.

[0020] The microphone system of a hearing aid can be adapted to distinguish the user's own voice from another person's voice, and possibly from non-voice sounds.

[0021] A hearing aid may include at least one processing unit.

[0022] The processing unit can be connected to the input unit.

[0023] The processing unit can be connected to the output unit.

[0024] The processing unit may include signal processing parameters of the hearing aid to provide a processed version of at least one electrical input signal.

[0025] Hearing aids may include a turn-taking determination unit.

[0026] The processing unit of a hearing aid may include a turn-by-turn determination unit.

[0027] The turn-taking determination unit can be configured to determine the turn-taking behavior of a hearing aid user.

[0028] Determining a hearing aid user's turn-taking behavior may include identifying pauses between the hearing aid user's and another person's speech. These pauses can be determined based on speech identified by the VAD (Voice over Action) and self-voice identified by the OVD (On-Voice over Action).

[0029] For example, based on VAD and OVD, discontinuity can be measured as the time period between when both VAD and OVD detect speech (T1) and when only VAD detects speech (T2), i.e., T1 (VAD on, OVD on) – T2 (VAD on, OVD off).

[0030] Determining a hearing aid user's turn-taking behavior may include identifying overlap between the hearing aid user's speech and that of another person. Overlap determination can be based on speech identified by VAD and self-voice identified by OVD.

[0031] Discontinuities can be determined by averaging across time periods / intervals (e.g., 2, 3, 4 minutes or longer), resulting in median discontinuities to increase the certainty of the measured discontinuities.

[0032] This allows us to determine the turn-taking pattern of the conversation between the hearing aid user and another person.

[0033] The processing unit can be configured to adjust and / or correct signal processing parameters based on the turn-switching behavior of a determined hearing aid user.

[0034] The processing unit can be configured to adjust and / or correct signal processing parameters in real time.

[0035] Adjusting and / or correcting signal processing parameters may include adjusting and / or correcting gain (e.g., including providing a mask), noise reduction, enhancement (e.g., spectral shaping), and / or other signal processing-related parameters.

[0036] For example, based on a defined turn-turning behavior, the hearing effort of a hearing aid user is estimated to be high because the defined interruptions are above an interruption threshold (large turn-turning interruption, e.g., >300ms) and / or the speech rate is below a speech rate threshold (slow speech rate, e.g., <4 syllables / second). The hearing aid (such as the hearing aid's processing unit) can be configured to, for example, enable beamforming.

[0037] For example, based on a given turn-taking behavior, the hearing effort of a hearing aid user is estimated to be low because the defined interruptions are below an interruption threshold (small turn-taking interruption) and / or the speech rate is above a speech rate threshold (fast speech rate). The hearing aid (such as the hearing aid's processing unit) can be configured, for example, to reduce high-frequency gain.

[0038] Therefore, turn-taking behavior and / or patterns can be used to modify the signal processing parameters of the hearing aid in real time.

[0039] The hearing aid can be configured to activate the turn-by-turn determination unit.

[0040] The hearing aid can be configured to activate the turn-by-turn determination unit when the VAD determines the speech and the OVD determines the hearing aid user's own voice in the input sound signal.

[0041] The hearing aid can be configured to disable the turn-by-turn determination unit.

[0042] The hearing aid can be configured to disable the turn-taking determination unit when, at a first time interval, e.g., >2 seconds, the voice input method (VAD) fails to determine the speech and the voice input method (OVD) fails to determine the hearing aid user's own voice in the input sound signal.

[0043] Therefore, the turn-by-turn determination unit can only be activated when needed, otherwise it is disabled, resulting in reduced power consumption and reduced processing power utilization.

[0044] Hearing aids may include a modulation filter. The modulation filter can be configured to determine the speech rate of the hearing aid user.

[0045] Determining the speech rate of a hearing aid user can be done by calculating the envelope of the speech signal (determined by VAD and OVD) and then low-pass filtering the speech signal at 15 Hz, resulting in a modulation spectrum. Peaks in the determined modulation spectrum that relate to the hearing aid user's self-voice can then be analyzed to determine the hearing aid user's speech rate (typically 4-5 Hz).

[0046] This provides simplified speech rate determination and enhanced turn-taking determination.

[0047] The hearing aid may also include a signal-to-noise ratio (SNR) estimator. The SNR estimator can be configured to determine the SNR in the hearing aid user's environment. The SNR may be determined based on the input sound signal and / or based on a processed version of at least one electrical input signal.

[0048] The hearing aid can be configured to disable (turn off) turn-by-turn determination when the SNR is below a first SNR threshold, such as <-3dB.

[0049] This reduces power consumption.

[0050] Hearing aids may also include sound pressure level (SPL) estimators.

[0051] The SPL estimator can be configured to measure the sound level at an input unit (such as an input converter).

[0052] The SPL estimator can be configured to measure the SPL at the ear canal microphone.

[0053] The SPL estimator can be configured to determine the SPL of a speaker's own voice and that of another speaker to determine the SPL difference between speakers. This is another useful idea for self-voice-other-voice determination.

[0054] Hearing aids may also include timers.

[0055] The timer can be configured to determine the start point for the turn-around.

[0056] For example, turn-taking can be determined to begin at discrete points in time.

[0057] For example, turn-taking can be determined at discrete time points, as long as the VAD and / or OVD detects speech and self-voice.

[0058] For example, a timer can be configured to determine a specific time period during which a turn-around can occur.

[0059] The hearing aid can be configured to activate the turn-by-turn determination unit when a timer determines the start point.

[0060] For example, when the timer determines the start point of the turn-by-turn determination, the hearing aid (and / or the hearing aid's processing unit) can be configured to activate the turn-by-turn determination unit, for example, by energizing the turn-by-turn determination unit.

[0061] For example, when the VAD and / or OVD does not detect speech, the hearing aid (and / or the hearing aid's processing unit) can be configured to disable the turn-by-turn determination unit, for example, by powering off the turn-by-turn determination unit.

[0062] Timer results can be averaged over a longer period of time to obtain more reliable intermittent measurement results.

[0063] As a result, the processing power and power consumption of the hearing aid are minimized.

[0064] Hearing aids may include a memory unit.

[0065] The memory unit can be configured to store reference signal processing parameters for the processing unit.

[0066] The processing unit can be configured to apply reference signal processing parameters when the OVD has not determined its own voice in the second time period, such as >10s.

[0067] Reference signal processing parameters can refer to the individualized signal processing parameters provided to the hearing aid's processing unit when the hearing aid is initially handed over to the hearing aid user. Reference signal processing parameters can also refer to individualized signal processing parameters adjusted during subsequent use of the hearing aid, for example, based on input from the hearing aid user. For example, individualized signal processing parameters can be adjusted through machine learning based on input from the user and / or based on monitored user behavior or preferences.

[0068] Reference signal processing parameters refer to the optimal signal processing parameters of the hearing aid during normal use.

[0069] The second time period can be timed and / or determined by a timer. For example, the second time period can be at least 5 seconds.

[0070] Therefore, when the hearing aid user is not in a turn-taking situation, the hearing aid can quickly adjust the signal processing parameters to the proven and generally preferred settings.

[0071] Hearing aids may include accelerometers.

[0072] The accelerometer can be configured to detect movement of the hearing aid user. The accelerometer can be configured to detect movement in the vertical and / or horizontal directions. The accelerometer can be configured to detect movement and / or acceleration and / or orientation and / or position of the hearing aid.

[0073] For example, an accelerometer can sense a hearing aid user's self-voice activity by, for example, detecting the fundamental frequency of the hearing aid user's self-voice and / or detecting jaw movements caused by the hearing aid user speaking.

[0074] For example, an accelerometer can pick up vibrations of the acoustic ligaments of a hearing aid user (in the form of F0, also known as pitch).

[0075] Hearing aids may include ear canal microphones.

[0076] The in-ear microphone can be placed in the ear canal of a hearing aid user and can record the user's own voice caused by the occlusion effect.

[0077] Hearing aids may include accelerometers and in-ear microphones.

[0078] OVD can be configured to determine the hearing aid user's self-voice based on an accelerometer.

[0079] OVD can be configured to determine the hearing aid user's self-voice based on the in-ear microphone.

[0080] OVD can be configured to determine the hearing aid user's self-voice based on an accelerometer and an in-ear microphone.

[0081] OVD may include an accelerometer and / or an ear canal microphone.

[0082] Hearing aids can be configured to determine the user's self-voice based on an accelerometer and an in-ear microphone. For example, when the accelerometer detects movement of the user and the in-ear microphone picks up the user's self-voice with the aid of occlusion, it can be determined that the user is speaking.

[0083] Therefore, it can provide highly deterministic self-voice detection.

[0084] Hearing aids may include an inertial measurement unit.

[0085] An inertial measurement unit (IMU) can refer to an electronic device configured to measure and report specific forces, angular rates, and / or body orientation of a hearing aid user. The IMU may be configured to perform these measurements based on a combination of accelerometers, gyroscopes, and / or magnetometers.

[0086] The inertial measurement unit can be configured to follow the head movements / rotations of the hearing aid user.

[0087] Therefore, the head movements / rotations and attention of hearing aid users can be used to determine the user's turn-taking behavior.

[0088] Hearing aids can be configured to transmit specific turn-taking behaviors of a hearing aid user to a server device. The transmission can be partially or fully wired and / or wireless.

[0089] Server devices can be configured to adjust reference signal processing parameters based on talk-turn behavior.

[0090] The server device can be configured to store the preferences of hearing aid users in different turn-taking scenarios (e.g., provided by the hearing aid user via the hearing aid).

[0091] Server equipment can refer to cloud servers.

[0092] Hearing aids can be configured to transmit determined turn-taking behavior to server equipment via an assistive device. The assistive device can refer to a mobile phone, a hearing aid docking station (such as a charging unit), and / or a local server equipment (such as one in the hearing aid user's home) configured to transmit determined turn-taking behavior (e.g., via a network) to the server equipment.

[0093] Therefore, the reference signal processing parameters of the hearing aid and / or user preferences can be easily restored whenever needed.

[0094] Hearing aids can be configured to receive regulated reference signal processing parameters from server equipment.

[0095] Hearing aids can be configured to store adjusted reference signal processing parameters on the hearing aid's memory unit.

[0096] The hearing aid can be configured to receive regulated reference signal processing parameters whenever it is available and / or whenever the hearing aid is connected to the server device.

[0097] Therefore, hearing aids can always operate based on optimal (e.g., adjusted / suitable for user preferences) signal processing parameters.

[0098] Hearing aids may be adapted to provide frequency-varying gain and / or level-varying compression and / or frequency shift (with or without frequency compression) of one or more frequency ranges to one or more other frequency ranges to compensate for the user’s hearing loss.

[0099] Hearing aids may include directional microphone systems adapted to spatially filter sound from the environment, thereby enhancing a target sound source among multiple sound sources in the local environment of the hearing aid wearer. The directional system is adapted to detect (e.g., adaptive detection) the direction from which a specific portion of the microphone signal originates. This can be achieved, for example, in a variety of different ways described in the prior art. In hearing aids, microphone array beamformers are commonly used to spatially attenuate background noise sources. Many beamformer variations can be found in the literature. Minimum variance distortionless response (MVDR) beamformers are widely used in microphone array signal processing. Ideally, an MVDR beamformer keeps the signal from the target direction (also known as the line of sight) unchanged while attenuating sound signals from other directions to the greatest extent possible. A generalized sidelobe canceller (GSC) structure is an equivalent representation of an MVDR beamformer, offering computational and digital representation advantages over a direct implementation of the original form.

[0100] Hearing aids may include antenna and transceiver circuitry (such as a wireless receiver) for receiving direct electrical input signals from another device, such as an entertainment device (e.g., a television), a communication device, a wireless microphone, or another hearing aid. The direct electrical input signals may represent or include audio signals and / or control signals and / or information signals. Hearing aids may include demodulation circuitry for demodulating the received direct electrical input signals, thereby providing direct electrical input signals representing audio signals and / or control signals, such as for setting operating parameters (e.g., volume) and / or processing parameters of the hearing aid. Generally, the wireless link established by the antenna and transceiver circuitry of the hearing aid can be of any type. The wireless link is established between two devices, such as between an entertainment device (e.g., a TV) and a hearing aid, or between two hearing aids, such as via a third intermediate device (e.g., a processing device, such as a remote control, smartphone, etc.). The wireless link is used under power-limited conditions, for example because the hearing aid may be constituted of or include a portable (typically battery-powered) device. Wireless links can be based on near-field communication, such as inductive links based on inductive coupling between the antenna coils of the transmitter and receiver sections. Wireless links can also be based on far-field electromagnetic radiation. Communication via a wireless link is arranged according to a specific modulation scheme, such as analog modulation schemes like FM (Frequency Modulation), AM (Amplitude Modulation), or PM (Phase Modulation), or digital modulation schemes like ASK (Amplitude Shift Keying) such as On-Key, FSK (Frequency Shift Keying), PSK (Phase Shift Keying) such as MSK (Minimum Frequency Shift Keying) or QAM (Quadrature Amplitude Modulation), etc.

[0101] Communication between a hearing aid and another device can be in baseband (an audio frequency range, such as between 0 and 20 kHz). Preferably, communication between hearing aids is based on a type of modulation at frequencies above 100 kHz. Preferably, the frequency used to establish a communication link between the hearing aid and the other device is below 70 GHz, for example, in the range from 50 MHz to 70 GHz, for example, above 300 MHz, for example, in the ISM range above 300 MHz, for example, in the 900 MHz range, or in the 2.4 GHz range, or in the 5.8 GHz range, or in the 60 GHz range (ISM = Industrial, Scientific and Medical, such standardized ranges are defined, for example, by the International Telecommunication Union ITU). The wireless link is based on standardized or proprietary technologies. The wireless link is based on Bluetooth technology (such as Bluetooth Low Energy).

[0102] Hearing aids and / or communication devices may include electrically small antennas. In this specification, "electrically small antenna" means an antenna whose spatial extension (such as its maximum physical size in any direction) is much smaller than the wavelength λ of the transmitted electrical signal. Tx The spatial extension of the antenna is a factor of 10, 50, 100, or more, for example, 1000 or more, smaller than the carrier wavelength λ of the transmitted signal. TxHearing aids are relatively small devices. In this specification, "relatively small device" means that its maximum physical size (and therefore the maximum physical size of the antenna used to provide the wireless interface to the hearing aid) is less than 10 cm, such as less than 5 cm. In this specification, "relatively small device" can also mean that its maximum physical size is much smaller than the operating wavelength of the wireless interface to which the antenna is intended to connect (e.g., more than 3 times smaller, more than 10 times smaller, more than 20 times smaller) (ideally, an antenna used to radiate electromagnetic waves at a given frequency should be greater than or equal to half the wavelength of the radiated wave at that frequency). At 860 MHz, the vacuum wavelength is approximately 35 cm. At 2.4 GHz, the vacuum wavelength is approximately 12 cm. Hearing aids have a maximum external size in the 0.15 m class (e.g., a handheld mobile phone). Hearing aids have a maximum external size in the 0.08 m class (e.g., headphones). Hearing aids have a maximum external size in the 0.04 m class (e.g., hearing instruments).

[0103] Hearing aids can be portable (i.e., configured to be wearable) devices or integral to them, such as devices that include an internal power source, such as a battery, for example a rechargeable battery. Hearing aids can be lightweight, easy-to-wear devices, for example, having a total weight of less than 100g.

[0104] Hearing aids may include a forward or signal path between an input unit (such as an input converter, for example a microphone or microphone system and / or a direct electrical input (such as a wireless receiver)) and an output unit such as an output converter. A signal processor is located in this forward path. The signal processor is adapted to provide frequency-varying gain according to the user's specific needs. Hearing aids may include an analysis path with functionalities for analyzing the input signal (such as determining level, modulation, signal type, acoustic feedback estimate, etc.). Some or all of the signal processing of the analysis path and / or signal path may be performed in the frequency domain. Some or all of the signal processing of the analysis path and / or signal path may be performed in the time domain.

[0105] Analog electrical signals representing sound signals can be converted into digital audio signals during analog-to-digital (AD) conversion, where the analog signal is sampled at a predetermined sampling frequency or sampling rate f. s Perform sampling, f s For example, in the range from 8kHz to 48kHz (to suit specific application needs) at discrete time points t n (or n) provides digital samples x n (or x[n]), each audio sample passes through a predetermined N b Bit represents the acoustic signal at t n The value of N at time b For example, in a range from 1 to 48 bits, such as 24 bits. Each audio sample therefore uses N. b Bit quantization (resulting in 2^n voltammetry of audio samples) Nb(Number of different possible values). The numerical sample x has 1 / f s The duration of the time, such as 50 μs, for f s =20kHz. Multiple audio samples can be arranged in time frames. A time frame can include 64 or 128 audio data samples. Other frame lengths can be used depending on the application.

[0106] Hearing aids may include analog-to-digital (AD) converters to digitize analog inputs (e.g., from an input converter such as a microphone) at a predetermined sampling rate such as 20 kHz. Hearing aids may also include digital-to-analog (DA) converters to convert digital signals into analog output signals, for example, for presentation to the user via an output converter.

[0107] Hearing aids, such as input units and / or antenna and transceiver circuitry, include a TF-T converter unit for providing a time-frequency representation of the input signal. The time-frequency representation may include an array or mapping of corresponding complex or real values ​​of the signal in question over a specific time and frequency range. The TF-T converter unit may include a filter bank for filtering the (time-varying) input signal and providing multiple (time-varying) output signals, each output signal encompassing a distinctly different frequency range of the input signal. The TF-T converter unit may include a Fourier transform unit for converting the time-varying input signal into a (time-)frequency signal. The hearing aid considers a frequency range starting from the minimum frequency f. min up to the maximum frequency f max The frequency range can include a portion of the typical human hearing range from 20Hz to 20kHz, such as a portion of the range from 20Hz to 12kHz. Typically, the sampling rate f... s Greater than or equal to the maximum frequency f max twice, that is, f s ≥2f max The signals from the forward and / or analysis pathways of the hearing aid can be divided into NI (e.g., uniformly wide) frequency bands, where NI is, for example, greater than 5, greater than 10, greater than 50, greater than 100, or greater than 500, and at least some of them are processed individually. The hearing aid is adapted to process the signals from the forward and / or analysis pathways (NP≤NI) in NP different channels. The channels can be of uniform or inconsistent width (e.g., width increases with frequency), overlapping or non-overlapping.

[0108] Hearing aids can be configured to operate in different modes, such as a normal mode and one or more specific modes, which may be user-selectable or automatically selectable. Operating modes can be optimized for specific acoustic conditions or environments. Operating modes may include low-power modes, where the hearing aid's functionality is reduced (e.g., for energy saving), such as disabling wireless communication and / or disabling specific features of the hearing aid.

[0109] Hearing aids may include multiple detectors configured to provide status signals relating to the hearing aid's current network environment (such as the current acoustic environment), and / or the current state of the user wearing the hearing aid, and / or the current state or operating mode of the hearing aid. Alternatively or additionally, one or more detectors may form part of an external device that communicates with the hearing aid (e.g., wirelessly). External devices may include, for example, another hearing aid, a remote control, an audio transmission device, a telephone (e.g., a smartphone), external sensors, etc.

[0110] One or more of a plurality of detectors can operate on a full-band signal (time domain). One or more of a plurality of detectors can operate on a band-split signal ((time-)frequency domain), for example, in a finite number of frequency bands.

[0111] Multiple detectors may include level (L) detectors for estimating the current level of the signal in the forward path. Detectors may be configured to determine whether the current level of the signal in the forward path is higher or lower than a given (L-) threshold. Level detectors operate on full-band signals (time domain). Level detectors operate on band-split signals ((time-)frequency domain).

[0112] The hearing aid may include a classification unit configured to classify the current situation based on input signals from (at least partially) a detector and possibly other inputs. In this specification, "current situation" is defined by one or more of the following:

[0113] a) Physical environment (including the current electromagnetic environment, such as the presence of electromagnetic signals (including audio and / or control signals) that are planned or unplanned to be received by the hearing aid, or other properties of the current environment that are different from acoustics);

[0114] b) Current acoustic conditions (input level, feedback, etc.);

[0115] c) The user's current mode or state (movement, temperature, cognitive load, etc.);

[0116] d) The current mode or state of the hearing aid and / or another device communicating with the hearing aid (selected program, time elapsed since the last user interaction, etc.).

[0117] The classification unit may be based on or include a neural network, such as a trained neural network.

[0118] Hearing aids may include acoustic (and / or mechanical) feedback control (such as suppression) or echo cancellation systems. Acoustic feedback occurs when the output speaker signal from an audio system that amplifies the signal picked up by the microphone returns to the microphone via an acoustic coupling section through air or other media. This portion of the speaker signal returning to the microphone is then amplified again by the audio system before reappearing at the speaker, and returns to the microphone again. As this cycle continues, the acoustic feedback effect becomes audible, such as unnatural signals or even worse, howling, when the audio system becomes unstable. This problem typically arises when the microphone and speaker are placed close together, such as in hearing aids or other audio systems. Some other typical applications with feedback problems include telephone systems, broadcast systems, headsets, audio conferencing systems, etc. Adaptive feedback cancellation is capable of tracking changes in the feedback path over time. It estimates the feedback path based on a linear time-invariant filter, but its filter weights are updated over time. The filter updates can be computed using stochastic gradient algorithms, including some form of least mean square (LMS) or normalized LMS (NLMS) algorithms. They all have the property of minimizing the mean square of the error signal, and NLMS further normalizes the filter update with respect to the square of the Euclidean norm of some reference signal.

[0119] The feedback control system may include a feedback estimation unit for providing a feedback signal representing an estimate of the acoustic feedback path, and a combination unit, such as a subtraction unit, for subtracting the feedback signal from a signal in the forward path (e.g., picked up by an input converter of a hearing aid). The feedback estimation unit may include an update section comprising an adaptive algorithm and a variable filter section for filtering the input signal according to variable filter coefficients determined by the adaptive algorithm, wherein the update section is configured to update at a configurable update frequency f. upd Update the filter coefficients of the variable filter section. The hearing aid is configured such that the configurable update frequency f upd It has a maximum value f upd,max Maximum value f upd,max The sampling frequency f of the AD converter for the hearing aid s A small part (f upd,max =f s / D).

[0120] The updating section of the adaptive filter may include an adaptive algorithm for calculating updated filter coefficients and passing them to the variable filter section of the adaptive filter. The calculation of the updated filter coefficients and / or the timing of their transmission from the updating section to the variable filter section may be controlled by a start control unit. The timing of the updates (e.g., their specific time points and / or their update frequency) is preferably influenced by multiple different characteristics of the signal in the forward path. The update control scheme is preferably supported by one or more detectors of the hearing aid, and is preferably included in a predetermined criterion containing detector signals.

[0121] Hearing aids may also include other suitable functions for the application in question, such as compression and noise reduction.

[0122] Hearing devices or hearing aids may include hearing devices such as hearing aids, hearing instruments such as hearing instruments adapted to be located at the user's ear or wholly or partially in the ear canal, such as headphones, headsets, ear protection devices, or combinations thereof. Hearing aid systems may include loudspeaker amplifiers (containing multiple input converters and multiple output converters, for example, for use in audio conferencing scenarios), and may include beamforming filter units, for example, providing multiple beamforming capabilities.

[0123] application

[0124] On the one hand, applications of the hearing aids described in detail in the "Detailed Description" section are provided as described above. Applications can be provided in systems including audio distribution. Applications can be provided in systems including one or more hearing devices / hearing aids (such as hearing instruments), headphones, headsets, active ear protection systems, etc., for example, in hands-free telephone systems, teleconferencing systems (e.g., including loudspeaker amplifiers), broadcasting systems, karaoke systems, classroom amplification systems, etc.

[0125] method

[0126] In one aspect, this application further provides a method. This method may include receiving an input sound signal from the environment of a hearing aid user via an input unit and providing at least one electrical input signal representing the input sound signal.

[0127] This method may include determining speech in an input sound signal via a VAD.

[0128] The method may include determining the hearing aid user's own voice in the input sound signal via OVD.

[0129] The method may include determining the turn-taking behavior of a hearing aid user through a turn-taking determination unit.

[0130] The method may include adjusting and / or correcting signal processing parameters based on determined turn-taking behavior of the hearing aid user.

[0131] The method may include providing a processed version of at least one electrical input signal via a processing unit.

[0132] The processing unit can be connected to the input unit.

[0133] The processing unit can be connected to the output unit.

[0134] The processing unit may include the hearing aid's adjusted signal processing parameters.

[0135] The method may include providing a hearing aid user with at least one set of stimuli that can be perceived as sound via an output unit based on a processed version of at least one electrical input signal.

[0136] When appropriately replaced by a corresponding process, some or all of the structural features of the apparatus described above and in detail in the "Detailed Description" section can be combined with the implementation of the method of the present invention, and vice versa. The implementation of the method has the same advantages as the corresponding apparatus.

[0137] Computer-readable media or data carrier

[0138] The present invention further provides a tangible computer-readable medium (data carrier) storing a computer program including program code (instructions), which, when the computer program is run on a data processing system (computer), causes the data processing system to perform (implement) at least some (such as most or all) of the steps of the method described above in detail in the "Detailed Description".

[0139] By way of example, but not limitation, the aforementioned tangible computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to execute or store required program code in the form of instructions or data structures and is accessible by a computer. As used herein, disks include compact discs (CDs), laser discs, optical discs, digital multipurpose discs (DVDs), floppy disks, and Blu-ray discs, wherein these disks typically magnetically copy data while simultaneously being optically copied using lasers. Other storage media include those stored in DNA (e.g., in synthetic DNA strands). Combinations of the aforementioned disks should also be included within the scope of computer-readable media. In addition to being stored on tangible media, computer programs may also be transmitted via transmission media such as wired or wireless links or networks such as the Internet and loaded into data processing systems to run at locations other than tangible media.

[0140] Computer program

[0141] In addition, this application provides a computer program (product) including instructions that, when run by a computer, cause the computer to perform the methods (steps) described above in detail in the "Detailed Description".

[0142] Data processing system

[0143] In one aspect, the present invention further provides a data processing system, including a processor and program code, the program code causing the processor to perform at least some (such as most or all) of the steps of the method described above in detail in the "Detailed Description".

[0144] Hearing system

[0145] On the one hand, it provides hearing systems (such as binaural hearing aid systems) that include the first and second hearing aids described above.

[0146] Each of the first and second hearing aids may include an antenna and transceiver circuitry for establishing a communication link to the other hearing aid, thereby enabling the exchange of information between the two hearing aids.

[0147] Each of the first and second hearing aids can be configured to transmit its determined turn-taking behavior to the other hearing aid.

[0148] Each of the first and second hearing aids can be configured to determine the confidence level of the corresponding determined turn-switching behavior.

[0149] The processing unit of each of the first and second hearing aids can be configured to determine a confidence level. The processing unit of the first or second hearing aid can determine a confidence level.

[0150] Determining confidence levels may include comparing similar parameters to the turn-switching behaviors determined by the corresponding first and second hearing aids.

[0151] Determining the confidence level may include determining the corresponding deviations between similar parameters being compared.

[0152] Determining the confidence level may include comparing the determined deviation with a corresponding threshold.

[0153] For example, when the determined deviation is below the corresponding deviation threshold, the confidence level of the parameters for the determined turn-around behavior can be higher than the confidence level threshold. A confidence level above the confidence level threshold can be called acceptable, and a confidence level below the confidence level threshold can be called unacceptable.

[0154] The parameters of turn-taking behavior can refer to the defined intervals between the hearing aid user's own voice and the voice of another person speaking to the hearing aid user. The parameters of turn-taking behavior can also refer to the defined overlaps between the hearing aid user's own voice and the voice of another person speaking to the hearing aid user. Finally, the parameters of turn-taking behavior can refer to the defined speaking rate of the hearing aid user.

[0155] When the determined confidence level is higher than the confidence level threshold, the processing unit can be configured to adjust the signal processing parameters of the hearing system (and the processing unit of the first and / or second hearing aid).

[0156] On the other hand, a hearing system is provided that includes the hearing aid and one or more assistive devices described above and in detail in the "Detailed Description".

[0157] Hearing systems are adapted to establish communication links between hearing aids and assistive devices so that information (such as control and status signals, and possibly audio signals) can be exchanged or forwarded from one device to another.

[0158] Auxiliary devices may include remote controls, smartphones, eyeglass frames (goggles), or other portable or wearable electronic devices such as smartwatches.

[0159] The assistive device may be constituted by or may include a remote control for controlling the functions and operation of the hearing aid. The functions of the remote control may be implemented in a smartphone, which may run an app that enables the control of audio processing devices via the smartphone (the hearing aid includes a suitable wireless interface to the smartphone, such as Bluetooth or some other standardized or proprietary solution).

[0160] The assistive device may be constituted by or may include an audio gateway device adapted to receive multiple audio signals (e.g., from an entertainment device such as a TV or music player, from a telephone device such as a mobile phone, or from a computer such as a PC) and adapted to select and / or combine appropriate signals (or combinations of signals) from the received audio signals to transmit to the hearing aid.

[0161] The hearing system may also include an infrared radiation (IR) sensor configured to monitor the gaze of the hearing aid user.

[0162] Hearing systems may include assistive devices that include IR sensors configured to monitor the gaze of a hearing aid user.

[0163] Hearing systems may include eyeglass frames (such as goggles) that include IR sensors configured to monitor the gaze of the hearing aid user.

[0164] The speech-turning behavior of hearing aid users can also be determined based on the monitoring of the user's eye gaze using an IR sensor.

[0165] Therefore, the determination of hearing aid users' turn-taking behavior can be further supported / enhanced.

[0166] Monitoring turn-taking behavior of hearing aid users by measuring head rotation (e.g., via an inertial measurement unit as described above) and eye gaze (e.g., via an IR sensor) may be important when attending to multiple speakers [1]. For both normal-hearing and hearing-impaired individuals, not only waiting time but also detailed movement patterns differ and can be used to assess auditory effort, speech intelligibility, and speech comprehension.

[0167] When listening to more than two speakers in a given scenario, following the turn-taking behavior of the hearing aid user can be useful. Head rotation may cause the hearing aid user to follow different speakers during turn-taking, thus enhancing acoustic solutions that do not differentiate between speakers.

[0168] Adjusting the reference signal processing parameters of the hearing aid based on turn-taking behavior (acoustically, head movements, and / or eye-gazing-based solutions) can be achieved through machine learning (e.g., by training a neural network). Training can be performed on a server device, the assistive device, or the hearing aid itself. Thus, training can be distributed across the server device and the hearing aid to receive a trained version of the reference signal processing parameters.

[0169] Training can be conducted at least partially within assistive devices such as mobile devices. Thus, training can be at least partially distributed to external devices and to trained versions of signal processing parameters that the hearing aid can receive.

[0170] Since training neural networks is a computationally intensive task, performing training on server devices or assistive devices can reduce the power consumption of hearing aids.

[0171] For example, training, or at least a portion thereof, can be performed by a smartphone connected to the hearing aid or by the hearing aid itself. Thus, the hearing aid can be configured to be trained during initial production and fitting for the user, and after the user has received the hearing aid, the hearing aid does not need to be connected to a server or assistive device, or at least only needs to be connected to a server or assistive device intermittently.

[0172] Additionally, training can be based on (recorded) user-derived benefit metrics such as program preference and / or direct ranking of listening effort and / or speech understanding. The entire recorded data can be uploaded to a server device, and machine learning can be applied to generalize patterns and optimize listening effort and / or speech understanding estimations from the proposed type of turn-taking behavior.

[0173] APP

[0174] On the other hand, the present invention also provides a non-transitory application called an APP. The APP includes executable instructions configured to run on an assistive device to implement a user interface for the hearing aid or hearing system described above in detail in the "Detailed Description". The APP is configured to run on a mobile phone, such as a smartphone, or another portable device enabled to communicate with said hearing aid or hearing system.

[0175] Tracking activity level

[0176] Hearing aids and / or hearing systems can be configured to track and estimate the activity level of hearing aid users and derive scores relative to goals in rehabilitation programs.

[0177] Hearing aids and / or hearing systems can be configured to record the duration of a hearing aid user's speech using OVD based on the hearing aid and / or hearing system, and combine the recorded time with data about the sound environment such as SNR or activity levels of other identified speech sources.

[0178] Therefore, hearing care specialists can set specific goals to help particular hearing aid users recover.

[0179] By observing a variable time window, the tracking of self-voice activity can be combined with the pause ratio. From this ratio, it can be determined whether the hearing aid is actively engaged in a conversation rather than passively listening to, for example, television. The ratio for a given time window is stored in the hearing aid and / or hearing system for periodic access.

[0180] One or more windows can be passed to different devices, which can further process the data and / or present the data in a user interface.

[0181] Therefore, compared to known wear time tracking, this application proposes a solution that also includes the social activities and active participation of hearing aid users.

[0182] definition

[0183] In this specification, "hearing aid" refers to a device suitable for improving, enhancing, and / or protecting a user's hearing ability, such as a hearing aid, a hearing instrument, an active ear protection device, or other audio processing device, which achieves this by receiving sound signals from the user's environment, generating corresponding audio signals, possibly modifying the audio signals, and providing the possibly modified audio signals as audible signals to at least one ear of the user. "Hearing aid" also refers to a device suitable for electronically receiving audio signals, possibly modifying the audio signals, and providing the possibly modified audio signals as audible signals to at least one ear of the user, such as headphones or headsets. The audible signals may be provided, for example, as sound signals radiating into the user's outer ear, sound signals transmitted as mechanical vibrations through the bone structures of the user's head and / or through parts of the middle ear to the user's inner ear, and electrical signals transmitted directly or indirectly to the user's cochlear nerve.

[0184] Hearing aids can be configured to be worn in any known manner, such as as a unit worn behind the ear (having a tube that directs radiated sound signals into the ear canal or having an output transducer, such as a speaker, arranged close to or located within the ear canal), as a unit wholly or partially arranged in the auricle and / or ear canal, as a unit connected to a fixed structure implanted in the skull, such as a vibrator, or as a connectable unit that is wholly or partially implanted. Hearing aids may include a single unit or several units that communicate with each other (e.g., acoustically, electrically, or optically). The speaker may be housed within the housing along with other components of the hearing aid, or it may be an external unit (possibly combined with a flexible guiding element such as a dome-shaped element).

[0185] More generally, a hearing aid includes an input transducer for receiving sound signals from the user's environment and providing a corresponding input audio signal, and / or a receiver for receiving the input audio signal electronically (i.e., wired or wirelessly); signal processing circuitry (typically configurable) for processing the input audio signal (such as a signal processor, for example including a configurable (programmable) processor, such as a digital signal processor); and an output unit for providing an audible signal to the user based on the processed audio signal. The signal processor may be adapted to process the input signal in the time domain or in multiple frequency bands. In some hearing aids, amplifiers and / or compressors may constitute the signal processing circuitry. The signal processing circuitry typically includes one or more (integrated or separate) storage elements for executing programs and / or for storing parameters used (or potentially used) in the processing and / or for storing information suitable for the hearing aid's functionality and / or for storing information used, for example, in conjunction with an interface to the user and / or an interface to a programming device (such as processed information, for example, provided by the signal processing circuitry). In some hearing aids, the output unit may include an output transducer, such as a loudspeaker for providing airborne sound signals or a vibrator for providing sound signals propagating through structures or fluids. In some hearing aids, the output unit may include one or more output electrodes for (e.g., to a multi-electrode array) providing electrical signals for electrical stimulation of the cochlear nerve (cochlear implant hearing aid). Hearing aids may include a loudspeaker amplifier (including multiple input converters and multiple output converters, for example, in audio conferencing situations).

[0186] In some hearing aids, the vibrator may be adapted to transmit structurally propagated sound signals to the skull transcutaneously or through the skin. In some hearing aids, the vibrator may be implanted in the middle ear and / or inner ear. In some hearing aids, the vibrator may be adapted to provide structurally propagated sound signals to the middle ear bones and / or cochlea. In some hearing aids, the vibrator may be adapted to provide fluid-propagated sound signals to the cochlear fluid, for example, through the oval window. In some hearing aids, the output electrode may be implanted in the cochlea or on the medial side of the skull and may be adapted to provide electrical signals to the hair cells of the cochlea, one or more auditory nerves, the auditory brainstem, the auditory midbrain, the auditory cortex, and / or other parts of the cerebral cortex.

[0187] Hearing aids can be adapted to the specific needs of users, such as those with hearing loss. The configurable signal processing circuitry of a hearing aid can be adapted to apply frequency- and level-variable compression and amplification of the input signal. Customized frequency- and level-variable gain (amplification or compression) can be determined during the fitting process by the fitting system based on the user's hearing data, such as an audiogram, using basic fitting principles (e.g., speech adaptation). This frequency- and level-variable gain can be reflected, for example, in processing parameters, uploaded to the hearing aid via an interface to a programming device (fitting system), and used by a processing algorithm executed by the hearing aid's configurable signal processing circuitry.

[0188] A “hearing system” refers to a system that includes one or two hearing aids. A “binaural hearing system” refers to a system that includes two hearing aids and is adapted to work together to provide audible signals to both of a user’s ears. A hearing system or a binaural hearing system may also include one or more “assistive devices” that communicate with the hearing aids and influence and / or benefit from the functionality of the hearing aids. The aforementioned assistive devices may include at least one of the following: a remote control, a remote microphone, an audio gateway device, an entertainment device such as a music player, a wireless communication device such as a mobile phone (e.g., a smartphone), a tablet computer, or another device including a graphical interface. Hearing aids, hearing systems, or binaural hearing systems may be used, for example, to compensate for hearing loss in persons with hearing impairments, enhance or protect the hearing ability of persons with normal hearing, and / or transmit electronic audio signals to persons. Hearing aids or hearing systems may, for example, be part of or interact with broadcasting systems, active ear protection systems, hands-free telephone systems, car audio systems, entertainment (e.g., TV, music playback, or karaoke) systems, teleconferencing systems, classroom amplification systems, etc. Attached Figure Description

[0189] Various aspects of the invention will be best understood from the following detailed description taken in conjunction with the accompanying drawings. For clarity, these drawings are schematic and simplified, showing only the details necessary for understanding the invention while omitting other details. Throughout the specification, the same reference numerals are used for the same or corresponding parts. Features of each aspect may be combined with any or all features of other aspects. These and other aspects, features, and / or technical effects will be apparent from and illustrated in the following figures, wherein:

[0190] Figure 1 An exemplary application of the hearing aid according to the present invention is shown;

[0191] Figure 2 An exemplary flowchart of a method for determining the turn-switching behavior of a hearing aid user is shown;

[0192] Figure 3A An exemplary overview of a system for static calibration is shown;

[0193] Figure 3B An exemplary head rotation and relative gaze are shown.

[0194] The further applicability of the invention will become apparent from the detailed description given below. However, it should be understood that while the detailed description and specific examples illustrate preferred embodiments of the invention, they are given for illustrative purposes only. Other embodiments of the invention will become apparent to those skilled in the art based on the following detailed description. Detailed Implementation

[0195] The detailed description below, taken in conjunction with the accompanying drawings, serves as a description of various different configurations. This detailed description includes specific details to provide a thorough understanding of several different concepts. However, it will be apparent to those skilled in the art that these concepts can be implemented without these specific details. Several aspects of the apparatus and method are described by various different blocks, functional units, modules, elements, circuits, steps, processes, algorithms, etc. (collectively, “elements”). Depending on the specific application, design constraints, or other reasons, these elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0196] Electronic hardware may include microelectromechanical systems (MEMS), (e.g., application-specific integrated circuits), microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), gating logic, discrete hardware circuits, printed circuit boards (PCBs) (e.g., flexible PCBs), and other suitable hardware configured to perform the various functions described in this specification, such as sensors for sensing and / or recording the physical properties of the environment, devices, users, etc. Computer programs should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, programs, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or other names.

[0197] Figure 1 An exemplary application of the hearing aid according to the present invention is shown.

[0198] Figure 1 The hearing aid is shown to include an input unit 2 and an output unit 3.

[0199] The input unit 2 may include one or more microphones for receiving input sound signals from the hearing aid user's environment and may provide at least one electrical input signal 2A representing the input sound signal.

[0200] The output unit 3 may include one or more speakers for providing at least one set of stimuli that can be perceived as sound to a hearing aid user based on a processed version of at least one electrical input signal 2A.

[0201] The positive or signal path 4 between the input unit 2 and the output unit 3 of the hearing aid 1 may include a first signal processor 5 (such as a digital signal processor), a hearing loss compensation unit 6, and a second signal processor 7. The first signal processor 5, the second signal processor 7, and the hearing loss compensation unit 6 may be configured to process (e.g., provide frequency-varying gain, and / or turn-by-turn behavior compensation) and provide a processed version 3A of at least one electrical input signal 2A to the output unit 3.

[0202] The analysis path 8 of the hearing aid 1 may include functional elements for analyzing the electrical input signal 2A (e.g., determining the level, modulation, signal type, acoustic feedback estimate, turn-by-turn, etc.).

[0203] Analysis path 8 may include modulation filter 9, VAD 10, OVD 11, SPL estimator 12, and / or SNR estimator 13. Inputs from modulation filter 9, VAD 10, OVD 11, SPL estimator 12, and / or SNR estimator 13 may be provided to turn-turn determination unit 14 configured to determine the turn-turn behavior of a hearing aid user.

[0204] Based on the determined turn-taking behavior of the hearing aid user, the hearing aid setting calculation unit 15 can calculate the corresponding adjusted signal processing parameters.

[0205] The adjusted signal processing parameters can be provided to the first signal processor 5, the second signal processor 7, and the hearing loss compensation unit 6, so that the processed version 3A of at least one electrical input signal 2A can be provided to the output unit 3.

[0206] For example, the first signal processor 5 can determine the turn-taking mode (interruption, speaking rate, etc.), and the second signal processor 7 can set noise reduction, directionality, gain, etc. as functions of the first signal processor 5.

[0207] exist Figure 1 The image shows that the hearing aid 1 may additionally include an ear canal microphone 16 and an accelerometer 17.

[0208] The in-ear microphone 16 can receive input sound signals from the hearing aid user's environment and can provide at least one electrical input signal 16A representing the input sound signal.

[0209] At least one electrical input signal 16A of the ear canal microphone 16 can be provided to the SPL estimator 12 to measure the sound level at the ear canal microphone 16, such as the sound level of one's own voice and that of another speaker, for example, to determine the SPL difference.

[0210] At least one electrical input signal 16A of the ear canal microphone 16 can be provided to the first F0 (fundamental frequency) determination unit 18 to determine whether there is an F0 of the hearing aid user's self-voice in the at least one electrical input signal 16A.

[0211] The motion measurement 17A detected by the accelerometer 17 can be provided to the second F0 determination unit 19 to determine whether there is an F0 of the hearing aid user's self-voice in the motion measurement 17A.

[0212] Comparison unit 20 can compare the F0 determined by the first F0 determination unit 18 and the second F0 determination unit 19 to estimate whether the hearing aid user's self-voice exists in the input received by the ear canal microphone 16 and accelerometer 17. This estimate can be provided to OVD 11 to support the determination of self-voice in the input sound signal.

[0213] Furthermore, at least one electrical input signal 16A from the ear canal microphone 16 and a motion measure 17A detected by the accelerometer 17 can be provided to the correlation unit 21 to determine the correlation between the electrical input signal 16A and the motion measure 17A. The correlation can be provided to the OVD 11 to support the determination of self-voice in the input sound signal by indicating a high cross-correlation in the F0 range (100-200Hz) when the user is speaking. Alternatively, it can be the cross-spectrum of two signals that has a peak at F0 when speaking.

[0214] For example, the turn-taking determination unit 14 can determine the turn-taking behavior of the hearing aid user according to the following description. Based on VAD 10 and OVD 11, the interruption can be measured as the time interval (T1) when both VAD 10 and OVD 11 detect speech and the time interval (T2) when only VAD 10 detects speech, i.e., T1 (VAD on, OVD on) – T2 (VAD on, OVD off). The measured interruption can be averaged over a time interval (e.g., 2, 3, 4 minutes or longer) to obtain the median interruption to increase the certainty of the measured interruption. Furthermore, based on the modulation filter 9, the speech rate of the hearing aid user can be determined based on the power envelope as the peak value of the corresponding modulation spectrum, such as 0-20Hz (measurement range), with a peak value at approximately 4-5Hz, corresponding to 4-5 syllables per second.

[0215] For example, typical values ​​could be median intervals of less than 200 ms indicating easy conditions (low hearing effort) and median intervals of more than 300 ms indicating difficult conditions (high hearing effort). Similarly, under difficult conditions (high hearing effort), the spread of the interval distribution can be widened. Intervals (interval duration) are individual intervals, and adaptive systems can learn the hearing aid user's behavior as a function of SNR to determine "typical patterns" and thus when "atypical patterns" exist (e.g., through machine learning, through statistical analysis, etc.).

[0216] like Figure 1 As shown, optionally, turn-taking behavior determined by another hearing aid 22 of another hearing aid user or from another hearing aid 23 (e.g., the hearing aid of a user conversing with the hearing aid user) can be provided to the turn-taking determination unit 14, so that the confidence level of the corresponding determined turn-taking behavior can be determined. When the determined confidence level is higher than the confidence level threshold, the corresponding adjusted signal processing parameters and adjustable signal processing parameters can be calculated.

[0217] Figure 2 An exemplary flowchart of a method for determining the turn-taking behavior of a hearing aid user is shown.

[0218] The method may include step S1, receiving an input sound signal from the environment of the hearing aid user through an input unit and providing at least one electrical input signal representing the input sound signal.

[0219] The method may include step S2, determining speech in the input sound signal. Step S2, determining speech, may be performed by a VAD. If speech is not determined, step S1, receiving the input sound signal, may be repeated.

[0220] When speech is detected, the method may include step S3, determining the user's own voice within the input sound signal. Step S3, determining the user's own voice, may be performed by the OVD. If the user's own voice is not detected, step S1, receiving the input sound signal, may be repeated.

[0221] When a self-voice is detected, the method may include step S4, determining the turn-taking behavior of the hearing aid user. Step S4, determining the turn-taking behavior, may be performed by a turn-taking determination unit.

[0222] The method may include step S5, adjusting signal processing parameters based on determined turn-taking behavior of the hearing aid user.

[0223] The method may include step S6, providing a processed version of at least one electrical input signal. Step S6, providing the processed version, may be performed by a processing unit connected to the input unit and the output unit. The processing unit may include adjusted signal processing parameters of the hearing aid.

[0224] The method may include step S7, providing a hearing aid user with at least one set of stimuli that can be perceived as sound, based on a processed version of at least one electrical input signal. Step S7, providing the stimuli, may be performed by an output unit.

[0225] Figure 3A An exemplary overview of a system for static calibration is shown.

[0226] Figure 3B An exemplary head rotation and relative gaze are shown.

[0227] By placing electrodes in the hearing aid user's ear canal, EOG signals can be picked up, providing information about where the user is looking. To determine where the user is looking, the values ​​measured from the in-ear electrodes (V) need to be converted into angles (radians or degrees). Static and dynamic processing of the EarEOG (ear EOG) eye-direction calibration hearing aid can be provided.

[0228] Hearing aids may include one or more of the following sensors, which may be used during calibration:

[0229] -Ear-EEG electrodes: Configured to pick up electrical signals indicating where the eyes are looking. By using only information from these sensors, it may be possible to estimate relative gaze (i.e., the angle of orientation relative to the hearing aid user's head).

[0230] - Inertial sensors (accelerometers, hearing instruments, and / or magnetometers): configured to provide information on how the user's head is oriented (e.g., head rotation, yaw angle). Absolute gaze can be estimated using information from the inertial sensors and ear-EEG electrodes;

[0231] - Microphones: Configured to collect information about the acoustic environment. More interestingly, by using multiple microphones, the direction of arrival of a sound source can be calculated. The direction of arrival of the sound source can be used for on-the-spot calibration / recalibration processing, assuming the hearing aid user is ultimately looking towards the sound source.

[0232] Figure 3A An exemplary system for static calibration is shown.

[0233] Static calibration has the advantage of using external devices for calibration. Such calibration procedures can be performed in more controlled environments, such as audiological clinics or homes.

[0234] exist Figure 3A In the diagram, the system used for static calibration is shown to include:

[0235] - Hearing aids (HAs) worn by hearing aid users. These hearing aids may include an ear-EEG sensor, on which the hearing aid is configured to measure ear-EOG signals. The hearing aid may also include one or more inertial sensors, on which the hearing aid is configured to measure head orientation (yaw angle);

[0236] - External screen for presenting / displaying the calibration sequence that the hearing aid user is assumed to follow by eye gaze;

[0237] - The calibration processor is configured to collect ear-EOG signals and inertial sensor measurement data and send a predetermined stimulation calibration sequence to an external screen. It generates a calibration dataset by analyzing all types of data (i.e., based on ear-EOG, inertial sensor, and stimulation calibration sequence) and sends the calibration results to the hearing aid.

[0238] Systems for static calibration can be embedded in a smartphone app. Alternatively, systems for static calibration can be fixed installations in audiology clinics.

[0239] Static calibration may include some or all of the following steps:

[0240] 1. Hearing aid users wear hearing aids that include an ear-EOG sensor;

[0241] 2. The hearing aid user is positioned in a specific location in front of a screen set at a certain distance from the user's eyes. In the case of a smartphone used for calibration, this distance can be estimated from the smartphone's front-facing camera.

[0242] 3. The hearing aid user is instructed to follow the signal on the screen (such as a red dot) with their eyes while keeping their head still or moving their head naturally. In the case of a natural head position, head orientation is tracked by the hearing aid's inertial sensors, or, if using a smartphone, by the front-facing camera.

[0243] 4. The calibration sequence begins with a point moving across the screen. This sequence is a combination of rapid shifts between positions (referred to as saccades in the prior art) and remaining stationary at certain positions (referred to as fixation in the prior art). The point position can vary along the horizontal and vertical axes;

[0244] 5. During the sequence, the hearing aid collects data from sensors (ear-EEG, inertial sensor) and the position (x, y) of the point;

[0245] 6. At the end of the sequence, the synchronized ear-EOG data, inertial sensor data, and point position data are used in the calibration process to estimate the eye gaze of the hearing aid user.

[0246] 7. The calibration results are uploaded to the hearing aid.

[0247] Figure 3B An exemplary head rotation and relative gaze are shown.

[0248] Dynamic calibration provides the possibility of calibration / recalibration while the hearing aid user is in a dynamic situation (e.g., the hearing aid user is walking around in a multi-speaker environment).

[0249] While systems used for static calibration require external devices and more complex setups, dynamic calibration uses assumptions that simplify the process. Current hearing aids use their microphones and algorithms to detect the direction of arrival (DoA) of sound sources. Since the microphone / hearing aid position is fixed on the head, the DoA algorithm provides an estimate of the angle at which the sound is coming. If we assume that at some point the hearing aid user is looking at those sound sources, we have a calibration point by comparing the DoA angle provided by the microphone data with ear-EOG data (from ear-EEG electrodes).

[0250] Simultaneous Localization and Mapping (SLAM) is typically used in autonomous vehicles, where you attempt to build / update a map of landmarks while keeping track of the location of an agent (such as a vehicle) within that map.

[0251] If we shift the SLAM problem to the acoustics domain, hearing aid users will navigate in a sound source map.

[0252] Therefore, a dynamic calibration system is provided, which is configured to build / update a sound source map and locate the hearing aid user in that map.

[0253] The construction / updating of sound source maps and the localization of hearing aid users can be based on data from hearing aid microphones, ear-EOG, and inertial sensors.

[0254] The system used for dynamic calibration calibrates / recalibrates the hearing aid (with ear-EEG) based on the sound source map and the hearing aid user's location on the map. This is achieved by assuming that the hearing aid user is looking towards the sound source at certain moments. Calibration points are provided each time this assumption is made.

[0255] The success of dynamic calibration processing depends on the assumption that the hearing aid user is looking towards the sound source. When the hearing aid user shifts their attention to a new sound source on their right, they may perform the following steps... Figure 3B The diagram shows (the angle of relative gaze and head rotation as a function of time):

[0256] 1) Quickly scan from the initial target to the new target: look to the right. This usually takes 150ms;

[0257] 2) Once the eyes are almost on the target, the hearing aid user's head begins to turn towards the target, thus the head begins to turn to the right;

[0258] 3) While the head is turning, the eyes are focused on the target, but because the head is turning, the eyes compensate for the head rotation and move to the left. This simultaneous head / eye movement is very special because:

[0259] a. The eyes and head move at the same speed;

[0260] b. The two signals have opposite directions (eyes compensate for head movements);

[0261] 4) Once the head reaches its final destination, the eyes and head remain fixed for a certain period of time.

[0262] When the inertial and ear-EOG data exhibit the aforementioned pattern, it can be assumed that the hearing aid user is looking at an object. Then, if the sound source simultaneously emits sound, new calibration points related to the ear-EOG, inertial, and DoA data can be added.

[0263] The equations below illustrate the calculations performed during online calibration. A set of (n) DoA measurements corresponding to a smaller number of directions (m) can be placed into matrix Y (m*n). Similarly, (n) gazes from EOG data corresponding to the corresponding (m) directions can be placed into another matrix A (m*n). It is then assumed that each DoA is paired with a corresponding gaze from the EOG data:

[0264] Y = αA

[0265]

[0266] Where α is the scalar sought, and It is the least squares estimator.

[0267] For the above equation to hold, the DoA and EOG measurement results can be placed in the same global coordinate frame.

[0268] When appropriately replaced by a corresponding process, the structural features of the hearing aid / hearing device / hearing system described above in detail in the "Detailed Description" section can be combined with the steps of the method of the present invention.

[0269] Unless explicitly stated otherwise, the singular forms “a” and “the” used herein include the plural forms (i.e., meaning “at least one”). It should be further understood that the terms “having,” “comprising,” and / or “including” as used in the specification indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. It should be understood that, unless explicitly stated otherwise, when an element is referred to as “connected” or “coupled” to another element, it may be a direct connection or coupling to the other element, or there may be intermediate inserting elements. The term “and / or” as used herein includes any and all combinations of one or more of the listed related items. Unless explicitly stated otherwise, the steps of any method disclosed herein do not necessarily have to be performed in the exact order disclosed.

[0270] It should be understood that references to "an embodiment," "an embodiment," "an aspect," or "may" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Furthermore, particular features, structures, or characteristics may be suitably combined in one or more embodiments of the invention. The foregoing description is provided to enable those skilled in the art to implement the various aspects described herein. Various modifications will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects.

[0271] The claims are not limited to the aspects shown herein, but encompass the full scope consistent with the language of the claims, wherein, unless expressly stated, an element referred to in the singular does not mean "one and only one," but rather "one or more." Unless expressly stated, the term "some" means one or more.

[0272] Therefore, the scope of this invention should be determined based on the claims.

[0273] References

[0274] [1] M. Harrison, "Evaluating the Use of Steering a Hearing Aid in ADynamic Multi-Talker Environment Using Body Signals," University of Glasgow, 2018.

Claims

1. A hearing aid suitable for wearing in or at the ear of a hearing aid user and / or suitable for complete or partial implantation in the head of a hearing aid user, said hearing aid comprising: An input unit is configured to receive an input sound signal from the environment of a hearing aid user and provide at least one electrical input signal representing the input sound signal; An output unit for providing a hearing aid user with at least one set of stimuli that can be perceived as sound, based on a processed version of at least one electrical input signal; A voice activity detector (VAD) is configured to determine speech in an input sound signal; An Oral Voice Detector (OVD) is configured to identify the hearing aid user's own voice in the input sound signal; A processing unit connected to an input unit and an output unit and including signal processing parameters for the hearing aid to provide a processed version of at least one electrical input signal; The turn-by-turn determination unit is configured to determine the turn-by-turn behavior of the hearing aid user. Antenna and transceiver circuitry is used to establish a communication link to another hearing aid, thereby enabling the exchange of information between hearing aids. The determination of a hearing aid user's turn-taking behavior includes the use of speech activity detectors to determine the speech activity and self-voice detectors to determine the pauses between the speech of the hearing aid user and another person. The hearing aid is configured to disable the turn-by-turn determination unit when, during a first time interval, the speech activity detector fails to determine speech and the self-voice detector fails to determine the hearing aid user's self-voice in the input sound signal. The hearing aid is configured to transmit a determined turn-around behavior to another hearing aid and receive a determined turn-around behavior from the other hearing aid, and to determine the confidence level of the corresponding determined turn-around behavior; The processing unit is configured to adjust the signal processing parameters when the determined confidence level is higher than the confidence level threshold.

2. The hearing aid of claim 1, wherein the hearing aid includes a modulation filter configured to determine the speech rate of the hearing aid user.

3. The hearing aid of claim 1, wherein the hearing aid further comprises a signal-to-noise ratio (SNR) estimator configured to determine the signal-to-noise ratio in the hearing aid user's environment.

4. The hearing aid of claim 1, wherein the hearing aid further comprises a sound pressure level (SPL) estimator for measuring the level of sound at the input unit.

5. The hearing aid of claim 1, wherein the hearing aid further comprises a timer configured to determine the start point of the turn-around determination.

6. The hearing aid of claim 5, wherein the hearing aid is configured to activate the turn-by-turn determination unit when the timer determines the start point.

7. The hearing aid of claim 1, wherein the hearing aid includes a memory unit configured to store reference signal processing parameters of a processing unit; and wherein the processing unit is configured to apply the reference signal processing parameters when the self-voice detector has not yet determined a self-voice in a second time period.

8. The hearing aid of claim 1, wherein the hearing aid includes an accelerometer and / or an in-ear microphone, and wherein a self-voice detector is configured to determine the self-voice of the hearing aid user based on the accelerometer and / or the in-ear microphone.

9. The hearing aid of claim 1, wherein the hearing aid includes an inertial measurement unit.

10. The hearing aid of claim 1, wherein the hearing aid is configured to transmit determined turn-switching behavior of a hearing aid user to a server device, and the server device is configured to adjust reference signal processing parameters of the hearing aid based on the turn-switching behavior.

11. The hearing aid of claim 1, wherein the hearing aid is configured to receive adjusted reference signal processing parameters from a server device and store the adjusted reference signal processing parameters on a memory unit.

12. A hearing system comprising first and second hearing aids according to any one of the preceding claims.

13. The hearing system of claim 12 further includes an infrared radiation (IR) sensor configured to monitor the gaze of the hearing aid user.

14. A method for operating a hearing aid, comprising: The input unit receives input sound signals from the hearing aid user's environment and provides at least one electrical input signal representing the input sound signals; Speech is determined in the input sound signal using a voice activity detector; The user's own voice is identified in the input sound signal using a self-voice detector. A processed version of at least one electrical input signal is provided by a processing unit connected to the input and output units and including the adjusted signal processing parameters of the hearing aid. The output unit provides the hearing aid user with at least one set of stimuli that can be perceived as sound based on a processed version of at least one electrical input signal. A communication link is established between hearing aids through antennas and transceiver circuits, enabling the exchange of information between hearing aids. The turn-taking determination unit determines the turn-taking behavior of the hearing aid user, wherein determining the turn-taking behavior of the hearing aid user includes determining the interruption between the speech determined by the speech activity detector and the self-voice determined by the self-voice detector, and the speech of the hearing aid user and another person. The turn-taking determination unit is disabled when the speech activity detector does not determine the speech and the self-voice detector does not determine the self-voice of the hearing aid user in the input sound signal during a first time interval. Transmit the determined turn-taking behavior to another hearing aid and receive the determined turn-taking behavior from the other hearing aid, and determine the confidence level of the corresponding determined turn-taking behavior; Adjust the signal processing parameters when the determined confidence level is higher than the confidence level threshold.

Images