A hearing aid and a corresponding supplementary device, storage medium and operating method

Abstract

A hearing aid and a corresponding supplementary device, a storage medium and a method of operation are disclosed, wherein the hearing aid comprises an antenna and a transceiver circuit for establishing a communication link with a supplementary device configured as a user control interface, an input device for picking up sound from the hearing aid environment to provide at least one electrical input signal, a signal processor configured to process the at least one electrical input signal and to provide a processed signal, an output unit for presenting a stimulus representing the processed signal perceptible as sound to the user, wherein the signal processor is configured to start measuring a corresponding head-related transfer function in a far-field calibration mode based on a test sound signal picked up by the at least one microphone from a loudspeaker of the supplementary device and an electrical version of the test sound signal received by the antenna and transceiver circuit from a transmission of the supplementary device at a point in time when the supplementary device informs the hearing aid.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202110224484.8, filed on March 1, 2021, entitled "Hearing Aid System for Estimating Acoustic Transfer Function". Technical Field

[0002] This application relates to the field of hearing aids. In particular, it relates to several different aspects of the acquisition and / or detection of a hearing aid user's self-voice, such as those related to the preservation or reconstruction of beamforming and / or spatial cues. Background Technology

[0003] EP2928215A1 describes a mobile device including a speaker for providing personalized beamforming, see, for example, Figure 1 of EP2928215A1 and the corresponding description.

[0004] US20120063610A1 relates to wireless audio streaming, in which acoustic audio signals and corresponding radio electromagnetic signals exist in parallel, such as audio streaming from a TV or audio streaming in a classroom. The two are used in several different examples to combine and present the signal to the user, one to improve the other. In one example, the acoustically propagated signal is improved by noise reduction using a “clean” wirelessly received signal before being presented to the user. US20120063610A1 specifies that the difference in transfer function between the acoustic and electromagnetic propagation paths of sound from another device to the hearing aid microphone can be estimated.

[0005] US20190115041A1 addresses the same situation as US20120063610A1 (representation of the corresponding acoustic and wireless propagation of the received target signal). It estimates the delay between the wirelessly and acoustically received target signal. This can be represented as an estimate of the transfer function from the microphone of the assistive device to the microphone of the hearing aid. Summary of the Invention

[0006] This application relates to hearing aid systems and hearing aids or headphones. The invention primarily relates to functions surrounding self-voice detection and / or self-voice estimation, i.e., focusing on, for example, detecting when a hearing aid (HA) user speaks, and / or self-voice acquisition, i.e., acquiring an enhanced version of the HA user's voice signal, for example, when speaking in potentially acoustically noisy conditions. Self-voice detection and acquisition are key technologies, for example, for hands-free phones, voice-controlled hearing devices, etc. The idea of ​​this invention is to focus on personalized solutions, i.e., methods for improving the performance of different algorithms, tailored to the specific (acoustic) characteristics of a particular user, such as voice characteristics, head and torso characteristics, etc.

[0007] Therefore, it is proposed to use mobile phones (or any other mobile (e.g., portable, wearable) communication devices that include microphones and / or speakers) as a) wireless microphones; and / or b) wireless speakers.

[0008] The underlying assumption is that most hearing aid (HA) or headphone (HS) users are equipped with (or will be equipped with) mobile phones (or similar portable (such as wearable) devices) and will be very proficient in using them.

[0009] Using a mobile phone (or similar mobile device) as a wireless microphone enables the user to i) (in the mobile device and / or hearing aid) pick up the user's speech samples; ii) estimate the acoustic transfer function of each microphone from the user's (just outside) mouth in the HA or HS to the HA (or HS) (we denote these acoustic transfer functions as self-voice transfer functions (OVTF)).

[0010] Using a mobile phone (or similar mobile device) as a wireless speaker allows the user to iii) estimate the acoustic transfer function (front head related transfer function (HRTF) of the microphone from the arm's length distance (e.g., HA (or HS) in front of the user, such as the typical position of a conversation partner) to the HA (or HS).

[0011] Below, several different ideas and applications are presented in the context of hearing aid systems. However, they can be equally applied to other head-mounted communication devices such as headphones, helmets, etc.

[0012] First Hearing Aid System

[0013] On the one hand, a hearing aid system including a hearing aid and an assistive device is provided.

[0014] In one aspect of this application, a hearing aid system is provided. The hearing aid system may include A) a hearing aid (e.g., at least one) suitable for wear by a user in or in the user's ear or suitable for complete or partial implantation in the head of the user's ear; and B) a portable assistive device, such as a communication device, e.g., a smartphone. The hearing aid system may be adapted to establish a communication link between the hearing aid and the assistive device, such that data, such as control signals, status signals, and / or audio signals, can be exchanged between them or forwarded from one device to another. The hearing aid may include at least one microphone for picking up sound from the hearing aid environment, configured to provide at least one corresponding electrical input signal representing the sound. The hearing aid may also include a signal processor configured to process the electrical input signal or signals derived therefrom and provide a processed signal. The hearing aid may also include an output unit, such as a speaker, for presenting a stimulus representing the processed signal that can be perceived by the user as sound. The assistive device may include at least one microphone for picking up sound from the assistive device environment, providing at least one corresponding auxiliary electrical input signal representing the sound. The assistive device may also include a user control interface that enables the user to activate the calibration operation mode of the hearing aid system. The signal processor of the hearing aid may be configured to compare at least one electrical input signal and at least one auxiliary electrical input signal, or their corresponding transformations (or selected frequency ranges), for the corresponding time period, and provide an estimate of the transfer function from at least one microphone of the assistive device to at least one microphone (or more) of the hearing aid.

[0015] This can provide an improved hearing aid system.

[0016] The term "corresponding transformation" may include, for example, Fourier transforms such as the short-time Fourier transform (STFT), the discrete-time Fourier transform (DTFT), or other transforms such as the Laplace transform, cosine or sine transforms (such as the discrete cosine or sine transform).

[0017] When transmitting electrical signals representing audio between devices, such as transmitting "at least one auxiliary electrical signal" from an assistive device to a hearing aid (or vice versa), it may be considered to transmit only a selected frequency range (e.g., the most important frequency range, such as the frequency range containing (as important) speech components) when attempting to limit the transmission bandwidth and thus power.

[0018] When at least one microphone of the assistive device is located near the user's mouth, such as in front of it, the transfer function can represent the self-voice transfer function (OVTF) when the user raises their voice, such as speaking, during the self-voice calibration operation mode of the hearing aid system. Preferably, the microphone of the assistive device is located at a distance less than the maximum distance from the user's mouth when the user speaks. This maximum distance could be, for example, 0.1 m, such as 0.05 m or 0.02 m. The user interface can be configured to initiate the measurement of the corresponding OVTF during the self-voice calibration operation mode of the hearing aid system. The hearing aid system can be configured, for example, via the user interface of the assistive device to instruct the user how to position (and / or orient) the assistive device relative to the user, and / or speak a phrase or sentence (e.g., with a specific vocal effort, such as loud or soft, etc., based on the current noise level around the user), see, for example, see Figure 4B .

[0019] A hearing aid system may include at least one voice activity detector, enabling the classification of an electrical input signal representing a sound (such as its time period) as originating from or not originating from human speech, and possibly as self-voice or non-self-voice, for example, classifying the sound as speech or non-speech. Classification of time periods may be performed at the sub-band level. Classification may be binary (such as 0 or 1, or "no" or "yes," etc.) or probabilistic (such as a value between 0 and 1). In this specification, self-voice means the speech of the hearing aid wearer (user).

[0020] Enabling the self-voice calibration operation mode of the hearing aid system may include, for example, simultaneously starting to record the user's self-voice (e.g., in the form of a time period of electrical signals from an available microphone), such as when the self-voice detector indicates the presence of the user's self-voice or its presence with a probability higher than, for example, a predetermined (or adaptively determined) threshold. The user interface is configured to enable the user to activate a specific calibration, such as starting to record (e.g., store) the corresponding current time period of at least one electrical input signal and at least one auxiliary electrical input signal.

[0021] The assistive device can be configured to generate a calibration control signal based on calibration initiated from a user interface. The assistive device can be configured to transmit the calibration control signal to a hearing aid. The assistive device can be configured to start and / or stop recording at least one auxiliary electrical input signal for a period of time based on the calibration control signal. The hearing aid can be configured to receive the calibration control signal from the assistive device. The hearing aid can be configured to start and / or stop recording at least one electrical input signal for a period of time based on the calibration control signal. The assistive device can be configured to start and / or stop transmitting at least one auxiliary electrical input signal for the current period of time to the hearing aid based on the calibration control signal.

[0022] A hearing aid can be configured to determine a transfer function (e.g., frequency-varying) based on at least one electrical input signal and at least one auxiliary electrical input signal over a recorded time period. A hearing aid can be configured to determine the transfer function based on a calibration control signal. A hearing aid can be configured to determine the transfer function based on a self-voice control signal, for example, using only the portion of the recorded time period that conforms to the indication of the presence of user voice given by the self-voice control signal. A hearing aid can be configured to determine the transfer function based on both the calibration control signal and the self-voice control signal.

[0023] Hearing aids and assistive devices may include corresponding antennas and transceiver circuitry, thereby enabling the establishment of a communication link between the hearing aid and the assistive device.

[0024] The user interface can be configured to control the functions of the hearing aid system (including enabling (and / or terminating) the self-voice calibration mode).

[0025] At least one electrical input signal and at least one auxiliary electrical input signal, or their corresponding transformations, or their selected frequency ranges, corresponding to a time period (such as the current time period), can be stored in the memory of the hearing aid system. The memory can be distributed between the hearing aid and the assistive device (or located in another device or system). At least one electrical input signal or its transformation for the current time period can be stored in the memory of the hearing aid. At least one auxiliary electrical input signal or its transformation for the current time period can be stored in the memory of the assistive device. At least one auxiliary electrical input signal or its transformation for the current time period can be transmitted to the hearing aid via a communication link. When received in the hearing aid, at least one auxiliary electrical input signal for the current time period can be stored in the memory of the hearing aid. Based on this, two corresponding electrical input signals for the current time period can be used (e.g., frequency-wise) to determine the (frequency-varying, acoustic) transfer function of the appropriate microphone in at least one microphone of the assistive device to the hearing aid. The determined (acoustic) transfer function or its representation can thus be stored in the memory of the hearing aid system, such as the memory of the hearing aid.

[0026] A hearing aid may include at least a portion of a memory, wherein a time period or corresponding transformation or a selected frequency range of at least one electrical input signal and / or at least one auxiliary electrical input signal may be stored.

[0027] Hearing aids may include a self-voice detector. Hearing aids may be configured to store only the time period during which the self-voice detector indicates the presence (or presence with a probability higher than a threshold, such as 50%) of at least one electrical signal.

[0028] The auxiliary device can be configured to generate a calibration control signal based on the calibration being initiated from the user interface.

[0029] The assistive device can be configured to transmit at least one assistive electrical input signal for the current time period to the hearing aid based on a calibration control signal.

[0030] At least one of the microphones in a hearing aid can be defined as a reference microphone. A hearing aid may include at least two microphones, each for picking up sound from the hearing aid environment and providing corresponding electrical input signals representing that sound. One of the at least two microphones can be defined as a reference microphone. The definition of the reference microphone can be used to determine the relative (acoustic) transfer function from the reference microphone to another microphone in the hearing aid system for a given sound source location.

[0031] The assistive device may include a speaker, and the assistive device may be configured to play test sound signals to the assistive device environment in a separate far-field calibration operation mode and transmit an electrical version of the test sound to the hearing aid based on input from a user control interface. The user interface may be configured to initiate a measurement of the corresponding head-related transfer function in the far-field calibration operation mode. The hearing aid system may be configured, for example, via the user interface of the assistive device to instruct the user on how to act during the far-field calibration mode (e.g., how to position (and / or orient) the assistive device relative to the user), see, for example, [see...]. Figure 4C In far-field calibration mode, the assistive device is positioned relative to the user (particularly relative to the hearing aid microphone whose (acoustic) transfer function will be estimated), in a preferred location, such as being held in the hand or placed on or beside a table or other support. The preferred location relative to the user (e.g., distance, angle, etc.) can be known in advance or estimated during calibration, for example, using one or more sensors of the assistive device and / or the hearing aid. The hearing aid system can be configured such that data representing the estimated location can be obtained by the hearing aid (e.g., transmitted to the hearing aid). Specifically, the distance between the assistive device and the hearing aid (e.g., between the speaker of the assistive device and one of the microphones of the hearing aid, such as a reference microphone) can be estimated and stored, for example, in the hearing aid. This distance can be estimated, for example, in the assistive device and transmitted, for example, to the hearing aid.

[0032] Hearing aid systems may include a distance sensor for estimating the distance between the assistive device and the hearing aid. In a calibration mode (far-field calibration mode) for estimating the user's head-related transfer function, the assistive device may be configured to estimate the distance between the assistive device (e.g., the assistive device's speaker) and the hearing aid. The distance sensor may include an image sensor, such as a camera (e.g., a high-speed camera).

[0033] A hearing aid system can be configured to provide a test signal (e.g., provided by a signal generator or via a stored waveform) in a far-field calibration operating mode, providing a test sound signal when played by a speaker. The hearing aid system, such as an assistive device, may include a test signal generator for providing the test signal, providing a test sound signal when played by a speaker. The test signal generator may be connected to a speaker and may be connected to a speaker, for example, as part of a far-field calibration mode. The test signal may be configured to include (or simulate) speech (thus ensuring the inclusion of relevant frequencies of speech). The hearing aid may include signal processor-accessible memory, for example for storing a time period during far-field operating mode for at least one electrical input signal and possibly at least one auxiliary electrical input signal, possibly and a representation of the resulting acoustic transfer function. The memory of the hearing aid system (such as a hearing aid or assistive device or another device) may include test signal segments, providing a test sound signal when played by a speaker of the assistive device.

[0034] The test audio signal can be configured to include frequencies important to the application in question, such as the energy of important frequencies like speech frequencies, for example, frequencies between 1 kHz and 5 kHz. Advantageously, the test audio signal is a chirped signal (e.g., a tone signal whose frequency increases over time). Thus, HRTF can be estimated using, for example, a procedure outlined in [Farina, 2000]. The level of the test audio signal can be adaptively configured relative to the current ambient noise level (to ensure that the signal-to-noise ratio of the test audio is above a certain (e.g., predetermined) minimum SNR threshold).

[0035] In far-field calibration operation mode, when the test sound signal is picked up by at least one microphone of the hearing aid, the hearing aid can be configured to store at least one electrical signal for a time period. In far-field calibration operation mode, the hearing aid system can be configured to transmit the test sound signal to the hearing aid (e.g., via a communication link). The hearing aid can be configured to receive the test sound signal and store it in its memory. The hearing aid's signal processor can be configured to receive the test sound signal and at least one electrical signal for a time period (e.g., a time period received by a reference microphone) and (based on this) determine the head-related transfer function (HRTF) from the speaker position to at least one microphone (such as the reference microphone).

[0036] A hearing aid system in which the signal processor of the hearing aid is configured in a far-field calibration operation mode to receive an electrical version of a test sound and at least one electrical signal for the time period and to determine, based on the signal, an acoustic transfer function from the speaker position of the assistive device to at least one microphone of the hearing aid. The acoustic transfer function from the speaker position of the assistive device to at least one microphone of the hearing aid is also referred to herein as a head-related transfer function (for the acoustic channel from the speaker position to the position of the microphone involved).

[0037] The hearing aid system may include a carrier such as a "selfie stick" adapted to receive and carry an assistive device, allowing the assistive device to be positioned further from the user than arm's length. The assistive device may be attached to the carrier such as the selfie stick. Based on (e.g., determined by the hearing aid) a correlation between, for example, the electrical signal of at least one microphone of the hearing aid and the electrical signal of at least one microphone of the assistive device, the length of the selfie stick may be adjusted to obtain a desired distance between the microphone of the hearing aid and a telephone in front of the user. The given distance may be indicated by a relevant measurement. The hearing aid system may be configured to initiate a calibration measurement when a certain (e.g., predetermined) distance is obtained (e.g., indicated by the user via a user interface). Thus, the user does not need to actively initiate the measurement. The user may be notified before the calibration measurement begins (to ensure the user does not move during the measurement). The notification may occur via the telephone screen, through audio from the telephone, or through audio played via the output unit of the hearing aid.

[0038] The hearing aid may include a beamforming filter configured to provide one or more beamformers, wherein the filter weights of the one or more beamformers are personalized using the transfer function. The one or more beamformers may include a self-voice beamformer aimed at picking up the user's voice (in which case a calibrated self-voice transfer function is used to determine the filter weights). The one or more beamformers may include a far-field beamformer aimed at picking up the voice of a communication partner (in which case a calibrated head-related transfer function is used to determine the filter weights).

[0039] One or more beamformers may include a self-voice beamformer incorporating personalized filter weights, configured to amplify signals originating from the direction of the user's mouth and suppress sound signals from other directions. The self-voice beamformer may be configured to provide an estimate of the user's self-voice. Hearing aids, for example in telephone operation mode, may be configured to transmit the estimate of the user's self-voice to another device, such as an assistive device (e.g., a smartphone).

[0040] One or more beamformers may also include a beamformer containing personalized filter weights, which is configured to suppress the audio signal from a far-field speaker.

[0041] Hearing aids may consist of or include hearing instruments used to compensate for a user’s hearing loss.

[0042] The auxiliary device may consist of or include a remote control, a smartphone, or other portable or wearable electronic devices such as a smartwatch.

[0043] The assistive device may constitute or include a remote control for controlling the hearing aid system or the functions and operation of the hearing aid. The functions of the remote control for the hearing aid system may be implemented in a smartphone. The assistive device, such as a smartphone, may be configured to run an application (APP) that enables control of the hearing aid via the assistive device. The hearing aid may include a suitable wireless interface to the assistive device, such as a smartphone, for example, based on Bluetooth or some other standardized or proprietary solution.

[0044] The assistive device may be constituted by or include an audio gateway device, which is adapted to receive multiple audio signals (e.g., from entertainment devices such as TVs or music players, telephone devices such as mobile phones, or computers such as PCs) and to enable the user to select and / or combine appropriate signals (or combinations of signals) of the received audio signals for transmission to the hearing aid.

[0045] A hearing aid system may include two hearing aids suitable for implementing a binaural hearing aid system. The two hearing aids may include appropriate antennas and transceiver circuitry, enabling them to exchange data (such as audio and / or control data). This allows for the development of (intended) symmetry (especially in determining the self-voice transfer function and the far-field (head-related) transfer function relative to the user's front direction).

[0046] Second hearing aid system

[0047] In another aspect of this application, a hearing aid system is provided. The hearing aid system may include A) a hearing aid (e.g., at least one) suitable for wear by a user; and B) a portable assistive device, such as a communication device, for example, a smartphone. The hearing aid system may be adapted to establish a communication link between the hearing aid and the assistive device to enable data exchange therebetween. The hearing aid may include at least one microphone for picking up sound from the hearing aid environment, configured to provide at least one corresponding electrical input signal representing the sound. The assistive device may include at least one microphone for picking up sound from the assistive device environment, providing at least one corresponding auxiliary electrical input signal representing the sound. The hearing aid system may also include a signal processor configured to compare, in the self-voice calibration mode of the hearing aid system, at least one electrical input signal and at least one auxiliary electrical input signal, or their corresponding transformations (or selected frequency ranges), for a corresponding current time period, wherein the corresponding current time period is recorded while the user is speaking, and to provide an estimate of the personalized self-voice transfer function from at least one microphone of the assistive device to at least one microphone (or more than two microphones) of the hearing aid (when worn by the user). When the user speaks in self-voice calibration mode, at least one microphone of the assistive device is preferably positioned close to the user's mouth (e.g., less than 0.1 m from the user's mouth).

[0048] This allows for the provision of hearing aid systems with improved functionality.

[0049] A signal processor may be located in the hearing aid. The hearing aid may be configured to receive at least one auxiliary electrical input signal from the assistive device. The signal processor of the hearing aid may be configured to receive at least one electrical input signal and at least one auxiliary electrical input signal or a corresponding transformation or a selected frequency range thereof for a corresponding time period, and to provide an estimate of the personalized self-voice transfer function from at least one microphone of the assistive device to at least one microphone of the hearing aid.

[0050] A signal processor may be located within the assistive device. The assistive device may be configured to receive at least one electrical input signal from the hearing aid. The signal processor of the assistive device may be configured to receive at least one electrical input signal and at least one auxiliary electrical input signal, or a corresponding transformation or a selected frequency range thereof, for a corresponding time period, and to provide an estimate of a personalized self-voice transfer function from at least one microphone of the assistive device to at least one microphone of the hearing aid. The assistive device may be configured to transmit the personalized self-voice transfer function to the hearing aid (or an external processing device).

[0051] The signal processor may be located in an external processing device (different from the hearing aid and assistive device). The external processing device may be configured to receive at least one electrical input signal from the hearing aid and at least one auxiliary electrical input signal from the assistive device. The signal processor of the external processing device may be configured to receive at least one electrical input signal and at least one auxiliary electrical input signal, or corresponding transformations or their selected frequency ranges, for corresponding time periods, and provide an estimate of the personalized self-voice transfer function from at least one microphone of the assistive device to at least one microphone of the hearing aid. The external processing device may be configured to transmit the personalized self-voice transfer function to the hearing aid. The external processing device may be a stationary device or part of it, such as a charging station or TV adapter, or a similar assistive device for a hearing aid. The external processing device may be configured to have more processing power and more energy than the hearing aid.

[0052] Hearing aids may also include a hearing aid signal processor configured to process electrical input signals or signals derived therefrom and to provide processed signals. The hearing aid signal processor may include the signal processor of the hearing aid system.

[0053] Hearing aids may also include an output unit, such as a speaker, for presenting a stimulus that represents a processed signal and can be perceived by the user as sound.

[0054] The assistive device (and / or external processing device) may include a user control interface that allows the user to activate the self-voice calibration operation mode of the hearing aid system.

[0055] A hearing aid system may include one or more detectors configured to determine whether (or with what probability) a user is currently wearing a hearing aid (or a hearing aid in a binaural hearing aid system) and provide a mode control signal identifying this. The hearing aid system may be configured to enable only self-voice calibration mode based on the mode control signal.

[0056] The hearing aid may include a beamforming filter configured to provide a self-voice beamformer comprising personalized filter weights determined based on an estimate of a personalized self-voice transfer function. This self-voice beamformer may be configured to amplify a signal originating from the user's mouth direction relative to sound signals from other directions.

[0057] The features of the first hearing aid system described above in detail in the specific embodiments can be combined with the second hearing aid system described above.

[0058] Hearing aids configured for use in hearing aid systems

[0059] On the other hand, the present invention provides hearing aids configured for use in the first and second hearing aid systems described in detail in the specific embodiments described above.

[0060] The hearing aid is suitable for wear by a user in or in the ear, or suitable for complete or partial implantation in the head. The hearing aid is suitable for establishing a communication link to an assistive device (such as a smartphone) so that data can be exchanged between them or forwarded from one device to another. The hearing aid may further include: an input unit comprising at least one microphone for picking up sound from the hearing aid environment and providing a corresponding electrical input signal representing the sound; a signal processor configured to process the at least one electrical input signal or a signal derived therefrom and provide a processed signal; and an output unit, such as including a speaker, for presenting a stimulus representing the processed signal that can be perceived by the user as sound.

[0061] Hearing aids may include antennas and transceiver circuitry, enabling the hearing aid to establish a communication link to assistive devices, allowing data to be exchanged between them or data to be forwarded from one device to another.

[0062] Hearing aids may include output converters for presenting stimuli that represent processed signals and can be perceived by the user as sound.

[0063] The hearing aid can be configured to receive an auxiliary electrical input signal provided by the microphone of the assistive device via the communication link. In the self-voice calibration operation mode of the hearing aid system, the signal processor can be configured to: A) receive at least one electrical input signal and at least one auxiliary electrical input signal, or a corresponding transformation or its selected frequency range, for a corresponding time period; and B) provide an estimate of the personalized self-voice transfer function from the microphone of the assistive device to at least one microphone of the hearing aid.

[0064] The signal processor can be configured to receive at least one electrical input signal for a corresponding time period and at least one auxiliary electrical input signal or a corresponding transformation or its selected frequency range provided by the microphone of the auxiliary device, and provide an estimate of the transfer function from at least one microphone of the auxiliary device to at least one microphone of the hearing aid.

[0065] The features of the hearing aid system described above in detail in the specific embodiments can be combined with hearing aids (where appropriate).

[0066] Hearing aids may include beamforming filters configured to provide a self-voice beamformer comprising personalized filter weights determined based on an estimate of a personalized self-voice transfer function. This self-voice beamformer can be used in a variety of applications requiring good estimates of the user's speech, such as hands-free calling, speech recognition (wake word, keyword detection), etc.

[0067] The hearing aid may include a beamforming filter configured to provide a self-voice cancellation beamformer that includes personalized filter weights determined based on an estimate of a personalized self-voice transfer function. This self-voice cancellation beamformer can be used in applications where only ambient sound is of interest (e.g., estimating noise during self-voice pickup or separating self-voice from ambient sound).

[0068] Hearing aids may include one or more self-voice-related algorithms, such as a voice control interface and / or a keyword detector. Self-voice-related algorithms can be optimized for speech from a specific physical user or artificial speech, such as using standard models like those from Brüel & The Head and Torso Simulator (HATS) 4128C from Sound & Vibration Measurement A / S, or the head and torso model KEMAR from GRAS Sound and Vibration A / S, or a similar computer model of the acoustic propagation characteristics of a human, are used to form a model. To this end, while still achieving improvements in personalized processing, the microphone signal from at least one microphone of the hearing aid can be pre-weighted (equalized) during a signal segment dominated by the self-voice signal (e.g., estimated using a self-voice detector). Specifically, when running self-voice-related algorithms during self-voice activity, the hearing aid (e.g., a signal processor) is configured to apply the following formula to the i-th microphone signal S... mics,i (k,l) weighted:

[0069] S i,modif (k,l)=d HATS,i (k) / d o,i (k)·S mics,i (k,l)

[0070] Where d o,i (k) is the OVTF of a specific user as estimated above, d HATS,i (k) is a set of OVTF coefficients measured on HATS or a similar physical or artificial model and stored in the hearing aid's memory (or in memory accessible by the hearing aid) (e.g., measured offline in the HA manufacturer's recording studio, estimated as described above). mics,i (k,l) refers to the time-frequency representation of the self-voice signal recorded by the i-th microphone for the user involved (e.g., provided by a Fourier transform algorithm such as STFT or DFT). Thus, the self-voice-related algorithm of the hearing aid can be optimized for another voice different from the user's self-voice (and thus again for different users), while still taking into account the acoustic propagation characteristics of the user's head and body.

[0071] The hearing aid may include one or more self-voice-related algorithms, such as a voice control interface and / or a speech recognition algorithm, which are optimized for voice from a specific physical person or for artificial or recorded voice from a standard model. During a signal segment dominated by the self-voice signal, at least one microphone signal is equalized according to the self-voice transfer function of the specific person or the model and the user, respectively.

[0072] Hearing aids may be adapted to provide frequency-varying gain and / or level-varying compression and / or frequency shifting (with or without frequency compression) from one or more frequency ranges to one or more other frequency ranges to compensate for a user's hearing loss. Hearing aids include a signal processor that can be configured to amplify the input signal and provide a processed signal.

[0073] Hearing aids may include an output unit for providing stimulation, perceived as an acoustic signal by a user, based on processed electrical signals. The output unit may include an electrode array of a cochlear implant (for CI-type hearing aids) or a vibrator of a bone conduction hearing aid. The output unit may include an output transducer. The output transducer may include a receiver (speaker) for providing the stimulation as an acoustic signal to the user (e.g., in acoustic (air conduction-based) hearing aids). The output transducer may include a vibrator for providing the stimulation as mechanical vibrations of the skull to the user (e.g., in bone-attached or bone-anchored hearing aids). The output unit may include a synthesis filter bank for converting frequency-domain signals to time-domain signals. The output unit may include a digital-to-analog (DA) converter to convert digital signals into analog output signals, for example, for presentation to the user via the output transducer.

[0074] Hearing aids may include an input unit for providing an electrical input signal representing sound. The input unit may include an input transducer, such as a microphone, for converting the input sound into an electrical input signal. The input unit may include a wireless receiver for receiving wireless signals that include or represent (e.g., sounds from the environment surrounding the hearing aid) and providing an electrical input signal representing said sound. The wireless receiver may, for example, be configured to receive electromagnetic signals in the radio frequency range (3 kHz to 300 GHz). The wireless receiver may, for example, be configured to receive electromagnetic signals in the optical frequency range (e.g., infrared light 300 GHz to 430 THz, or visible light, e.g., 430 THz to 770 THz). The input unit may include one or more analog-to-digital converters (where appropriate) for converting analog signals into signals having a certain sampling rate f. s The input unit may include one or more analytical filter banks (where appropriate) for converting the time-domain signal into a frequency-domain signal.

[0075] Hearing aids may include directional microphone systems (beamformers) adapted to spatially filter sound from the environment, thereby enhancing (or suppressing) a target sound source among multiple sound sources in the local environment of the hearing aid wearer. The directional system may be adapted to detect (e.g., adaptive detection) the direction from which a specific portion of the microphone signal originates. This can be achieved, for example, in a variety of different ways described in the prior art. In hearing aids, microphone array beamformers are commonly used to spatially attenuate background noise sources. Many beamformer variations can be found in the literature. Minimum variance distortionless response (MVDR) beamformers are widely used in microphone array signal processing. Ideally, an MVDR beamformer keeps the signal from the target direction (also known as the line of sight) unchanged while attenuating sound signals from other directions to the greatest extent possible. A generalized sidelobe canceller (GSC) structure is an equivalent representation of an MVDR beamformer, offering computational and digital representation advantages over a direct implementation of the original form.

[0076] Hearing aids may include antenna and transceiver circuitry (such as a wireless receiver) for receiving direct electrical input signals from another device, such as a communication device (e.g., a smartphone), a wireless microphone, an entertainment device (e.g., a television), or from another hearing aid. The direct electrical input signal may represent or include audio signals and / or control signals and / or status or information signals. Hearing aids may include demodulation circuitry for demodulating the received direct electrical input to provide the direct electrical input signal. Generally, the wireless link established by the antenna and transceiver circuitry of the hearing aid can be of any type. The wireless link may be based on near-field communication, such as an inductive link based on inductive coupling between the antenna coils of the transmitter and receiver sections. The wireless link may be based on far-field electromagnetic radiation.

[0077] Communication between the hearing aid and another (e.g., assistive) device can be in baseband (audio frequency range, such as between 0 and 20 kHz). Preferably, the frequency used to establish the communication link between the hearing aid and the other device is below 70 GHz, for example, in the range from 50 MHz to 70 GHz, for example, above 300 MHz, for example, in the ISM range above 300 MHz, for example, in the 900 MHz range, or in the 2.4 GHz range, or in the 5.8 GHz range, or in the 60 GHz range (ISM = Industrial, Scientific and Medical, such standardized ranges are defined, for example, by the International Telecommunication Union ITU). The wireless link can be based, for example, on standardized or proprietary technologies. The wireless link can be based, for example, on Bluetooth technology (such as Bluetooth Low Energy technology).

[0078] Hearing aids can have a maximum external size of 0.08m or 0.04m.

[0079] Hearing aids can be portable (i.e., configured to be wearable) devices or integral to them, such as devices that include an internal power source, such as a battery, for example a rechargeable battery. Hearing aids can be lightweight, easy-to-wear devices, for example, having a total weight of less than 20g.

[0080] A hearing aid may include a forward or signal path between an input unit (such as an input converter, e.g., a microphone or microphone system and / or a direct electrical input, e.g., a wireless receiver) and an output unit, such as an output converter, e.g., a speaker. A signal processor is located in this forward path. The signal processor may be adapted to provide frequency-varying gain according to the user's specific needs. The hearing aid may include an analysis path with functionalities for analyzing the input signal (e.g., determining level, modulation, signal type, acoustic feedback estimate, self-voice estimate, predetermined keywords, etc.). Some or all of the signal processing of the analysis path and / or signal path may be performed in the frequency domain. Some or all of the signal processing of the analysis path and / or signal path may be performed in the time domain.

[0081] Hearing aids can be configured to convert analog electrical signals representing sound signals into digital audio signals during analog-to-digital (AD) conversion, wherein the analog signals are sampled at a predetermined sampling frequency or sampling rate f. s Perform sampling, f s For example, in the range from 8kHz to 48kHz (to suit specific application needs) at discrete time points t n (or n, where n is the time exponent) provides a numerical sample x n (or x[n]). Each audio sample can be obtained through a predetermined N. b Bit represents the acoustic signal at t n The value of N at time b For example, in a range from 1 to 48 bits, such as 24 bits. Each audio sample therefore uses N. b Bit quantization (resulting in 2^n voltammetry of audio samples) Nb (Number of different possible values). The numerical sample x has 1 / f s The duration of the time, such as 50 μs, for f s =20kHz. Multiple audio samples can be arranged in time frames. A time frame may include, for example, 64 or 128 (or more) audio data samples. Other frame lengths may be used depending on the application.

[0082] Hearing aids may include filter banks, which include analysis filter banks that provide multiple sub-band signals from time-domain signals and synthesis filter banks that provide time-domain signals from multiple sub-band signals.

[0083] Hearing aids, such as input units and / or antenna and transceiver circuitry, may include a time-frequency (TF) conversion unit for providing a time-frequency representation of the input signal. The time-frequency representation may include an array or mapping corresponding to complex or real values ​​of the signal involved over a specific time and frequency range. The TF conversion unit may include an analytical filter bank for filtering the (time-varying) input signal and providing multiple (time-varying) output signals, each output signal comprising a distinctly different frequency range (sub-band signal) of the input signal. The TF conversion unit may include a Fourier transform unit for converting the time-varying input signal into a (time-)frequency signal in the (time-frequency) domain. The hearing aid considers a frequency range starting from the minimum frequency f. min up to the maximum frequency f max The frequency range can include a portion of the typical human hearing range from 20Hz to 20kHz, such as a portion of the range from 20Hz to 12kHz. Typically, the sampling rate f... s Greater than or equal to the maximum frequency f max twice, that is, f s ≥2f maxThe signals from the forward and / or analytical pathways of the hearing aid can be divided into NI (e.g., uniformly wide) frequency bands, where NI is, for example, greater than 5, greater than 10, greater than 50, greater than 100, or greater than 500, and at least some of these bands are processed individually. The hearing aid may be adapted to process the signals from the forward and / or analytical pathways (NP≤NI) on NP different channels. The channels may have consistent or inconsistent widths (e.g., width increases with frequency), and may overlap or not overlap.

[0084] The hearing aid can be configured to operate in different modes, such as a normal mode and one or more specific modes, which may be user-selectable or automatically selectable. Operating modes can be optimized for specific acoustic conditions or environments. Operating modes may include low-power modes, where the hearing aid's functionality is reduced (e.g., for energy saving), such as disabling wireless communication and / or disabling specific features of the hearing aid. Operating modes may include calibration modes, where the user's head-related transfer function can be determined according to the invention.

[0085] Hearing aids may include multiple detectors configured to provide status signals relating to the hearing aid's current network environment (such as the current acoustic environment), and / or the current state of the user wearing the hearing aid, and / or the current state or operating mode of the hearing aid. Alternatively or additionally, one or more detectors may form part of an external device that communicates with the hearing aid (e.g., wirelessly). External devices may include, for example, another hearing aid, a remote control, an audio transmission device, a telephone (e.g., a smartphone), external sensors, etc.

[0086] One or more of a plurality of detectors can operate on a full-band signal (time domain). One or more of a plurality of detectors can operate on a band-split signal ((time-)frequency domain), for example, in a finite number of frequency bands.

[0087] Multiple detectors may include level detectors for estimating the current level of the signal in the forward path. Detectors may be configured to determine whether the current level of the signal in the forward path is above or below a given (L-) threshold. Level detectors operate on full-band signals (time domain). Level detectors operate on band-split signals ((time-)frequency domain).

[0088] Hearing aids may include a voice activity detector (VAD) for estimating whether (or with what probability) an input signal (at a specific point in time) includes a voice signal. In this specification, a voice signal includes speech signals from humans. It may also include other forms of vocalization produced by the human speech system (such as singing). The voice activity detector unit is adapted to classify the user's current acoustic environment as a "voice" or "no-voice" environment. This has the advantage that time periods including electrophonic signals of human vocalizations (such as speech) in the user's environment can be identified and thus separated from time periods that include only (or primarily) other sound sources (such as artificially generated noise). The voice activity detector may be adapted to also detect the user's own voice as "voice." Alternatively, the voice activity detector may be adapted to exclude the user's own voice from the detection of "voice."

[0089] Hearing aids may include a self-voice detector for estimating whether (or with what probability) a particular input sound (such as speech) originates from the user of the hearing aid system. The microphone system of the hearing aid may be adapted to distinguish the user's own voice from another person's voice and possibly from non-voice sounds.

[0090] Multiple detectors may include motion detectors, such as accelerometers. Motion detectors may be configured to detect movements of a user's facial muscles and / or bones, such as those caused by speech or chewing (e.g., jaw movements), and provide detector signals indicating those movements.

[0091] The classification unit may be based on or include neural networks, such as trained neural networks, recurrent neural networks, or gated recurrent units (GRUs).

[0092] Hearing aids may also include other suitable functions for the applications involved, such as compression, noise reduction, and feedback control.

[0093] Hearing aids may include hearing instruments, such as hearing instruments adapted to be located in the user's ear or wholly or partially in the ear canal.

[0094] First step in hearing aids

[0095] On the other hand, hearing aids are provided that incorporate data-driven algorithms such as trained deep neural networks (DNNs). This neural network may include, for example, a recurrent neural network, such as a gated recurrent unit (GRU). The data-driven algorithm can be configured to implement an oral self-voice activity detector (OVAD). Other data-driven algorithms related to oral self-voice exist, such as keyword detection algorithms, hands-free calling algorithms, etc. The training of data-driven algorithms is described in the "Application 3: Online Personalization of Oral Self-Voice Driven Algorithms" section below.

[0096] The features of the hearing aid system or hearing aid described above in detail in the specific embodiments may be combined with a first further hearing aid (where appropriate).

[0097] The second step in hearing aids

[0098] On the other hand, hearing aids that include a signal processor are provided, which is used to apply multiple processing algorithms, possibly including data-driven algorithms such as neural network algorithms, and self-voice processing algorithms (such as self-voice detection algorithms, speech recognition algorithms such as keyword detection algorithms, etc.). The self-voice processing algorithm of the hearing aid can be optimized for the same OVTF, for example based on a standard model, such as a head and torso model (such as HATS or similar models). This is advantageous for development, troubleshooting, maintenance, and logistics. Personalized self-voice transfer functions for hearing aid users can be used to correct (equalize) the microphone signal of the hearing aid, as described in the "Application 4: OVTF Equalization" section below.

[0099] The features of the hearing aid system or hearing aid described above in detail in the specific embodiments may be combined with a second, further hearing aid (where appropriate).

[0100] The third step in hearing aids

[0101] On the other hand, a hearing aid is provided for playing audio to the user via an output converter of the hearing aid configured to apply a head-related transfer function to an audio signal wirelessly received from another device or system, or an internally stored or generated audio signal (beep or stored audio or audio generated in the hearing aid), before playing the audio to the user. This has the advantage that sounds such as telephone calls, voice notifications, and tinkling noises can be perceived by the user as originating from a location outside the user's body. A combination of a set of measured personal HRTFs and a set of HRTFs pre-measured for other directions not covered by the personal HRTFs (e.g., from a model such as HATS) can be used, as described in the "Application 5: Acoustic Presentation Reproduction Using HRTFs" section below.

[0102] The features of the hearing aid system or hearing aid described above in detail in the specific embodiments may be combined with a third, further hearing aid (where appropriate).

[0103] Hearing system including headphones

[0104] On the other hand, the present invention provides a hearing system including headphones and an assistive device. The headphones are configured to replace the hearing aid in the hearing aid system described above in detail in the specific embodiments.

[0105] The headset is suitable for use by a user, worn on or in the user's ear. The headset is suitable for establishing a communication link to an assistive device (such as a smartphone) so that data can be exchanged between them or forwarded from one device to another. The headset may also include: an input unit comprising at least one microphone for picking up sound (including the user's own voice) from the headset environment and providing a corresponding electrical input signal representing the sound; and an output unit, such as a speaker, for presenting stimuli perceived as sound to the user. The headset may also include a signal processor configured to receive at least one electrical input signal corresponding to a time period and at least one auxiliary electrical input signal or a corresponding transformation or selected frequency range thereof provided by the microphone of the assistive device, and to provide an estimate of the acoustic transfer function from the microphone of the assistive device to at least one microphone of the headset. The input unit may include two or more microphones.

[0106] The headset can be configured (via its input unit) to pick up the user's own voice and transmit it to a remote communication partner, and to receive sound from the remote communication partner and present it to the user (via the headset's output unit).

[0107] The headset's input unit may include at least two microphones, each providing an electrical input signal. These microphones may be located at or inside the user's ear. The headset may include a beamforming filter that incorporates one or more beamformers by applying appropriate (predetermined or adaptively determined) filter weights to the at least two electrical input signals. The one or more beamformers may include a self-voice beamformer incorporating personalized filter weights, configured to enhance signals originating from the user's mouth direction and suppress sound signals from other directions. The personalized filter weights may be determined from an estimate of the transfer function from at least one microphone of the assistive device to at least two microphones of the headset. The personalized self-voice beamformer may be configured to provide an improved estimate of the user's self-voice. The headset may be configured in a communication operation mode to transmit an estimate of the user's self-voice signal to another device, such as an assistive device (e.g., a smartphone).

[0108] The assistive device, such as a smartphone, may include at least one microphone for picking up sound from the assistive device's environment and providing at least one corresponding auxiliary electrical input signal representing the sound. The assistive device may also include a user control interface, allowing the user to activate a calibration operation mode for the hearing system. The assistive device may also include a speaker for playing test sounds. The assistive device is adapted to establish a communication link to the headset to enable data exchange or forwarding from one device to another.

[0109] The headset and auxiliary devices may include antennas and transceiver circuitry, thereby enabling the establishment of a communication link between them.

[0110] The headphones may include a single earpiece adapted to be positioned in the user's left and / or right ear.

[0111] The headset may include left and right earpieces adapted to be located at the user's left and right ears, respectively. The left and right earpieces may be configured to establish a communication link to enable data exchange therebetween. Each of the left and right earpieces may include corresponding input and output units. Each of the left and right earpieces may include at least two microphones, for example, located at or within each of the left and right ears (auricles).

[0112] The features of the hearing aid system and corresponding methods described above in detail in the specific embodiments can be combined with this hearing system and headphones (where appropriate).

[0113] application

[0114] On the one hand, the hearing aid system or hearing system described above, in detail in the specific embodiments, is used to determine personalized parameters for the processing algorithm of the hearing aid or headphones. The processing algorithm may be, for example, a directional algorithm, such as one for providing beamforming signals as a combination of multiple electrical input signals from multiple microphones (e.g., microphones of a hearing aid or headphones).

[0115] On the one hand, we provide an application for a hearing aid or headphone as described in detail in the "Detailed Description" section above. Personalized parameters can be used to determine the processing algorithm for the hearing aid or headphone.

[0116] For example, it can be used in systems that include one or more hearing aids (such as hearing instruments), headphones, headsets, active ear protection systems, etc., and in hands-free telephone systems, teleconferencing systems (such as those including loudspeakers), broadcasting systems, karaoke systems, classroom amplification systems, etc.

[0117] method

[0118] On the one hand, this application also provides a method for operating a hearing aid system (or hearing system). The hearing aid system (or hearing system) includes:

[0119] - Hearing aids (or headphones) suitable for wearing by a user in or in the ear, or suitable for being fully or partially implanted in the head; and

[0120] - Portable auxiliary devices, such as communication devices, like smartphones;

[0121] The hearing aid system (or hearing system) is adapted to establish a communication link between the hearing aid (earphone) and the assistive device so that data such as control signals, status signals and / or audio signals can be exchanged between them or forwarded from one device to another.

[0122] The method includes:

[0123] -In hearing aids (headphones),

[0124] --At least one electrical input signal representing the sound of the listening device (headphone) environment is provided by at least one microphone;

[0125] --Process at least one electrical input signal or a signal derived therefrom and provide a processed signal;

[0126] -- Present the processed signal, perceived as sound, to the user;

[0127] -In the auxiliary device,

[0128] --At least one auxiliary electrical input signal representing sound from the environment of the auxiliary device is provided by at least one microphone;

[0129] --Provides a user control interface that allows users to activate the calibration operation mode of the hearing aid system (or hearing system).

[0130] The method may further include: using at least one electrical input signal and at least one auxiliary electrical input signal for a corresponding time period, or their corresponding transformations or their selected frequency regions, to provide an estimate of the transfer function from at least one microphone of the auxiliary device to at least one microphone of the hearing aid (earphone).

[0131] The method, in self-voice calibration mode, includes positioning an assistive device in front of the user's mouth. The method may include positioning at least one microphone of the assistive device at a distance less than a maximum distance from the user's mouth. The maximum distance may be, for example, ≤0.2m or less than 0.1m. The method may include the user being instructed via a user control interface regarding the position of the assistive device. The method may include the user being instructed in self-voice calibration mode to provide one or more of the following: a) duration; b) loudness (vocal effort); and c) content and / or type of vocalization.

[0132] When appropriately replaced by a corresponding process, some or all of the structural features of the hearing aid system or hearing aid (or hearing system or headphones) described above in detail in the "Detailed Description" section can be combined with the implementation of the method of the present invention, and vice versa. The implementation of the method has the same advantages as the corresponding system and device.

[0133] The method may include playing a test sound signal to the environment via an assistive device based on input from a user control interface in the calibration operation mode. The method may also include transmitting the test sound signal from the assistive device to a hearing aid (or headphones).

[0134] Computer-readable media or data carrier

[0135] The present invention further provides a tangible computer-readable medium (data carrier) storing a computer program including program code (instructions), which, when the computer program is run on a data processing system, causes the data processing system (computer) to perform (complete) at least some (such as most or all) of the steps of the method described above in detail in the "Detailed Description".

[0136] By way of example, but not limitation, the aforementioned tangible computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to execute or store required program code in the form of instructions or data structures and is accessible by a computer. As used herein, disks include compact discs (CDs), laser discs, optical discs, digital multipurpose discs (DVDs), floppy disks, and Blu-ray discs, wherein these disks typically magnetically copy data while simultaneously being optically copied using lasers. Other storage media include those stored in DNA (e.g., in synthetic DNA strands). Combinations of the aforementioned disks should also be included within the scope of computer-readable media. In addition to being stored on tangible media, computer programs may also be transmitted via transmission media such as wired or wireless links or networks such as the Internet and loaded into data processing systems to run at locations other than tangible media.

[0137] Computer program

[0138] In addition, this application provides a computer program (product) including instructions that, when run by a computer, cause the computer to perform the methods (steps) described above in detail in the "Detailed Description".

[0139] Data processing system

[0140] In one aspect, the present invention further provides a data processing system, including a processor and program code, the program code causing the processor to perform at least some (such as most or all) of the steps of the method described above in detail in the "Detailed Description".

[0141] APP

[0142] On the other hand, the present invention also provides a non-transitory application called an APP. An APP includes executable instructions configured to run on an assistive device to implement a user interface for the hearing aid or hearing aid system (or headphones or hearing system) described above in detail in the "Detailed Description". The APP can be configured to run on a mobile phone such as a smartphone or another portable device enabled to communicate with said hearing aid or hearing aid system.

[0143] The user interface can be configured to enable users to control the functions of the hearing aid system, including activating the hearing aid system or the hearing aid's calibration operation mode.

[0144] The hearing aid system (including an app) can be configured to allow a user to activate the calibration operation mode of the hearing aid system via a user interface. The hearing aid system (including the app) can be configured to instruct the user, through the user interface of the assistive device, on how to position the assistive device relative to the user according to the selected calibration mode. In self-voice calibration mode, the user interface can be configured to instruct the user to position the assistive device such that at least one microphone of the assistive device is located next to the user's mouth. In self-voice calibration mode, the hearing aid system (including the app) can be configured to instruct the user, through the user interface of the assistive device, to speak a phrase or sentence (e.g., with a specific vocal effort, such as loud or soft, for example, based on the current noise level around the user) or to sustain it for a specific or minimum duration. In far-field calibration mode, the user interface can be configured to instruct the user to position the assistive device relative to the user (especially relative to the hearing aid microphone that will estimate its (acoustic) transfer function) in a preferred location, such as holding it in the hand, on a stick, or next to or on a table or other support.

[0145] definition

[0146] In this specification, a hearing aid, such as a hearing instrument, refers to a device suitable for improving, enhancing, and / or protecting a user's hearing ability. This is achieved by receiving sound signals from the user's environment, generating corresponding audio signals, possibly modifying those audio signals, and providing the possibly modified audio signals as audible signals to at least one of the user's ears. The audible signals may be provided, for example, as sound signals radiating into the user's outer ear, sound signals transmitted as mechanical vibrations through the bone structures of the user's head and / or through parts of the middle ear to the user's inner ear, and electrical signals transmitted directly or indirectly to the user's cochlear nerve.

[0147] Hearing aids can be configured to be worn in any known manner, such as as a unit worn behind the ear (having a tube that directs radiated sound signals into the ear canal or having an output transducer, such as a speaker, arranged close to or located within the ear canal), as a unit wholly or partially arranged in the auricle and / or ear canal, as a unit connected to a fixed structure implanted in the skull, such as a vibrator, or as a connectable unit that is wholly or partially implanted. Hearing aids may include a single unit or several units that communicate with each other (e.g., acoustically, electrically, or optically). The speaker may be housed within the housing along with other components of the hearing aid, or it may be an external unit (possibly combined with a flexible guiding element such as a dome-shaped element).

[0148] More generally, a hearing aid includes an input transducer for receiving sound signals from the user's environment and providing a corresponding input audio signal, and / or a receiver for receiving the input audio signal electronically (i.e., wired or wirelessly); signal processing circuitry (typically configurable) for processing the input audio signal (such as a signal processor, for example including a configurable (programmable) processor, such as a digital signal processor); and an output unit for providing an audible signal to the user based on the processed audio signal. The signal processor may be adapted to process the input signal in the time domain or in multiple frequency bands. In some hearing aids, amplifiers and / or compressors may constitute the signal processing circuitry. The signal processing circuitry typically includes one or more (integrated or separate) storage elements for executing programs and / or for storing parameters used (or potentially used) in the processing and / or for storing information suitable for the hearing aid's functionality and / or for storing information used, for example, in conjunction with an interface to the user and / or an interface to a programming device (such as processed information, for example, provided by the signal processing circuitry). In some hearing aids, the output unit may include an output transducer, such as a loudspeaker for providing airborne sound signals or a vibrator for providing sound signals propagating through structures or fluids. In some hearing aids, the output unit may include one or more output electrodes for (e.g., to a multi-electrode array) providing electrical signals for electrical stimulation of the cochlear nerve (cochlear implant hearing aid).

[0149] In some hearing aids, the vibrator may be adapted to transmit structurally propagated sound signals to the skull transcutaneously or through the skin. In some hearing aids, the vibrator may be implanted in the middle ear and / or inner ear. In some hearing aids, the vibrator may be adapted to provide structurally propagated sound signals to the middle ear bones and / or cochlea. In some hearing aids, the vibrator may be adapted to provide fluid-propagated sound signals to the cochlear fluid, for example, through the oval window. In some hearing aids, the output electrode may be implanted in the cochlea or on the medial side of the skull and may be adapted to provide electrical signals to the hair cells of the cochlea, one or more auditory nerves, the auditory brainstem, the auditory midbrain, the auditory cortex, and / or other parts of the cerebral cortex.

[0150] Hearing aids can be adapted to the specific needs of users, such as those with hearing loss. The configurable signal processing circuitry of a hearing aid can be adapted to apply frequency- and level-variable compression and amplification of the input signal. Customized frequency- and level-variable gain (amplification or compression) can be determined during the fitting process by the fitting system based on the user's hearing data, such as an audiogram, using basic fitting principles (e.g., speech adaptation). This frequency- and level-variable gain can be reflected, for example, in processing parameters, uploaded to the hearing aid via an interface to a programming device (fitting system), and used by a processing algorithm executed by the hearing aid's configurable signal processing circuitry.

[0151] A “hearing aid system” refers to a system that includes one or two hearing aids. A “binaural hearing aid system” refers to a system that includes two hearing aids and is adapted to work together to provide audible signals to both of a user’s ears. A hearing aid system or a binaural hearing aid system may also include one or more “assistive devices” that communicate with the hearing aids and influence and / or benefit from the functionality of the hearing aids. The aforementioned assistive devices may include at least one of the following: a remote control, a remote microphone, an audio gateway device, an entertainment device such as a music player, a wireless communication device such as a mobile phone (e.g., a smartphone), a tablet computer, or another device including a graphical interface. Hearing aids, hearing aid systems, or binaural hearing aid systems may be used, for example, to compensate for hearing loss in persons with hearing impairments, enhance or protect the hearing ability of persons with normal hearing, and / or transmit electronic audio signals to persons. Hearing aids or hearing aid systems may, for example, be part of or interact with broadcasting systems, active ear protection systems, hands-free telephone systems, car audio systems, entertainment (e.g., TV, music playback, or karaoke) systems, teleconferencing systems, classroom amplification systems, etc.

[0152] The present invention can be used, for example, in applications such as beamforming, hands-free calling, voice control, and keyword detection. Attached Figure Description

[0153] Various aspects of the invention will be best understood from the following detailed description taken in conjunction with the accompanying drawings. For clarity, these drawings are schematic and simplified, showing only the details necessary for understanding the invention while omitting other details. Throughout the specification, the same reference numerals are used for the same or corresponding parts. Features of each aspect may be combined with any or all features of other aspects. These and other aspects, features, and / or technical effects will be apparent from and illustrated in the following figures, wherein:

[0154] Figure 1A This illustrates the process of estimating the self-voice transfer function (OVTF) during calibration mode of a hearing aid system comprising a hearing aid and an assistive device such as a mobile phone, where the user's self-voice signal s ov (n) picked up by the microphone of a mobile phone and by the microphone in a hearing aid (signal s) i (n)), and these signals are used to estimate OVTFH ov,i (ω)(and relative OVTFd) ov,i (ω));

[0155] Figure 1B This shows the self-voice control signal during calibration mode from the start time t. start until stopping time t stop Temporal coherence;

[0156] Figure 2 The process of estimating (front) HRTF using an auxiliary device such as a mobile phone is shown, wherein the sound signal s is tested. f (n) The sound signal s emitted from the speaker of the mobile phone. i (n) picked up by the HA microphone;

[0157] Figure 3 A hearing aid system according to an embodiment of the present invention is shown;

[0158] Figures 4A-4C Together, we illustrate exemplary application scenarios of hearing systems according to embodiments of the present invention, wherein

[0159] Figure 4A The user, binaural hearing aid system, and assistive device are shown during the calibration process of the noise reduction system.

[0160] Figure 4B The app shown is used by the assistive device to initiate a calibration procedure that personalizes the self-voice transfer function.

[0161] Figure 4C The assistive device is shown running an app to initiate a calibration procedure that personalizes the head-related transfer function;

[0162] Figure 5 Embodiments of headphones or hearing aids including self-voice estimation are shown, along with options for transmitting the self-voice estimation to another device and receiving sound from the other device to present to the user via a speaker, for example, mixed with sound from the user's environment.

[0163] The further applicability of the invention will become apparent from the detailed description given below. However, it should be understood that while the detailed description and specific examples illustrate preferred embodiments of the invention, they are given for illustrative purposes only. Other embodiments of the invention will become apparent to those skilled in the art based on the following detailed description. Detailed Implementation

[0164] The detailed description below, taken in conjunction with the accompanying drawings, serves as a description of various different configurations. This detailed description includes specific details to provide a thorough understanding of several different concepts. However, it will be apparent to those skilled in the art that these concepts can be implemented without these specific details. Several aspects of the apparatus and method are described by various different blocks, functional units, modules, elements, circuits, steps, processes, algorithms, etc. (collectively, “elements”). Depending on the specific application, design constraints, or other reasons, these elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0165] Electronic hardware may include microelectromechanical systems (MEMS), (e.g., application-specific integrated circuits), microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), gating logic, discrete hardware circuits, printed circuit boards (PCBs) (e.g., flexible PCBs), and other suitable hardware configured to perform the various functions described in this specification, such as sensors for sensing and / or recording the physical properties of the environment, devices, users, etc. Computer programs should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, programs, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or other names.

[0166] An individual's self-voice transfer function (OVTF) can be easily estimated using a portable electronic device such as a mobile phone (or a similar communication device including a microphone and transmitter) or a wireless microphone. Assume the hearing aid (HA) system is in OVTF estimation mode (calibration mode), triggered, for example, by the HA user or a hearing care specialist (HCP) via a user interface such as an app (on the mobile phone). In this calibration mode, the hearing aid system can be configured to prompt the HA user to place the mobile phone in front of their mouth in a natural manner for a certain period, e.g., 1-10 seconds. For simplicity, the user can be asked to say specific sound elements, such as specific sentences (e.g., presented at the user interface, e.g., with a particular articulation effort, e.g., based on the ambient noise level). For OVTF estimation, the precise content of the speech signal is irrelevant. The OVTF estimation procedure should preferably take place in an acoustically quiet environment. This can be verified by the HA system, such as the hearing aid or the mobile phone, or a combination of both, before starting the estimation (calibration) procedure. Ideally, during calibration, the user should preferably be located away from reflective surfaces such as walls. Furthermore, ideally, auxiliary devices (such as mobile phones) should be positioned in a way that reduces reflections from the phone's surface to the microphone at the HA (e.g., when the user is standing, positioning it so that its maximum surface, such as its display, is horizontal, see [reference]). Figure 1A ).

[0167] Figure 1A The process of estimating the self-voice transfer function (OVTF) using a hearing aid system according to the present invention is illustrated. The hearing aid system includes a hearing aid (HD) and an assistive device (AD) such as a mobile phone. Figure 1A The calibration operation mode of the hearing aid system is shown, in which the user's self-voice signal s ov (n) The signal is picked up by the microphone ADM of the mobile phone AD and by the microphones (M1, M2, M3) in the hearing aid (providing the signal s). i(n), i = 1, 2, 3), and these signals are used to estimate the frequency-varying OVTFH from the mouth of the HA user (actually from the microphone of the mobile phone AD) to the microphone of the HA system. ov,i (ω)(i=1,2,3)(possible and relative OVTFd) ov,i (ω)), where ω refers to the (angular) frequency (2πf, where f is the frequency). Figure 1A The hearing aid includes a BTE portion adapted to be placed at or behind the auricle and an ITE portion adapted to be placed at or within the ear canal. The two portions are connected by a connecting element IC such as a sound tube or a cable (or both).

[0168] The voice signal of the HA user is generated by the microphone ADM in the telephone AD and the microphone M in the user's HA(HD). i Pick-up. From these signals, the acoustic transfer function from the HA user's mouth (actually from the telephone microphone) to the HA system's microphone can be estimated. The user can wear the hearing aid in one or both ears.

[0169] More specifically, let s ov (n) refers to the self-voice time-domain signal picked up by the microphone in the mobile phone, which is positioned in front of (and close to) the mouth reference point of the HA user. Furthermore, let s1(n),…,s M (n) refers to the corresponding speech signal picked up by M microphones of the HA (or in one HA at one ear, or in two HAs at both ears, or by other devices such as a single wireless microphone). Consider the Fourier transform of the picked-up signals and denote them as S. ov (ω) and S1(ω),…,S M (ω). Clearly, the acoustic transfer function (OVTF) from the mouth reference point to microphone i is given by the following equation:

[0170]

[0171] In practice, S i (ω) and S ov (ω) By applying the Discrete Fourier Transform (DFT) to the microphone signal s ov (n) and s1(n),…,s M (n) yields the following discrete acoustic transfer function:

[0172]

[0173] Where k is the frequency window exponent and K is the order of the DFT, such as 64 or 128.

[0174] For signal processing applications, it is often useful to aggregate the OVTFs of all microphones into a single vector:

[0175] H ov (k)=[H ov,1 (k)…H ov,M (k)] T

[0176] It is generally appropriate to consider defining the relative OVTF as follows (see the example below):

[0177]

[0178] Where 1≤i′≤M is the number of a pre-selected reference microphone (one of the microphones in the HA system, such as the front microphone of a hearing aid), and these are set into a relative OVTF vector, defined as:

[0179] d ov (k)=[d ov,1 (k)…d ov,M (k)] T

[0180] In summary, OVTFH ov (k)=[H ov,1 (k)…H ov,M (k)] T and relative OVTFd ov (k)=[d ov,1 (k)…d ov,M (k)] T From the microphone signal s ov (n) and s1(n),…,s M (n) Estimation. It should be noted that when estimating as described here, these OVTFs are individual OVTFs, meaning they reflect the individual acoustics (head shape, size, auricle, HA position) of a particular HA user. In practice, slightly more advanced, noise-robust, and data-efficient methods can be applied to estimate OFTFHs. ov,i (k)[Farina,2000], rather than simply forming H ov,i (k)=S i (k) / S ov The ratio of (k). The estimation process described above assumes that all corresponding signals are available for processing in one place, thus we assume that the corresponding signals are transmitted, for example, from a mobile phone (e.g., wirelessly) to the hearing aid system (or elsewhere).

[0181] Figure 1B The self-voice control signal OVD during calibration mode is shown from start time t. start until stopping time t stopTemporal coherence (time). A self-voice control signal equal to 1 indicates that within the calibration time period (t... start to t stop The value equals 0, indicating that user self-voice exists (or exists with a probability higher than a certain (e.g., the first) threshold) within the time range of t1 to t2; or equals 0, indicating that user self-voice does not exist (or exists with a probability lower than a certain (e.g., the second) threshold) outside the calibration time period of the time range [t1; t2].

[0182] Similarly, for those interested in estimating the (relative) acoustic transfer function of the microphone from the typical position of a conversation partner (or competing speaker) to the HA, we denote this acoustic transfer function as the preceding head-related transfer function (HRTF). The HRTF can be estimated by using a mobile phone as a wireless speaker. EP2928215A1 describes using an auxiliary device (such as a mobile phone) for self-calibration of a beamformer to identify non-self-voice sources of interest.

[0183] Assume the HA system is in (front-facing) HRTF estimation mode, for example, triggered by an HA user or hearing care specialist (HCP) via an app. The user holds the mobile phone in a front-facing position (typical position for a conversation partner) at height corresponding to the user's mouth and arm's length away, and the mobile phone's speaker emits a test sound signal from its speaker. f (n), and the probe signal is picked up by the microphone of the HA system worn by the user (see reference). Figure 2 ).

[0184] Figure 2 The process of estimating (previous) HRTF using an auxiliary device such as an AD converter, like a mobile phone, is illustrated, where a test sound signal (“test sound”) is used. f (n) The resulting sound signal s is emitted from the speaker AD-SPK of the mobile phone AD. i (n) picked up by HA microphone (M i (i = 1, 2, 3). Based on the transmitted and received signals, the acoustic transfer function H can be estimated. f,i (ω)(i=1,2,3)(or the corresponding impulse response h) f,i (n)).

[0185] The mobile phone's camera can be used (e.g., based on predetermined criteria) to provide feedback to the user that the phone is in the correct location. The duration of the test sound signal can range from hundreds of milliseconds to several seconds (e.g., from 1 second to 10 seconds. The longer the duration, the more accurately the HRTF can be estimated, but the higher the risk that the user cannot keep the phone or their head still). The precise content of the test sound signal is not critical, as long as the signal contains energy for all relevant frequencies (e.g., speech frequencies). Ideally, the estimation process takes place in acoustically quiet conditions and in a room with minimal reflections, such as a room with soft carpets, curtains, etc. Even if the measurement occurs in a reflective environment, later reflections can be removed from the estimated impulse response (IR) by truncating the "reverberation" IR tail.

[0186] In one embodiment, the telephone is mounted in a selfie stick. Based on the correlation between the hearing aid microphone and the mobile phone microphone (e.g., estimated by a hearing aid system such as a hearing aid or assistive device), the length of the selfie stick can be adjusted to achieve a desired distance between the hearing aid microphone and the telephone in front of the user.

[0187] exist Figure 2 The distance L between the hearing aid HD (and its microphones M1, M2, M3) and the assistive device AD ​​(and its speaker AD-SPK) is indicated in the diagram. This distance can be determined, for example, by a distance sensor located in the assistive device, such as a light sensor, or by the (predetermined) length of an arm or selfie stick. The hearing aid HD can be of any type, such as... Figure 2 As shown, it includes a BTE portion adapted to be located behind the user's ear (auricle) and an ITE portion adapted to be located in or within the user's ear canal. The two portions are connected via a connecting element IC (mechanical (e.g., acoustic) and / or electrical). Figure 2 In this embodiment, the BTE portion includes three microphones (M1, M2, M3), but other numbers may also be suitable. Two of the three microphones (planned) are positioned in the horizontal plane when the user is in an upright position to enable beamforming in the intended direction of the communication partner. By positioning one microphone outside the horizontal plane, more beamforming options are provided, such as in the direction of the user's mouth, for example, to pick up the user's voice.

[0188] When the assistive device is in the expected position relative to the user, the user can initiate (calibrate) the measurement (e.g., via a user interface, such as through the assistive device). The measurement can also begin when a certain distance is reached (e.g., determined by a distance sensor). In this way, the user does not need to actively initiate the measurement.

[0189] In this embodiment, the user is notified before the measurement begins (so that the user does not move during the measurement). The notification may occur via the phone screen, through audio from the phone, or through audio played via the hearing aid's output unit. This has the advantage of letting the user know not to move.

[0190] As mentioned earlier, let s1(n),…,s M (n) refers to the corresponding signal picked up by the microphone of the HA system. Now, from the mobile phone to the HRTFH in front of the i-th microphone... f,i (k) and the preceding relative HRTFd f,i (k)=H f,i (k) / H f,i′ (k) can be precisely combined Figure 1A , 1B The estimation is performed, but the self-voice signal s o (n) The test sound signal s generated by the mobile phone f (n) Substitution. The resulting preceding HRTF vector is denoted as:

[0191] h f (k)=[H f,1 (k)…H f,M (k)] T

[0192] And relative to the preceding HRTF, it is denoted as:

[0193] d f (k)=[d f,1 (k)…d f,M (k)] T

[0194] In practice, HRTF can be estimated using a slightly more complex procedure than that described in the previous sections. Specifically, advantageously, the test sound signal is a chirped signal (a tone signal whose frequency increases over time). In this case, HRTF can be estimated using the procedure outlined in [Farina, 2000].

[0195] HRTF can be measured for multiple sound source locations (angles), not just the front. Obviously, it's difficult for a person to hold a mobile phone at an angle relative to their nose, such as 25 degrees. However, hearing aid systems can be configured so that the assistive device (such as a telephone) provides feedback to the user (e.g., via speaker or screen) on whether / when the phone is in the correct position. This can be achieved using the phone's camera (e.g., based on user input about the location of interest, such as selecting from multiple predetermined locations, e.g., via a user interface). Once in the correct position, the phone emits a test sound signal and measures the HRTF as described above. This process can be repeated for a series of front-facing positions of the mobile phone.

[0196] example

[0197] Application 1: Personalized self-voice beamforming / noise reduction system

[0198] The application uses the OVTFd estimated as described above. o (k)=[d o,1 (k)…d o,M (k)] T .

[0199] For hands-free calling applications such as HA and voice control within HA systems, it is essential to obtain a clean (estimated) version of the user's voice signal, even in acoustically noisy conditions. To achieve this, a beamforming system based on the microphone signal from the HA system can be designed to enhance the signal originating from the user's mouth direction and suppress sound signals from other directions.

[0200] For example, it is well known that the filter coefficients of a minimum variance distortionless response (MVDR) beamformer are given by the following formula:

[0201]

[0202] Among them, C v (k,l) refers to the cross-power spectral density matrix at frequency k and time l (see, for example, [Jensen et al., 2015] and its use in estimating C). v (referencing the method in (k,l), where d(k) is the relative acoustic transfer function from the sound source of interest to the microphone that provides input to the MVDR beamformer.)

[0203] The estimated OVTF vector d o Substituting (k) into this expression yields a personalized self-voice beamformer:

[0204]

[0205] This results in a better self-voice acquisition / denoising tradeoff than when using, for example, an impersonal d(k) estimated from a head and torso simulator (HATS). Alternative self-voice acquisition systems readily follow, such as those based on multi-channel Zener filters, delay and summation beamformers [Brandstein et al., 2001], post-filter solutions informed by beamformers [Jensen et al., 2015], etc.

[0206] For 2: Personalized self-voice beamformer with front (front) interference immunity.

[0207] The application uses the OVTFd estimated as described above. o(k)=[d o,1 (k)…d o,M (k)] T And the preceding HRTFd estimated as described above f (k)=[d f,1 (k)…d f,M (k)] T .

[0208] This idea is an extension of the one described in "Application 1" above, where, in addition to capturing the user's self-voice signal, the spatial zeros are oriented forward to maximally suppress the assumed competing speaker. It is well known that the beamformer capable of this task is a special case of the linearly constrained minimum variance (LCMV) beamformer. The beamformer coefficient vector is obtained by solving the following equation:

[0209]

[0210] Subject to constraints:

[0211] w H (k,l)d o (k)=1

[0212] and

[0213] w H (k,l)d f (k)=0

[0214] As is well known, this problem follows a simple, closed-form solution [Haykin, 2001].

[0215] There are alternative solutions for LCMV beamformers, such as extending them directly with a post-filter.

[0216] Application 3: Online Personalization of Self-Voice-Driven Algorithms

[0217] The application uses the OVTFd estimated as described above. o (k)=[d o,1 (k)…d o,M (k)] T It is (not mandatory) assumed that a group of users' self-voices are recorded using HA microphones. An extension of this idea also uses the (previous) HRTFd estimated as described above. f (k)=[d f,1 (k)…d f,M (k)] T .

[0218] Assume that a data-driven algorithm exists within the HA system. Such algorithms typically involve trained deep neural networks (DNNs) to solve the relevant tasks. In the example below, we assume the algorithm is an Own Voice Activity Detector (OVAD), but this is just one example; other data-driven algorithms related to own voice activity exist, such as keyword detection algorithms, hands-free calling algorithms, etc.

[0219] For example, suppose OVAD is based on a deep neural network (DNN) and is trained to classify each time-frequency unit in the input signal as a) predominantly self-voice; b) predominantly non-self-voice (including background noise, external speakers, silence, etc.), see [Garde, 2019]. OVAD is used as a detector required by other algorithms, such as for estimating the noise cross-power spectral density matrix C. v Algorithms such as (k,l) are available, for example, see [Garde, 2019]. Traditionally, the aforementioned DNN-OVAD training is performed offline, i.e., before the HA is used, speech signals from many different speakers (male, female, children) are recorded by the HA in their individual ear canals. The resulting OVAD algorithm works well on average across a representative group of users; this is a speaker-independent algorithm.

[0220] However, if we use an individual OVTFd along with examples of voices from the users involved... o (k) The DNN can be retrained online (or further trained, also known as transfer learning), that is, during the use of HA, using artificially generated self-voice microphone signals. Specifically, the artificial self-voice signal can be generated according to the following formula:

[0221] S i (k,l)=d o,i (k)·S o (k,l)

[0222] Among them, S i (k,l) represents the short-time Fourier transform of the artificially personalized self-voice signal recorded at microphone i, and d o,i (k) represents the OVTF estimated as described above, and S o (k,l) represents the STFT of the user's self-voice recording. A time-domain version of the artificial self-voice microphone signal can be constructed by applying the inverse STFT to the STFT signal. If a recording of the user's self-voice is unavailable, other speech signals can be collected, such as from a speaker of the same gender as the user, if such information is available. In this case, the data-driven algorithm will personalize based on the OVTF rather than the user's voice characteristics.

[0223] Retraining (or continuous training) of the DNN during HA usage can be difficult due to the memory and computational complexity limitations of HA. This problem can be circumvented by wirelessly transmitting the relevant data (OVTF and optional self-voice signals and optional DNN parameters) to an external computing unit, which then feeds the resulting DNN weights back to the HA system after retraining.

[0224] As already mentioned, the proposed idea of ​​using OVTF and (optionally) recording of user self-voice is not limited to the OVAD example described above, but can be applied to personalize any data-driven algorithm onboard HA.

[0225] Extending this idea involves including preceding competing speakers in artificially generated training data. Specifically, noisy self-voice signals can be generated according to the following formula:

[0226] X i (k,l)=d o,i (k)·S o (k,l)+d f,i (k)·S f (k,l)+V(k,l)

[0227] Where, d f,i (k) is the (previous) HRTF, as measured as described in section 2.2, S f Let (k,l) be the STFT of the competing speakers' speech signals, and V(k,l) be an arbitrary noise signal representing an incoherent noise source in the acoustic environment. The competing speech signal S f (k,l) can be generated from any speech signal from a large number of male and female speakers (since competing speakers are usually unknown in practice), and V(k,l) can be generated from relevant acoustic noise, such as noise from a cafeteria setting or passengers on a train, recorded by the HA microphone on the HATS. Assume signal S... f (k,l) and V(k,l) exist in an external computing device, where (re)training of network weights occurs.

[0228] Application 4: OVTF Equalization

[0229] This idea uses the OVTFd estimated as described above. o (k)=[d o,1 (k)…d o,M (k)] T .

[0230] One approach to achieving personalized self-voice processing is to modify the actual signal processing algorithm occurring in the HA system, such as (re)training DNN weights to suit an individual's head acoustics (Example 3) or modifying beamformer weights to reflect an individual's head and torso acoustics. However, it may be desirable to maintain the same signal processing algorithm implementation (including DNN weights) for all users (such processing algorithms may include self-voice-related algorithms such as self-voice detection algorithms, speech recognition algorithms such as keyword detection algorithms, etc.). Specifically, it would be desirable if the onboard self-voice processing algorithm of the HA system were optimized for the same OVTF, such as HATS, which would simplify system development, troubleshooting, maintenance, and logistics.

[0231] To this end, while still achieving improvements in personalized processing, we propose pre-weighting or equalizing the microphone signal during signal regions dominated by self-voice (e.g., using OVAD estimation). Specifically, when running self-voice-related algorithms during self-voice activity, we propose pre-weighting or equalizing the i-th microphone signal S according to the following formula. mics,i (k,l) weighted:

[0232] S i,modif (k,l)=d HATS,i (k) / d o,i (k)·S mics,i (k,l)

[0233] Where, d o,i (k) is the OVTF of a specific user as estimated as described above, d HATS,i (k) is a set of OVTF coefficients measured on the HATS and stored in the HA memory (measured offline in the HA manufacturer's recording studio, for example, estimated as described above), and S mics,i (k,l) refers to the STFT of the self-voice signal recorded by the user on the i-th microphone.

[0234] The proposed equalization scheme transforms the self-voice microphone signal of a specific user into the self-voice microphone signal of the HATS (High-Amount Self-Voice System). This allows subsequent processing applied in the HA system to be optimized for the HATS, independent of the actual user. In other words, the post-equalization processing is the same for all users.

[0235] Application 5: Acoustic Reproduction Using HRTF

[0236] This idea uses the (previous) absolute HRTFH estimated as described above. f (k)=[H f,1 (k)…H f,M (k)] TOptionally, this idea uses the preceding HRTF in addition to the absolute HRTF measured from other directions different from the preceding ones.

[0237] We propose combining a set of measured individual HRTFs with a set of pre-measured HRTFs for other directions not covered by the individual HRTFs (e.g., from HATS). We propose using the combined HRTFs for spatially realistic reproduction of acoustic signals for hearing device users. Specifically, the combined HRTFs make it possible to reproduce sounds of interest to the user, such as telephone calls, audio notifications, jingles, etc., as if they originated from a location outside the user's body, such as in front or slightly to the left, or to more realistically reproduce environmental signals using more or all of the combined HRTFs.

[0238] Specifically, without loss of generality, let i = 1 refer to the number of the HA microphone closest to the user's left eardrum, and let i = 2 refer to the number of the HA microphone closest to the user's right eardrum. Similarly, still without loss of generality, consider reproducing the sound source as originating from the previous location (for example). Therefore, H f,1 (k) refers to the acoustic transfer function from the position in front of the user to their left ear, while H f,2 (k) refers to the acoustic transfer function from the same position in front of the user to their right ear.

[0239] Then, the sounds that the user is interested in can be reproduced from the previous source using the following formula:

[0240] S i (k,l)=H f,i (k)S(k,l),i=1,2

[0241] Where S(k,l) is the STFT of the sound of interest, and S1(k,l) and S2(k,l) are the STFTs of the signals presented to the user's left and right ears, respectively.

[0242] This method can be generalized to the synthesis of more complex sound fields according to the following formula:

[0243]

[0244] Where S j (k,l) represents the STFT of the component of the sound of interest originating from position j, H j,i (k) is the HRTF (personalized or HATS-based) from position j to the microphone closest to the i-th ear, and S i(k,l) represents the STFT of the sound to be presented to the i-th ear. The position index j can span some or all of the combined HRTF set (i.e., personal and HATS-based HRTFs). The advantage of including personal HRTFs over using all HATS-based HRTFs is that spatial sound perception becomes more realistic for individual users.

[0245] Figure 3 A hearing aid system according to an embodiment of the present invention is illustrated. The hearing aid system HAS includes a hearing aid HD adapted for wear by a user U in or in the user's ear, or adapted for complete or partial implantation in the head of the user's ear. The hearing aid system HAS also includes a portable or wearable assistive device AD, such as a communication device, like a smartphone or similar device, and / or an application (APP) configured to run on the assistive device (see [link to documentation]). Figure 4A , 4B The hearing aid system (HAS) is adapted to establish a communication link (WL-RF) between the hearing aid (HD) and the assistive device (AD) so that data can be exchanged between them or data can be forwarded from one device to another.

[0246] The hearing aid HD includes an input unit IU comprising at least one microphone (in this case, two, M1, M2) for picking up sound from the hearing aid environment and providing at least one electrical input signal (S1(ω), S2(ω)) representing the sound (where ω may represent a frequency). The input unit IU may include an analog-to-digital converter to provide the electrical input signal in digital form as digital samples, and an analysis filter bank for providing the electrical input signal as a sub-band signal, as appropriate for the application in question. The hearing aid HD also includes a signal processor SPU configured to perform processing within the hearing aid. The signal processor SPU may include a hearing aid processor portion HAP configured to process at least one electrical input signal or a signal derived therefrom and provide a processed signal OUT. The hearing aid HD also includes an output unit OU, such as including a loudspeaker, a vibrator, or a multi-electrode array, for presenting a perceptible sound stimulus (such as acoustic vibration or electrical stimulation) representing the processed signal OUT to the user, see [link to relevant documentation]. Figure 3 The solid arrow in the diagram, denoted as "stimulus," runs from the output unit OU to the user U. The signal path from the input unit to the output unit (via the hearing aid processor section HAP) can be referred to as the "forward path" in the hearing aid.

[0247] The assistive device AD ​​includes at least one microphone AD-M for picking up sound from the environment of the assistive device AD ​​and providing at least one auxiliary electrical input signal ADM-IN corresponding to the sound. The assistive device AD ​​also includes a user control interface (UI), such as a touchscreen keyboard, enabling the user U to activate the calibration operation mode of the hearing aid system (HAS), see [link to relevant documentation]. Figure 3The user interface (UI) from user U to auxiliary device AD ​​is represented by a solid arrow labeled "V control" and a symbolic hand labeled "T control". (Example:) Figure 3 As shown, the user control interface (UI) may include a touch-sensitive display, such as one or more fingers operated by the user (see “T control” provided by the user’s fingers), and / or may include a voice control interface that responds to spoken commands (see the thick arrow from the user’s mouth to the user interface UI, denoted as “V control”).

[0248] Figure 3 An embodiment of the hearing aid system includes a memory MEM (located in the hearing aid) for storing a time period of each of at least one electrical input signal (here, signals (S1(ω), S2(ω))). The memory MEM is also configured to store a time period of at least one auxiliary electrical input signal ADM-IN, represented by a signal ADin received from the assistive device AD ​​via a wireless communication link WL-RF, such as a Bluetooth-based or similar technology communication link. The communication link is implemented via corresponding antennas and transceiver circuitry (TX / RX, ANT) of the two devices.

[0249] The signal processor (SPU) of the hearing aid HD is configured to compare at least one electrical input signal (S1(ω), S2(ω)) and at least one auxiliary electrical input signal ADin or their corresponding transformations for corresponding time periods, and provide estimates of the transfer functions (HRTF, OVTF) from the auxiliary device AD ​​(e.g., from at least one microphone ADM or from the loudspeaker AD-SPK of the auxiliary device) to at least one microphone (M1, M2) of the hearing aid HD. Figure 3 In this embodiment, the task is performed by the controller TF-PRO, which receives the mode control signal MCtr from the user interface (UI) of the assistive device via the communication link WL-RF. In the calibration operation mode of the hearing aid system, the controller TF-PRO is configured to compare the electrical input signals for corresponding time periods (Tseg) and determine the corresponding acoustic transfer functions of the system based on them, as described in detail above. As outlined and illustrated above, these acoustic transfer functions can be, for example, the self-voice transfer function (OVTF) from the user's mouth to each of the microphones (M1, M2). The OVTF can be used, for example, in a self-voice beamformer OV-BF with a custom (personalized) beamformer filter, which can be used to provide an estimate of the user's voice, OwnV, based on two electrical input signals S1(ω), S2(ω) from the corresponding microphones (M1, M2) of the hearing aid HD, for example, in a (subsequent) telephone operation mode (see, for example, see...). Figure 5In this system, an estimate of the user's own voice is transmitted to the telephone (here, for example, an auxiliary device) and then further transmitted to a remote receiver via a communication network (such as the PSTN or an Internet-based service like Skype or WhatsApp). Figure 3 In one embodiment, the estimated user voice, OwnV, is further processed in the processor portion of OVP (e.g., undergoing a noise reduction algorithm, such as a post-filter) to provide an improved estimate of the user's own voice, OVest. Figure 3 In this embodiment, the improved estimate of the user's voice, OVest, is passed to a remote receiver. This estimate of the user's voice can also be used in a keyword detector, such as in a hearing aid that supports a voice control interface, or for controlling the functions of an assistive device, or for verification by a processor in the assistive device, or for transmission to a server (such as in the cloud) for further processing.

[0250] When at least one microphone ADM of the auxiliary device AD ​​is positioned close to the user's mouth, for example, in front of the user's mouth (e.g., as in combination) Figure 1A (as described above), when the user raises their voice, for example, when speaking, during the calibration operation mode of the hearing aid system (see...). Figure 1B The transfer function can represent the self-voice transfer function (OVTF) within the time interval [t1; t2]. The hearing aid system can be configured, for example, via a user interface (UI) through an assistive device (AD) (see, for example, see...). Figure 4B The system instructs the user to say a phrase or sentence (e.g., with a specific pronunciation effort, such as loud or soft, and / or a specific duration, such as based on the current noise level around the user).

[0251] The mode control signal MCtr from the user interface (UI) can be used, for example, to control the hearing aid signal processor HAP in the positive path between the input unit IU and the output unit OU of the hearing aid HD; see control signal HActr. Figure 3 In one embodiment, the forward path further includes a beamformer filter comprising a (far-field) beamformer FF-BF that enables the beamforming signal YBF to be focused onto a (far-field) communication partner (e.g., 1 m or more away from the user's head (hearing aid)). The (far-field) beamformer is connected to a hearing aid signal processor HAP, which, for example, applies one or more processing algorithms to the beamforming signal YBF (or a signal derived therefrom) and provides a processed signal OUT, which is fed to an output unit OU for presentation to the user. One or more processing algorithms may include, for example, one or more of the following: noise reduction, compression amplification (to compensate for the user's hearing impairment), feedback control, etc.

[0252] The filter weights of a user-customized (personalized) (far-field) beamformer FF-BF can be used to determine the head-related transfer function HRTF (see, for example). Figure 2 Instead of the self-voice transfer function OVTF (see e.g., see [link]). Figure 1A In the calibration mode, the test sound (calibration sound) is played through the speaker of the auxiliary device and performed by this embodiment of the hearing aid system (as described and illustrated above).

[0253] The auxiliary device AD ​​(in this embodiment) may therefore preferably include a speaker AD-SPK, and the auxiliary device may be configured to play test sound signals to the environment of the auxiliary device via the speaker according to input from the user control interface UI in calibration operation mode (see [link]). Figure 2 "Test Sounds" f (n)), for example see Figure 4C .

[0254] In calibration mode, the assistive device is positioned in a preferred location relative to the user (the hearing aid microphone from which the estimated (acoustic) transfer function will be derived), such as being held in the hand or positioned next to a table or other support. The preferred location relative to the user (e.g., distance, angle, etc.) can be known in advance (e.g., by mounting the assistive device on a pole of known length (e.g., a selfie stick)) or estimated during calibration, for example using one or more sensors such as the assistive device and / or the hearing aid, such as camera and / or radar sensors. The hearing aid system HAS can be configured such that data representing the estimated position of the microphones (M1, M2) of the speaker AD-SPK relative to the hearing aid HD can be obtained by the hearing aid (e.g., transmitted to the hearing aid) (e.g., via the communication link WL-RF) and, for example, formed as part of the mode control signal MCtr fed to the controller TF-PRO.

[0255] The auxiliary device AD ​​includes a controller CNT configured to provide a test or calibration signal CalS in (far-field) calibration operation mode, which is fed to and played by the speaker AD-SPK to provide a test sound signal (see [link]). Figure 2 The controller CNT may include a test signal generator for providing a test signal CalS. The test signal may be transmitted directly (electromagnetically) to the hearing aid via the communication link WL-RF, see the signal CalS' fed to the transceiver TX / RX of the assistive device. In the hearing aid, the wirelessly received test signal (based on CalS') may be represented by the signal ADin, and in calibration operation mode, electrical input signals (S1(ω), S2(ω)) representing the (electrical) version of the test sound signal received at the hearing aid microphones (M1, M2) for the corresponding time period are stored in the memory MEM. Thus, the hearing aid controller TF-PRO can determine the (frequency-varying) HRTF (as described above) of the sound propagating from the current position of the speaker AD-SPK relative to the hearing aid microphones (M1, M2).

[0256] The assistive device AD ​​is configured to enable the control input UCtr from the user control interface UI to control the transmission of the microphone signal ADM-IN and / or the test / calibration signal CalS' and / or other control signals UCtr from the assistive device to the hearing aid, such as mode control signals for turning on and / or off calibration mode and / or other operating modes of the hearing aid (such as telephone mode).

[0257] exist Figure 1A-3 In some embodiments, communication between the hearing aid HD and the assistive device AD ​​may be in the baseband (audio frequency range, such as 0 to 20 kHz). However, preferably, communication between the hearing aid and the assistive device is based on some type of modulation at a frequency higher than 100 kHz. Preferably, the frequency used to establish a communication link between the hearing aid and the assistive device is below 70 GHz, for example, in the range from 50 MHz to 70 GHz, for example, above 300 MHz, for example, in the ISM range above 300 MHz, for example, in the 900 MHz range, or in the 2.4 GHz range, or in the 5.8 GHz range, or in the 60 GHz range (ISM = Industrial, Scientific and Medical, such standardized ranges are defined, for example, by the International Telecommunication Union ITU). In some embodiments, the wireless link is based on standardized or proprietary technologies. In some embodiments, the wireless link is based on Bluetooth technology (such as Bluetooth Low Energy technology) or related technologies.

[0258] Figures 4A-4C Together, we illustrate exemplary application scenarios of a hearing system embodiment according to the present invention.

[0259] Figure 4A The image shows a user, a binaural hearing aid system or hearing system, and an assistive device during a calibration process (e.g., processing algorithms such as noise reduction systems). Figure 4A An embodiment of a head-mounted binaural hearing aid system or hearing system is shown, comprising left and right hearing devices (HDs) communicating with a portable (wearable) assistive device (AD) serving as a user interface UI for the binaural hearing aid system or hearing system. l HD r (e.g., the earpiece of a hearing aid or earphone). In an embodiment, a binaural hearing aid system or hearing system includes an assistive device (AD) (and a user interface UI). An exemplary screen of the user interface UI of the assistive device AD ​​is shown below. Figure 4B and 4CAs shown in the diagram. The user interface includes a display (such as a touch-sensitive display) that shows the user instructions on selecting and activating (or deactivating) the calibration operation mode of the hearing aid system or hearing system. The user interface is implemented as an application (APP) on an assistive device (such as a smartphone). This APP is referred to as the "Calibration APP". Through the display of the user interface UI, the user is instructed to select either far-field calibration (HRTF) or self-voice calibration (OVTF). The calibration type is selected by pressing the relevant "button", and when the selected type is indicated by a gray shade, instructions for performing the specific calibration procedure are shown at the bottom of the screen. The screens for the two calibration types are respectively displayed on... Figure 4B and 4C As shown in the image.

[0260] Figure 4B The auxiliary device AD ​​is shown running an APP to enable a calibration procedure that personalizes the self-voice transfer function.

[0261] The instructions used to calibrate the OVTF (Autonomous Voice Transfer Function) are:

[0262] - Horizontal positioning device (microphone close to the mouth);

[0263] - Keep your head still and do not move the device during calibration;

[0264] - Normally, it takes about 10 seconds.

[0265] These instructions will prompt the user:

[0266] - The device is positioned such that its microphone input is close to the user's mouth (e.g., ≤0.1m away) while attempting to minimize reflections of the user's voice (reflections may provide reverberation-like interference, thus degrading the quality estimated by the OVTF).

[0267] - Preferably, the device (and body) should be kept as still as possible during calibration (estimated to be 10 seconds);

[0268] - Speak normal sentences during the calibration period (e.g., with normal pronunciation effort). Another instruction could be to ask the user to read specific text that is known to "excite" the corresponding frequency range of the user's voice;

[0269] Press the Start / Stop button to begin the calibration process.

[0270] Figure 4C The auxiliary device is shown running an app to initiate a calibration procedure that personalizes the head-related transfer function.

[0271] The instructions for calibrating the Head-Related Transfer Function (HRTF) are as follows:

[0272] Position (e.g., hold) the device in the planned location so that the screen faces you (speakers at ear height);

[0273] -Activate selfie mode;

[0274] - Keep your head still and do not move the device during calibration (while the test sound is playing).

[0275] These instructions will prompt the user:

[0276] - Place the auxiliary device relative to the user at the expected location (direction and distance) of the target sound source, such as in front of the user, such as at a distance of ≥1m from the user, such as holding the auxiliary device in the hand or mounting it on a pole (such as a selfie stick);

[0277] - Activate the camera operating mode, where the screen displays "Mirror Yourself." This helps position the assistive device at the correct height (and facilitates automatic positioning sensing using the camera image). Preferably, the assistive device should be at the same level as the user's eyes (and ears);

[0278] - Preferably, the device (and body) are kept as still as possible during calibration, which can be verified by the user through the perception of test sounds (the calibration process is estimated to be 10 seconds, for example). The auxiliary device's camera can record the user while the sound is playing (thus enabling estimation of possible movements during calibration);

[0279] Press the Start / Stop button to begin the calibration process.

[0280] The Start / Stop button can also be used to terminate the calibration process, for example, if something is wrong (sudden movement, noise, other activity, etc.).

[0281] This may include an acceptance step, requesting the user to accept the calibration measurement results (giving the user the opportunity to discard the results if, for some reason, the results are not as expected, such as due to noise or other unplanned events during the measurement).

[0282] Preferably, the start time of the calibration procedure (pressing "Start") (and possibly the start time (and / or end time) of the calibration signal), the selected location (e.g., angle and distance relative to the user), and possibly the characteristics of the calibration signal (quantity versus frequency, spectrum, or the calibration signal itself (or a portion thereof), etc.) are transmitted to the left and right hearing devices to determine a customized Head-Related Transfer Function (HRTF) or OVTF. The customized (personalized) transfer function can be used, for example, to select an appropriate (e.g., predetermined) set of filter weights, or to calculate such weights, for example, for an appropriate beamformer (see, for example, see...). Figure 3 (FF-BF and OV-BF in the text).

[0283] Figure 5 An example of applying the personalized transfer function according to the present invention is shown. Figure 5Embodiments of headphones or hearing aids including self-voice estimation are illustrated, along with options for transmitting the self-voice estimation to another device and receiving sound from the other device for presentation to the user via a speaker, for example, mixed with sound from the user's environment. The hearing aid or headphones (HD, referred to herein as a hearing device) include two microphones (M1, M2) configured to provide electrical input signals (IN1, IN2) representing sound in the user's environment. The hearing device also includes two beamformers, FF-BF and OV-BF, each beamformer providing spatially filtered signals (ENV and OV, respectively) based on the electrical input signals (IN1, IN2) and personalized beamformer weights according to the invention. The (far-field) beamformer FF-BF may, for example, implement a target-preserving, target-cancelling beamformer, which includes beamformer weights determined by a personalized acoustic transfer function (HRTF) estimated according to the invention. The self-voice beamformer OV-BF is configured to pick up user voice (originating from the user's mouth and throat) and includes beamformer weights determined by the personalized acoustic transfer function OV-TF estimated according to the present invention. The hearing device may, for example, include a self-voice detector configured to detect (in, for example, at least one electrical input signal) whether (or with what probability) user self-voice is present at a given time point and provide a self-voice presence control signal indicating its presence. The self-voice beamformer OV-BF may, for example, be activated based on a self-voice presence control signal and / or a telephone mode control signal and / or a remote speaker presence control signal and / or a user-initiated control signal (e.g., via a user interface UI). In telephone operation mode (such as normal headset operation mode), the user's self-voice is picked up by microphones M1 and M2 and spatially filtered by the self-voice beamformer OV-BF, thus providing an enhanced self-voice signal OVOUT, which is fed to transmitter Tx and transmitted (via cable or wireless link to another device or system (such as a telephone, see the dashed arrow labeled "to telephone" and the telephone symbol)). In telephone operation mode (such as normal headset operation mode), the signal PHIN can be received by the (wired or wireless) receiver Rx from another device or system (such as a telephone, as shown by the telephone symbol and the dashed arrow labeled "from telephone"). When the remote speaker is active, the signal PHIN contains the voice from the remote speaker, for example, transmitted via telephone line (e.g., entirely or partially wireless, but usually at least partially cable-based). The "remote" telephone signal PHIN can be selected in the combination unit (here, the selector / mixer SEL-MIX) or mixed with the ambient signal ENV from the far-field beamformer FF-BF. The selected or mixed signal PHENV is fed to the output converter SPK (such as a speaker or the vibrator of a bone conduction hearing device) to be presented to the user as sound. (Optionally, such as...) Figure 5As shown, the selected or mixed signal PHENV can be fed to the processor PRO, thereby applying one or more processing algorithms to the selected or mixed signal PHENV to provide the processed signal OUT, which is fed to the output converter SPK. Figure 5 An embodiment of the hearing device HD can represent headphones, in which the received signal PHIN can be selected to be presented to the user without being mixed with ambient signals. Figure 5 An example of this can represent a hearing aid in which the received signal PHIN can be mixed with an ambient signal before being presented to the user (so that the user can retain a sense of their surroundings; of course, this is also suitable for headphone applications, depending on the usage). Furthermore, in the hearing aid, the processor PRO can be configured to compensate for the user's hearing loss (signal PHENV).

[0284] When appropriately replaced by a corresponding process, the structural features of the apparatus described above in detail in the "Detailed Embodiments" can be combined with the steps of the method of the present invention.

[0285] Unless explicitly stated otherwise, the singular forms “a” and “the” used herein include the plural forms (i.e., meaning “at least one”). It should be further understood that the terms “having,” “comprising,” and / or “including” as used in the specification indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. It should be understood that, unless explicitly stated otherwise, when an element is referred to as “connected” or “coupled” to another element, it may be a direct connection or coupling to the other element, or there may be intermediate inserting elements. The term “and / or” as used herein includes any and all combinations of one or more of the listed related items. Unless explicitly stated otherwise, the steps of any method disclosed herein do not necessarily have to be performed in the exact order disclosed.

[0286] It should be understood that references to "an embodiment," "an embodiment," "an aspect," or "may" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Furthermore, particular features, structures, or characteristics may be suitably combined in one or more embodiments of the invention. The foregoing description is provided to enable those skilled in the art to implement the various aspects described herein. Various modifications will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects.

[0287] This invention is not limited to the aspects shown herein, but encompasses the full scope consistent with the language of this invention, wherein, unless expressly stated, an element referred to in the singular does not mean "one and only one," but rather "one or more." Unless expressly stated, the term "some" means one or more.

[0288] Therefore, the scope of this invention should be determined based on the claims.

[0289] References

[0290] [Farina,2000]:Farina,Angelo."Simultaneous measurement of impulseresponseand distortion with a swept-sine technique."Audio Engineering SocietyConvention108.Audio Engineering Society,2000

[0291] [Jensen et al., 2015]: J.Jensen and MSPedersen, "Analysis ofBeamformerDirected Single-Channel Noise Reduction System for Hearing AidApplications", Proc.Int.Conf.Acoust.,Speech,Signal Processing,pp.5728-5732, April 2015.

[0292] [Brandstein et al., 2001]: M. Brandstein and D. Ward (Eds.), “MicrophoneArrays–Signal Processing Techniques and Applications,” Springer, 2001.

[0293] [Haykin,2001]: S.Haykin, "Adaptive Filter Theory", Prentice Hall, 2001.

[0294] [Heymann,et al.,2017]J.Heymann,L.Drude,R.Haeb-Umbach,”A GenericNeuralAcoustic Beamforming Architecture for Robust Multi-Channel SpeechProcessing,”Computer,Speech and Language,Vol.46,pp.374-385,Nov.2017.

[0295] [Garde,2019].J.Garde,“Own-Voice Retrieval for Hearing AssistiveDevices:ACombined DNN-Beamforming Approach,”Master’s Thesis,AalborgUniversity,2019.

[0296] EP2928215A1(Oticon)07.10.2015.

Claims

1. A hearing aid suitable for a user to wear on or in their ear, comprising: Antenna and transceiver circuitry for establishing a communication link with an auxiliary device configured as a user control interface; An input device comprising at least one microphone for picking up sound from the hearing aid environment to provide at least one electrical input signal representing the sound; A signal processor configured to process the at least one electrical input signal or a signal derived from the at least one electrical input signal and provide a processed signal; The output unit is used to present the processed signal, which can be perceived as sound, to the user. The signal processor is configured to, at the time when the assistive device notifies the hearing aid, based on the test sound signal picked up by the at least one microphone from the loudspeaker of the assistive device and the electrical version of the test sound signal received by the antenna and transceiver circuit from the transmission of the assistive device, start measuring the corresponding head-related transfer function in far-field calibration operation mode.

2. The hearing aid according to claim 1, comprising a memory wherein at least one electrical input signal, or a corresponding transformation, or a selected frequency region thereof, for a plurality of time periods can be stored.

3. The hearing aid of claim 1, wherein the hearing aid comprises at least two microphones, wherein one of the at least two microphones is defined as a reference microphone.

4. The hearing aid according to claim 1, comprising a distance sensor for estimating the distance between the assistive device and the hearing aid.

5. The hearing aid of claim 1, wherein the hearing aid is configured to store at least one electrical input signal for a period of time when a test sound signal is picked up by at least one microphone of the hearing aid in far-field calibration operation mode.

6. The hearing aid of claim 1, wherein the signal processor of the hearing aid is configured in far-field calibration operation mode to receive the electrical version of the test sound signal and the at least one electrical input signal for a period of time and determine, based thereon, a head-related transfer function from the speaker position of the assistive device to at least one microphone of the hearing aid.

7. The hearing aid of claim 5, wherein the signal processor of the hearing aid is configured in far-field calibration operation mode to receive an electrical version of a test sound signal and at least one electrical input signal for the time period and to determine a relative head-related transfer function between at least two microphones based thereon.

8. The hearing aid of claim 1, comprising a beamforming filter configured to provide one or more beamformers, the filter weights of the one or more beamformers being personalized using the head-related transfer function.

9. The hearing aid of claim 1, configured to exchange data with another hearing aid via the antenna and transceiver circuitry.

10. The hearing aid of claim 1, wherein the assistive device is an electronic device configured to provide a user control interface that enables a user to control the functions of the hearing aid.

11. An assistive device configured to provide a user control interface for a hearing aid, the hearing aid being adapted for wear by a user in or on their ear and including a signal processor, the assistive device comprising: Antenna and transceiver circuitry for establishing a communication link with the hearing aid; speaker; and The processor is configured to execute the user control interface by enabling measurements of the corresponding head-related transfer functions in the far-field calibration operation mode; The assistive device is configured to play a test sound signal and transmit an electrical version of the test sound signal to the hearing aid based on input from the user control interface; and The auxiliary device is configured to notify the start time of the head-related transfer function measurement.

12. The auxiliary device according to claim 11, configured to generate a calibration control signal when far-field calibration is initiated from the user control interface.

13. The auxiliary device according to claim 12, comprising a carrier for supporting the auxiliary device.

14. A non-transitory computer-readable medium storing an application program including instructions configured to execute on an assistive device to implement a user interface for a hearing aid according to claim 1, wherein the user interface is configured to enable a user to control the functions of the hearing aid, including activating a calibration operation mode of the hearing aid.

15. The non-transitory computer-readable medium of claim 14, wherein the application is configured to run on a mobile phone or another portable device to enable communication with the hearing aid.

16. A method of operating a hearing aid suitable for a user to wear on or in their ear, wherein the hearing aid is adapted to establish a communication link with an assistive device configured as a user control interface via an antenna and transceiver circuitry, such that data can be exchanged between them or forwarded from one device to another; the method comprising: At least one electrical input signal is provided by at least one microphone, which picks up the test sound signal from the loudspeaker of the auxiliary device; Processes at least one electrical input signal or a signal derived from at least one electrical input signal and provides a processed signal; The electrical version of the test sound signal is received from the transmission of the auxiliary device via an antenna and transceiver circuit. Presenting the processed signal, which can be perceived as sound, to the user; At the time when the assistive device notifies the hearing aid, based on the test sound signal picked up by the at least one microphone and the electrical version of the test sound signal received by the antenna and transceiver circuit, the measurement of the corresponding head-related transfer function is initiated in the far-field calibration operation mode.

Images