Improving hearing loss simulation via neural networks

Abstract

The invention discloses improved hearing loss simulation via a neural network. A corresponding method of determining and setting signal processing of a hearing aid comprises: providing at least one electrical input signal representing at least one input sound signal from a sound environment of a hearing aid user; determining a normal hearing representation of the at least one electrical input signal on the basis of the selected normal hearing hearing model fj; determining optimized training parameters of a neural network, wherein the neural network represents a hearing impairment representation of the at least one electrical input signal based on a hearing impairment auditory model; wherein determining the optimized training parameters comprises training a hearing impaired hearing model based on the provided at least one electrical input signal, minimizing a difference between a normal hearing representation and a hearing impaired representation, comprising the following steps: determining frequency distribution beta j and level and frequency distribution alpha j, l of at least one electric input signal based on the balance of the sound pressure level of the at least one electric input signal; and determining signal processing parameters based on the optimized training parameters.

Description

Technical Field

[0001] The present application relates to a method of determining and setting signal processing of a hearing aid configured to be worn at or in an ear by a user.

[0002] The application also relates to a hearing aid adapted to be worn in or at the ear of a user.

[0003] The application also relates to a hearing system comprising a left and a right hearing aid.

[0004] The present application also relates to a computer program product comprising a computer program containing instructions which, when executed by a computer, cause the computer to perform the method of the present invention. Background Art

[0005] Some modern hearing aids use neural networks within the hearing aid to perform some signal processing. As an example, a deep neural network may be implemented to perform part of the noise reduction. Currently, such neural networks are fixed. In other words, the same neural network is given to each hearing aid user, and the same neural network is used in all acoustic situations.

[0006] However, such neural networks will perform better if they are adapted to the specific hearing aid user and / or acoustic situation. In other words, ideally, different neural networks can be used for different hearing aid users or acoustic situations, e.g., as indicated by user data (e.g., audiograms), behavioral data, user preferences, etc.

[0007] Training a neural network to simulate an auditory model is a conventional nonlinear recursive problem. However, when a deep neural network simulator is trained to simulate an auditory model with a data set spanning a large dynamic input range (which is required for hearing loss compensation), the simulator does not perform well as a function of the input sound level and the channel of the auditory model. This means that the resulting hearing loss compensation and noise reduction strategies will not perform as expected, or will only perform as expected at a narrow set of sound input levels and frequencies, and will begin to perform erratically or unpredictably once the auditory model simulator performs poorly. Summary of the invention

[0008] method

[0009] The present application provides a new optimization scheme that greatly increases performance across sound input levels and auditory model channels without increasing the computational burden at inference, while still using the same known model architecture.

[0010] In one aspect of the present application, a method of determining and setting signal processing for a hearing aid is provided.

[0011] The hearing aid may be configured to be worn at or in the ear by a user.

[0012] The method of the invention may comprise providing at least one electrical input signal representing at least one input sound signal from the sound environment of a hearing aid user.

[0013] The method of the present invention may include selecting a normal hearing auditory model f j A normal hearing representation of at least one electrical input signal is determined.

[0014] “Selection” may mean that a person performing (at least part of) the method of determining and setting the signal processing of a hearing aid (e.g. fitting of a hearing aid and / or training of a neural network of a hearing aid) may select a specific auditory model to be applied in the method of the present invention.

[0015] Selecting a specific auditory model to be applied in the method of the present invention may refer to selecting / choosing an auditory model suitable for the specific mode to be applied by the hearing aid.

[0016] The method of the invention may comprise determining optimized training parameters of a neural network. The neural network may represent (may be) a hearing-impaired representation of at least one electrical input signal based on an auditory model of hearing impairment.

[0017] Determining the optimized training parameters may include determining a frequency distribution β of at least one electrical input signal j .

[0018] Determining the optimized training parameters may include determining the level and frequency distribution α of at least one electrical input signal. j,l .

[0019] Determine the frequency distribution β of at least one electrical input signal j and level and frequency distribution α j,l The equalization may be based on a sound pressure level of at least one electrical input signal.

[0020] Thereby, an optimization scheme is provided with increased performance across sound input levels and auditory model channels.

[0021] Determining the optimized training parameters may include training the hearing-impaired auditory model based on the provided at least one electrical input signal to minimize the difference between the normal hearing representation and the hearing-impaired representation, including determining the frequency distribution β of the at least one electrical input signal based on equalization of the sound pressure level of the at least one electrical input signal. j and level and frequency distribution α j,l .

[0022] The method of the present invention may further include determining signal processing parameters based on the optimized training parameters.

[0023] The equalization of the sound pressure levels has a role in determining the optimized training parameters with relatively low sound levels being counted more and relatively high sound levels being counted less. Thus, the sound levels will be considered more equally during the determination of the optimized training parameters, rather than relatively high sound levels being counted the most.

[0024] Frequency distribution β j A normal hearing auditory model f may be relied upon for determining a normal hearing representation of at least one electrical input signal j .

[0025] Thus, the frequency distribution β j The frequency distribution β j The auditory model f j change.

[0026] Frequency distribution β j It can be determined by the following formula:

[0027]

[0028] Where L is the maximum sound pressure level of at least one electrical input signal, l is the input sound pressure level, j is the frequency, J is the number of channels, X defines the electrical input signal space, x is the electrical input signal, and θ is the normal hearing auditory model f j , where l≤L and j≤J.

[0029] Level and frequency distribution α j,l It can be determined by the following formula:

[0030]

[0031] Where l is the input sound pressure level, j is the frequency, J is the number of channels, X defines the electrical input signal space, x is the electrical input signal, and θ is the normal hearing auditory model f j A free parameter of , where j≤J.

[0032] The hearing impaired auditory model may be selected based on a predetermined audiogram of the user.

[0033] The hearing-impaired auditory model can be selected based on the user's electroencephalogram (EEG).

[0034] The hearing-impaired auditory model may be selected based on the user's distortion product otoacoustic emission (DPOAE) response.

[0035] For example, the audiogram of a hearing aid user may be determined before the signal processing of the hearing aid is determined and set (ie at an early stage in time), for example during fitting of the hearing aid.

[0036] Thus, the auditory model most suitable for the user can be selected in consideration of the user's audiogram.

[0037] The step of determining a normal hearing representation of at least one electrical input signal based on the selected normal hearing auditory model may comprise selecting the normal hearing auditory model from a plurality of normal hearing auditory models. j .

[0038] Multiple can refer to more than two auditory models.

[0039] Each normal hearing auditory model f selected from a plurality of normal hearing auditory models j May depend on the specific type of sound environment of the hearing aid user.

[0040] The sound environment may refer to a noisy sound environment such as a cocktail party or a factory, or may refer to a low-noise sound environment.

[0041] Each normal hearing auditory model f selected from a plurality of normal hearing auditory models j May depend on the specific type of hearing pattern of the hearing aid user.

[0042] A listening mode may refer to a listening mode of a hearing aid (eg, selected / chosen by a user), such as a music mode.

[0043] Optimized training parameters of the neural network may be determined (trained) based on at least one electrical input signal representing a specific sound environment.

[0044] The present application provides a method for training a neural network, such as a deep neural network (DNN), to simulate an auditory model. This can be done by training the model across a large dataset of acoustic signals to minimize a certain optimization objective that minimizes the true auditory model representation f(x) and the estimated representation Minimize the difference between them.

[0045] Thus, the method of the present invention also comprises minimizing the optimization target by minimizing the difference between the normal hearing representation and the hearing impaired representation.

[0046] For example, first, a general notation can be defined that can be used for both auditory models.

[0047] Defining the signal space and internal representation space In this specification, T is the length of the input signal, J is the number of channels with a given center frequency (CF), and T * is the time dimension in the internal representation space. Usually, T = T * , but in some cases, the internal representation may not be downsampled or upsampled, which means that T≠T * Therefore, the internal representation can be viewed as a spectrogram-like representation of the input signal. We denote the auditory model as the function f θ:X→I, where θ is a free parameter of the model. For ease of reading, we will suppress θ notation except where it is needed for proper understanding. To emphasize the single CF of the auditory model, we use f j The jth channel for the internal representation. In the following, we will Used as an estimate of f(x).

[0048] Traditionally, deep neural network (DNN) auditory model simulators have been trained using mean absolute error (MAE) or mean squared error (MSE):

[0049]

[0050] In this application, the full data set T is defined as the input-output pairs x and f θ (x) Composition:

[0051] T:={(x,f θ (x))|x∈X} (3)

[0052] At a given input level SPL of x, the subset of T is:

[0053] T l :={(x,f θ (x))∈T|p 0 10 l / 20} (4)

[0054] Where L is the maximum level, i.e. l≤L. Therefore, T is all T l The union (combination) of.

[0055]

[0056] because Estimation of β j f j (x), which follows Estimate f j (x). Therefore, we write to follow the previous notation.

[0057] Depending on the linearity you want to capture, you can use f in Equation 3-8. j (x) Remove the mean and give only the variance of the signal importance in the weighting. This is important for auditory models that use DC offsets (IHC DC potentials, auditory nerve spontaneous firing, etc.)

[0058] Equation 6-8 gives α l,j and β j It should be noted that “j” refers to the frequency and “l” refers to the input level dependency.

[0059] In addition, since the l The calculated levels are discrete and can be interpolated between two levels in the log domain, which helps stabilize the training of DNNs.

[0060] To measure the performance of the model, a new metric Signal to Error Ratio (SER) can be introduced, which is similar to the signal to noise ratio. This metric enables the measurement of relative performance and can be calculated for each CF at a set of input levels.

[0061] Thus, the method of the present invention may include determining the signal error ratio (SER) by the following formula:

[0062]

[0063] Hearing aids

[0064] In one aspect, a hearing aid adapted to be worn in or at an ear of a user is also provided.

[0065] The hearing aid may comprise an input unit for receiving input sound signals from the environment of the hearing aid user.

[0066] The input unit may be configured to provide at least one electrical input signal representing the input sound signal.

[0067] The hearing aid may comprise an output unit for providing at least one set of stimuli perceivable as sounds to a user based on a processed version of at least one electrical input signal.

[0068] The hearing aid may comprise a selector configured to select one of a plurality of sound environment modes.

[0069] Hearing aids may include neural networks.

[0070] Each mode may represent optimized training parameters of the hearing aid's neural network determined according to the selected particular type of sound environment of the hearing aid user.

[0071] Each pattern may represent optimized training parameters of the hearing aid's neural network determined according to a particular type of listening pattern selected by the hearing aid user.

[0072] For example, each mode may represent signal processing parameters of a hearing aid, wherein the signal processing parameters may be determined (by the method described above) from optimized training parameters. For example, for each mode, the optimized training parameters have been trained based on an electrical input signal that identifies / represents a selected specific type of sound environment for a hearing aid user. There may be more than two modes.

[0073] The hearing aid may comprise a processing unit connected to an input unit and an output unit.

[0074] The hearing aid may comprise signal processing parameters of the hearing aid to provide a processed version of at least one electrical input signal.

[0075] The signal processing parameters may be determined based on the optimized training parameters determined according to the method described above.

[0076] The processing unit may include a deep neural network that provides optimized training parameters.

[0077] Deep neural networks can be trained according to the methods described above.

[0078] The input unit may further comprise an input transducer such as a microphone for converting input sound into an electrical input signal. The input unit may comprise a wireless receiver for receiving a wireless signal comprising or representing a sound and providing an electrical input signal representing said sound.

[0079] The output unit may further comprise an output transducer. The output transducer may comprise a receiver (speaker) for providing the stimulus as an acoustic signal to the user (eg in an acoustic (air conduction based) hearing aid).

[0080] The output unit may (additionally or alternatively) comprise a transmitter for transmitting the sound picked up by the hearing aid to another device, such as a remote communication partner (eg via a network, such as in telephone operating mode, or in a headset configuration).

[0081] The hearing aid may be adapted to provide frequency dependent gain and / or level dependent compression and / or frequency transposition of one or more frequency ranges to one or more other frequency ranges (with or without frequency compression), for example to compensate for hearing impairment of a user.

[0082] Thereby, the processing unit provides a processed version of at least one electrical input signal.

[0083] The hearing aid may include a memory storing weights of a neural network of the hearing aid.

[0084] The hearing aid may include an antenna and transceiver circuitry for establishing a communication link to an auxiliary device.

[0085] The communication link may be a wireless link to an entertainment device (such as a television), a communication device (such as a telephone), a wireless microphone, another hearing aid, a server device (such as a cloud server), or a processor unit, etc. The hearing aid may thus be configured to wirelessly receive a direct electrical input signal from another device. Similarly, the hearing aid may be configured to wirelessly transmit a direct electrical output signal to another device. The direct electrical input or output signal may represent or include an audio signal and / or a control signal and / or an information signal.

[0086] In general, the wireless link established by the antenna and the transceiver circuit of the hearing aid may be of any type. The wireless link may be a link based on near field communication, such as an inductive link based on inductive coupling between antenna coils of a transmitter part and a receiver part. The wireless link may be based on far field electromagnetic radiation. Preferably, the frequency used to establish the communication link between the hearing aid and the other device is lower than 70 GHz, such as in the range from 50 MHz to 70 GHz, such as higher than 300 MHz, such as in the ISM range higher than 300 MHz, such as in the 900 MHz range or in the 2.4 GHz range or in the 5.8 GHz range or in the 60 GHz range (ISM = Industrial, Scientific and Medical, such standardized ranges are defined, for example, by the International Telecommunication Union ITU). The wireless link may be based on standardized or proprietary technologies. The wireless link may be based on Bluetooth technology (such as Bluetooth low energy technology) or ultra-wideband (UWB) technology.

[0087] The hearing aid, such as the input unit and / or the antenna and the transceiver circuitry, may comprise a transform unit for transforming a time domain signal into a signal in a transform domain (e.g., frequency domain or Laplace domain, etc.). The transform unit may be constituted by or include a time-frequency (TF) conversion unit for providing a time-frequency representation of the input signal. The time-frequency representation may include an array or mapping of corresponding complex or real values ​​of the signal concerned in a specific time and frequency range. The TF conversion unit may include a filter bank for filtering the (time-varying) input signal and providing a plurality of (time-varying) output signals, each output signal comprising a distinct frequency range of the input signal. The TF conversion unit may include a Fourier transform unit (e.g., a discrete Fourier transform (DFT) algorithm, a short-time Fourier transform (STFT) algorithm, or a similar algorithm) for transforming the time-varying input signal into a (time-varying) signal in the (time-)frequency domain. The minimum frequency f is taken into account by the hearing aid. min To the maximum frequency f max The frequency range of may include a portion of the typical human hearing range from 20 Hz to 20 kHz, for example a portion of the range from 20 Hz to 12 kHz. Typically, the sampling rate f s Greater than or equal to the maximum frequency f max twice, that is, f s ≥2f max The signals of the forward path and / or analysis path of the hearing aid may be split into NI frequency bands (e.g. of uniform width), wherein NI is, for example, greater than 5, such as greater than 10, such as greater than 50, such as greater than 100, such as greater than 500, at least parts of which are processed separately. The hearing aid may be adapted to process the signals of the forward and / or analysis path in NP different frequency channels (NP≤NI). The frequency channels may be of uniform or non-uniform width (e.g. the width increases with frequency), overlapping or non-overlapping.

[0088] The weights of the neural network can be adaptively adjustable weights.

[0089] Adaptively adjustable weights may refer to weights or parameters of a neural network that may be updated / adjusted / corrected one or more times.

[0090] Adaptively adjustable weights may refer to weights or parameters and bias units of a neural network that may be updated / adjusted / corrected one or more times.

[0091] The hearing aid may further comprise a signal-to-noise ratio (SNR) estimator configured to determine the SNR in an environment of the hearing aid user.

[0092] The hearing aid may further comprise a sound pressure level (SPL) estimator for measuring the level of the sound at the input unit.

[0093] The SPL estimator or SPL detector may estimate the current level of the signal of the forward path of the hearing aid. The SPL estimator or SPL detector may be configured to determine whether the current level of the signal of the forward path is above or below a given (L-)threshold. The SPL estimator or SPL detector may operate on a full-band signal (time domain). The level detector may operate on a band-split signal ((time-)frequency domain).

[0094] The hearing aid may comprise at least one accelerometer.

[0095] The accelerometer may be configured to detect movement of the hearing aid user's facial muscles and / or bones, e.g. due to talking or chewing (e.g. jaw movement), or movement / rotation of the hearing aid user's face, e.g. longitudinally and / or laterally, and provide a detection signal indicative of the detection result.

[0096] The accelerometer may be configured to detect jaw movement.The hearing aid may be configured to apply jaw movement as an additional cue for self-voice detection.

[0097] For example, a movement may be detected when the hearing aid user nods, eg as an indication that the hearing aid user is following and is interested in the voice signal / speech of the conversation partner / speaking partner.

[0098] The accelerometer may be configured to detect movement of the hearing aid user after speech onset (e.g., as determined by a voice detector (VD), a voice activity detector (VAD), and / or an own voice detector (OVD)). For example, movement of the head, for example, after speech onset may be an attention cue indicating a sound source of interest.

[0099] The hearing aid may comprise a voice activity detector (VAD). The VAD may be configured to estimate whether (or with what probability) an input signal (at a given point in time) comprises a voice signal. In the present specification, a voice signal may be meant to comprise a speech signal from a human being. It may also comprise other forms of vocalizations (such as singing) produced by a human speech system. The voice activity detector unit may be adapted to classify the user's current acoustic environment as a "voice" or "no-voice" environment. This has the following advantage: time periods comprising an electric microphone signal of human vocalizations (such as speech) in the user's environment may be identified and thus separated from time periods comprising only (or mainly) other sound sources (such as artificially generated noise). The voice activity detector may be adapted to detect the user's own voice as "voice" as well. As an alternative, the voice activity detector may be adapted to exclude the user's own voice from the detection of "voice".

[0100] The hearing aid may also include one or more different types of physiological sensors for providing physiological parameters, wherein the one or more different types of physiological sensors are configured to measure one or more physiological signals of the user, such as an electrocardiogram (ECG), a photoplethysmogram (PPG), an electroencephalogram (EEG), an electrooculogram (EOG), etc.

[0101] Electrodes of one or more different types of physiological sensors may be disposed at an outer surface of the hearing aid. For example, the electrodes may be disposed at an outer surface of a behind-the-ear (BTE) portion and / or an inside-the-ear (ITE) portion of the hearing aid. Thus, when the user wears the hearing aid, the electrodes are in contact with the user's skin (either behind the ear or in the ear canal).

[0102] The hearing aid may include multiple (eg more than two) detectors / sensors and / or estimators, which may work in parallel. For example, more than two physiological sensors may work simultaneously to increase the reliability of the measured physiological signals.

[0103] The hearing aid may comprise a sound scene classifier configured to classify the acoustic environment of the hearing aid user into a plurality of different sound scene categories.

[0104] The hearing aid may comprise a sound scene classifier configured to provide a current sound scene category based on a current representation of the at least one electrical input signal, such as an extracted feature.

[0105] The sound scene classifier may be configured to classify the current situation based on input signals from (at least part of) the detector / sensor / estimator / accelerometer and possibly other inputs. In this specification, the "current situation" may be defined by one or more of the following:

[0106] a) the physical environment (e.g. including the current electromagnetic environment, such as the presence of electromagnetic signals (including audio and / or control signals) intended or not intended to be received by the hearing aid, or other properties of the current environment other than acoustics);

[0107] b) Current acoustic conditions (input level, feedback, etc.);

[0108] c) the user’s current mode or state (motion, temperature, cognitive load, etc.);

[0109] d) The current mode or status of the hearing aid and / or another device communicating with the hearing aid (selected program, time elapsed since last user interaction, etc.).

[0110] Hearing System

[0111] In another aspect, a hearing system is provided comprising left and right hearing aids as described above.

[0112] The left and right hearing aids may be configured to be worn in or at the left and right ears of a user, respectively, and configured to establish a wired or wireless connection therebetween, thereby enabling data such as audio data to be exchanged therebetween, optionally via an intermediate device.

[0113] In another aspect, there is provided a hearing system comprising a hearing aid as described above and comprising an auxiliary device.

[0114] Each of the hearing aid and the auxiliary device may include an antenna and transceiver circuitry for establishing a communication link therebetween, thereby enabling information to be exchanged between the hearing aid and the auxiliary device.

[0115] For example, the auxiliary device may include another hearing aid, a remote control, an audio transmission device, a phone (such as a smart phone), an external sensor, a server device, a cloud server, etc.

[0116] The auxiliary device may consist of or comprise a remote control for controlling the functions and operation of the hearing aid. The functions of the remote control are implemented in a smartphone, which may run an APP enabling control of the functions of the audio processing device via the smartphone (the hearing aid comprises a suitable wireless interface to the smartphone, e.g. based on Bluetooth or some other standardized or proprietary solution).

[0117] The auxiliary device may consist of or include an audio gateway device, which is suitable for receiving a plurality of audio signals (e.g. from an entertainment device such as a TV or a music player, from a telephone device such as a mobile phone or from a computer such as a PC) and for selecting and / or combining appropriate signals (or signal combinations) from the received audio signals for transmission to the hearing aid.

[0118] When appropriately replaced by corresponding procedures, some or all structural features of the hearing aid and hearing system described above, described in detail in the "Detailed Description of the Invention" or defined in the claims may be combined with the implementation of the method, and vice versa. The implementation of the method has the same advantages as the corresponding hearing aid and hearing system.

[0119] application

[0120] On the one hand, there is provided an application of a hearing aid as described above, in detail in the "Detailed Description" section and in the claims. The application may be provided in a hearing system including one or more hearing aids (such as hearing instruments), headphones, headsets, active ear protection systems, etc., such as a hands-free telephone system, a teleconferencing system (such as including a loudspeaker amplifier), a broadcasting system, a karaoke system, a classroom amplification system, etc.

[0121] Computer readable medium or data carrier

[0122] The present invention further provides a tangible computer-readable medium (data carrier) storing a computer program including program code (instructions), which, when executed on a data processing system (computer), enables the data processing system to execute (implement) at least part (such as most or all) of the steps of the method described above, described in detail in the "Specific Implementation Method" and defined in the claims.

[0123] Computer Programs

[0124] In addition, the present application provides a computer program (product) comprising instructions, which, when executed by a computer, causes the computer to execute (the steps of) the method described above, described in detail in the “Detailed Description” and defined in the claims.

[0125] Data processing system

[0126] On the one hand, the present invention further provides a data processing system comprising a processor and a program code, wherein the program code enables the processor to perform at least part (such as most or all) of the steps of the method described above, described in detail in the "Specific Implementation Method" and defined in the claims.

[0127] APP

[0128] On the other hand, the present invention also provides a non-transient application called APP. The APP includes executable instructions configured to run on an auxiliary device to implement a user interface for the hearing aid or hearing system described above, described in detail in the "Detailed Description of the Invention" and defined in the claims. The APP can be configured to run on a mobile phone such as a smart phone or another portable device that enables communication with the hearing aid or hearing system.

[0129] definition

[0130] In this specification, a hearing aid, such as a hearing instrument, refers to a device suitable for improving, enhancing and / or protecting the hearing ability of a user by receiving an acoustic signal from the user's environment, generating a corresponding audio signal, possibly modifying the audio signal, and providing the possibly modified audio signal as an audible signal to at least one ear of the user. The audible signal may be provided, for example, in the form of an acoustic signal radiated into the user's outer ear, an acoustic signal transmitted to the user's inner ear as mechanical vibrations through the bone structure of the user's head and / or through parts of the middle ear, and an electrical signal transmitted directly or indirectly to the user's cochlear nerve.

[0131] The hearing aid may be configured to be worn in any known manner, such as as a unit worn behind the ear (with a tube directing the radiated acoustic signal into the ear canal or with an output transducer such as a loudspeaker arranged close to or in the ear canal), as a unit arranged wholly or partly in the auricle and / or ear canal, as a unit connected to a fixed structure implanted in the skull, such as a vibrator, or as a connectable or wholly or partly implanted unit, etc. The hearing aid may comprise a single unit or several units that communicate with each other (e.g. acoustically, electrically or optically). The loudspeaker may be arranged in a housing together with the other components of the hearing aid, or it may itself be an external unit (possibly in combination with a flexible guiding element such as a dome-shaped element).

[0132] The hearing aid may be adapted to the needs of a specific user, such as hearing loss. The configurable signal processing circuit of the hearing aid may be adapted to apply frequency- and level-dependent compression amplification of the input signal. The customized frequency- and level-dependent gain (amplification or compression) may be determined during the fitting process by the fitting system based on the user's hearing data, such as an audiogram, using the basic principles of fitting (e.g. adaptation to speech). The frequency- and level-dependent gain may, for example, be embodied in a processing parameter, uploaded to the hearing aid, for example, via an interface to a programming device (fitting system), and used by a processing algorithm executed by the configurable signal processing circuit of the hearing aid.

[0133] "Hearing system" refers to a system including one or two hearing aids. "Binaural hearing system" refers to a system including two hearing aids and adapted to provide audible signals to the two ears of a user in a coordinated manner. A hearing system or binaural hearing system may also include one or more "auxiliary devices" that communicate with the hearing aids and affect and / or benefit from the functions of the hearing aids. The aforementioned auxiliary devices may include at least one of the following: a remote controller, a remote microphone, an audio gateway device, an entertainment device such as a music player, a wireless communication device such as a mobile phone (e.g., a smart phone) or a tablet computer or another device, for example, including a graphical interface. Hearing aids, hearing systems or binaural hearing systems may be used, for example, to compensate for the loss of hearing ability of hearing-impaired persons, enhance or protect the hearing ability of normal hearing persons, and / or transmit electronic audio signals to people. Hearing aids or hearing systems may, for example, form part of or interact with a broadcasting system, an active ear protection system, a hands-free telephone system, a car audio system, an entertainment (e.g., TV, music playback or karaoke) system, a teleconferencing system, a classroom amplification system, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0134] Various aspects of the present invention will be best understood from the detailed description below in conjunction with the accompanying drawings. For clarity, the drawings are schematic and simplified, and only the details necessary for understanding the present invention are given, while other details are omitted. Throughout the specification, the same reference numerals are used for the same or corresponding parts. The various features of each aspect may be combined with any or all features of the other aspects. These and other aspects, features and / or technical effects will be apparent from and illustrated in conjunction with the following figures, in which:

[0135] Figure 1 An exemplary hearing system according to the present application is shown;

[0136] Figure 2 An exemplary overview of the framework is shown;

[0137] Figure 3 It shows that α l,j and β j An exemplary diagram of

[0138] Figure 4 An exemplary speech error metric for the Verhulst auditory model under 3 different conditions is shown;

[0139] Figure 5 Example error metrics for 4 different audiograms are shown.

[0140] By the detailed description given below, the further scope of application of the present invention will be apparent. However, it should be understood that while the detailed description and specific examples show the preferred embodiments of the present invention, they are only provided for illustrative purposes. For those skilled in the art, based on the following detailed description, other embodiments of the present invention will be apparent. DETAILED DESCRIPTION

[0141] The detailed description proposed below in conjunction with the accompanying drawings serves as a description of a variety of different configurations. The detailed description includes specific details for providing a thorough understanding of a number of different concepts. However, it is apparent to those skilled in the art that these concepts can be implemented without these specific details. Several aspects of the apparatus and method are described by a number of different blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as "elements"). Depending on the specific application, design limitations or other reasons, these elements can be implemented using electronic hardware, computer programs or any combination thereof.

[0142] Figure 1 An exemplary hearing system according to the present application is shown.

[0143] exist Figure 1 In FIG. 1 , a hearing aid 1 and an auxiliary device 2 are shown. The hearing aid 1 and the auxiliary device 2 may together form a hearing system.

[0144] The hearing aid 1 may be adapted to be worn in or at the ear of a hearing aid user.

[0145] The auxiliary device 2 may comprise another hearing aid located at the other ear of the hearing aid user. Alternatively, the auxiliary device 2 may comprise a smartphone or a server device.

[0146] The hearing aid 1 may comprise an input unit 3 for receiving an input sound signal 4 from the acoustic environment of a hearing aid user and providing at least one electrical input signal 5A, 5B representing the input sound signal.

[0147] exist Figure 1 In FIG. 4 , it is shown that the input unit 3 may further include two or more input converters 6A, 6B, such as microphones, for converting the input sound signal 4 into at least one electrical input signal 5A, 5B.

[0148] The hearing aid may comprise an output unit 7 for providing at least one set of stimuli 7A perceptible as sounds to a user of the hearing aid based on a processed version of the at least one electrical input signal 5A, 5B.

[0149] The hearing aid 1 may comprise a processing unit 8 connected to an input unit 3 and an output unit 7 .

[0150] The processing unit 8 may comprise a neural network 9 and the processing unit 8 may be configured to determine signal processing parameters of the hearing aid 1 based on weights of the neural network.

[0151] Thereby, the processing unit 8 can provide a processed version of the at least one electrical input signal 5A, 5A.

[0152] The hearing aid 1 may comprise a memory 10 storing weights of a neural network 9 of the hearing aid 1. Thus, the memory 10 may send and receive currently used weights and / or reference weights.

[0153] The hearing aid 1 may comprise an antenna and transceiver circuitry 11 for establishing a communication link to the auxiliary device 2 .

[0154] The hearing aid 1 may further comprise a selector 14 configured to select one of a plurality of sound environment patterns, wherein each pattern may represent optimized training parameters of the neural network 9 of the hearing aid 1 determined according to the selected particular type of sound environment or the hearing pattern of the hearing aid user.

[0155] The hearing aid 1 may further comprise a sound scene classifier 12 configured to classify the acoustic environment of the hearing aid user into a plurality of different sound scene categories.

[0156] The hearing aid 1 may further comprise a detector / sensor / estimator 13, such as an SNR estimator, an SPL estimator, at least one physiological sensor, and / or at least one accelerometer.

[0157] The auxiliary device 2 may comprise an antenna and transceiver circuitry (not shown) for establishing a communication link between the hearing aid 1 and the auxiliary device 2 , thereby enabling information to be exchanged between the hearing aid 1 and the auxiliary device 2 .

[0158] Figure 2 An exemplary overview of the framework is shown.

[0159] exist Figure 2 In , the goal is to train a deep neural network DNN to simulate an auditory model AM. This can be achieved by training the auditory model AM across a large dataset of acoustic signals AS and then minimizing some optimization objective OptO that minimizes the true auditory model representation f(x) with an estimated representation of the hearing-impaired representation Minimize the difference between .

[0160] The bold line indicates the signal pathway, the dashed line indicates the model parameter MP, and the dotted line indicates the back propagation ΔOO of the optimization objective OptO, which is used to train the deep neural network DNN. x is the electrical input signal.

[0161] Figure 3 It shows that α l,j and β j An example diagram of .

[0162] As noted above, Equations 6-8 provide α l,j and β j It should be noted that “j” refers to the frequency and “l” refers to the input level dependency.

[0163] exist Figure 3 , illustrates how these variables would look for N3 hearing loss (moderate, oblique) from the Zilany model [2], where l∈{40,70,100,120}. Note that the j dimension (x-axis) is converted to the CF of the jth channel.

[0164] Figure 4 Exemplary speech error metrics for the Verhulst auditory model under 3 different conditions are shown.

[0165] DNNs can be trained with the CoNNEar architecture to emulate the Verhulst auditory model [1], and with Wave-U-Net to emulate the Zilany auditory model [2].

[0166] For example, 2500 sentences from the LibriTTS dataset can be used to generate input-output pairs for each parameter configuration. The Verhulst auditory model [1] can use the following configurations: Normal, Slope20_5, and Flat20. "Normal" refers to normal hearing, Slope20_5 refers to mild, sloping hearing loss, and Flat20 refers to mild, flat hearing loss. The Zilany auditory model [2] can use the following configurations: Normal, N3, N5, and S1. "Normal" refers to normal hearing, N3 refers to moderate, sloping hearing loss, N5 refers to severe, sloping hearing loss, and S1 refers to profound sloping hearing loss.

[0167] Performance can be measured in three ways. For each auditory model, two neural networks can be trained, one using MAE as the optimization objective and the other using a variation of FPAE in Equation 5, called FMAE, which means we set P = 1, i.e., use the 1 norm in Equations 3-8. These two neural networks can be denoted as DNN-MAE and DNN-FMAE, respectively.

[0168]

[0169] Afterwards, the DNN and the reference model are used to generate 20 sentences of output, i.e., the ground truth y(x), at each of the 5 different input sound levels (40, 60, 80, 100, 120) dB. Then, the normal MAE (the MAE in the entire training set is called the global error (GE)), SER, and ΔSER, which is defined as the SER difference between DNN-FMAE and DNN-MAE, can be calculated at each level. This is Figure 4-5 Shown in.

[0170] It can be seen that there is a significant increase at low sound input levels and high frequency components, even though the overall error of DNN-FMAE tends to be higher (meaning worse performance) compared to DNN-MAE.

[0171] thus, Figure 4 Speech error metrics for the Verhulst auditory model [1] under three different conditions are shown: N (normal hearing), Slope20_5 (S), Flat20 (F). DNN-MAE refers to the model trained with MAE as the optimization objective, and DNN-FMAE refers to the model trained with FMAE as the optimization objective. The first column shows the log(MAE) of DNN-MAE and DNN-FMAE across different levels, and shows the overall MAE (GE): the average MAE across all levels. The second column shows the SER error of DNN-MAE and DNN-FMAE across levels. The third column shows the SER difference between DNN-MAE and DNN-FMAE across different levels, and shows the average increase in SER. Note that the scales in the charts are different.

[0172] Figure 5 Example error metrics for 4 different audiograms are shown.

[0173] Figure 5 Error metrics are shown for 4 different audiograms measured across 10 sentences not seen during training: N0 (normal hearing) and 3 types of hearing loss. DNN-MAE refers to the model trained with MAE as the optimization objective, and DNN-FMAE refers to the model trained with FMAE as the optimization objective. The first column shows the log(MAE) of DNN-MAE and DNN-FMAE across different levels, and the overall MAE (GE) is shown: the average MAE across all levels. The second column shows the SER error of DNN-MAE and DNN-FMAE across levels. The third column shows the SER difference between DNN-MAE and DNN-FMAE across different levels, and the average increase in SER is shown. Note that the scales in the charts are different.

[0174] The structural features of the apparatus described above, described in detail in the “Detailed Description of the Invention” and defined in the claims may be combined with the steps of the method of the present invention when appropriately replaced by corresponding processes.

[0175] Unless expressly stated, the singular forms "one", "the" used herein include the plural form (i.e., have the meaning of "at least one"). It should be further understood that the terms "having", "including" and / or "comprising" used in the specification indicate the presence of the described features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or combinations thereof. It should be understood that, unless expressly stated, when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate intervening element. As used herein, the term "and / or" includes any and all combinations of one or more listed related items. Unless expressly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed.

[0176] It should be appreciated that reference to "an embodiment" or "embodiment" or "aspect" or features that "may" include in this specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. In addition, the specific features, structures or characteristics may be appropriately combined in one or more embodiments of the present invention. The foregoing description is provided to enable those skilled in the art to implement the various aspects described herein. Various modifications will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects.

[0177] The claims are not limited to the various aspects shown herein, but rather have the full scope consistent with the claim language wherein, unless expressly stated otherwise, elements referred to in the singular do not mean "one and only one" but rather "one or more." Unless expressly stated otherwise, the term "some" means one or more.

[0178] References

[0179] [1]Baby, D., Van Den Broucke, A., & Verhulst, S. (2021). A convolutional neural-network model of human cochlear mechanics and filter tuning for real-time applications. Nature machine intelligence, 3(2), 134-143.

[0180] [2]Zilany,M.S.,Bruce,I.C.,&Carney,L.H.(2014).Updated parameters andexpanded simulation options for a model of the auditory periphery.The Journalof the Acoustical Society of America,135(1),283–286.

Claims

1. A method of determining and setting signal processing for a hearing aid, the hearing aid being configured to be worn at or in an ear by a user, the method comprising: providing at least one electrical input signal representing at least one input sound signal from the sound environment of a hearing aid user; Based on the selected normal hearing auditory model f j determining a normal hearing representation of at least one electrical input signal; Determining optimized training parameters of a neural network, wherein the neural network represents a hearing-impaired representation of at least one electrical input signal based on a hearing-impaired auditory model; wherein determining the optimized training parameters comprises training the hearing-impaired auditory model based on the provided at least one electrical input signal to minimize a difference between the normal hearing representation and the hearing-impaired representation, comprising determining a frequency distribution β of the at least one electrical input signal based on an equalization of a sound pressure level of the at least one electrical input signal j and level and frequency distribution α j,l ; Signal processing parameters are determined based on the optimized training parameters.

2. The method according to claim 1, wherein: The frequency distribution β j relying on the normal hearing auditory model f for determining the normal hearing representation of the at least one electrical input signal j .

3. A method according to any preceding claim, wherein: The frequency distribution β j Determined by the following formula: Where L is the maximum sound pressure level of at least one electrical input signal, l is the input sound pressure level, j is the frequency, J is the number of channels, X defines the electrical input signal space, x is the electrical input signal, and θ is the normal hearing auditory model f j , where l≤L and j≤J.

4. A method according to any preceding claim, wherein: The level and frequency distribution α j,l Determined by the following formula: Where l is the input sound pressure level, j is the frequency, J is the number of channels, X defines the electrical input signal space, x is the electrical input signal, and θ is the normal hearing auditory model f j A free parameter of , where j≤J.

5. A method according to any preceding claim, wherein: The hearing-impaired auditory model is selected based on a user's predetermined audiogram, electroencephalogram, or distortion product otoacoustic emission responses.

6. A method according to any preceding claim, wherein: The step of determining a normal hearing representation of at least one electrical input signal based on the selected normal hearing auditory model comprises selecting the normal hearing auditory model from a plurality of normal hearing auditory models. j .

7. The method according to claim 6, wherein: Each normal hearing auditory model f selected from a plurality of normal hearing auditory models j Depending on the specific type of sound environment or listening pattern of the hearing aid user, the optimized training parameters of the neural network are determined based on at least one electrical input signal representing the specific sound environment.

8. A hearing aid adapted to be worn in or at the ear of a user, comprising: an input unit for receiving an input sound signal from an environment of a hearing aid user and providing at least one electrical input signal representing the input sound signal; an output unit for providing at least one set of stimuli perceivable as sound to a user based on a processed version of at least one electrical input signal; A processing unit connected to the input unit and the output unit, the processing unit comprising signal processing parameters of the hearing aid to provide a processed version of at least one electrical input signal, wherein the signal processing parameters are determined according to the method of any one of claims 1-7.

9. The hearing aid according to claim 8, further comprising a selector configured to select one of a plurality of sound environment patterns, wherein each pattern represents optimized training parameters of the neural network of the hearing aid determined according to a selected specific type of sound environment or listening pattern of a hearing aid user.

10. The hearing aid according to claim 8, wherein: The processing unit comprises a deep neural network providing optimized training parameters, wherein the deep neural network is trained according to the method according to any one of claims 1-7.

11. A hearing system comprising left and right hearing aids according to any one of claims 8 to 10, wherein: The left and right hearing aids are configured to be worn in or at the left and right ears of the user, respectively, and to establish a wired or wireless connection therebetween, thereby enabling data to be exchanged therebetween.

12. A computer program product, comprising a computer program including instructions, which, when the program is run by a computer, cause the computer to perform the method according to any one of claims 1 to 7.

Images