Method of operating a hearing aid system and a hearing aid system

Abstract

A method (100) of operating a hearing aid system in order to enable improved fine-tuning and such a hearing aid system (200).

Description

[0001] METHOD OF OPERATING A HEARING AID SYSTEM AND A HEARING AID

[0002] SYSTEM

[0003] FIELD OF THE INVENTION

[0004] The present invention relates to a method of operating a hearing aid system. More specifically the invention relates to a method of operating a hearing aid system that enables improvement of speech intelligibility in specific sound environments and for specific user preferences. Further the present invention relates to a hearing aid system adapted to enable said improved speech intelligibility.

[0005] BACKGROUND OF THE INVENTION

[0006] Generally, a hearing aid system according to the invention is understood as meaning any device which provides an output signal that can be perceived as an acoustic signal by a user or contributes to providing such an output signal, and which has means which are customized to compensate for an individual hearing loss of the user or contribute to compensating for the hearing loss of the user. They are, in particular, hearing aids which can be worn on the body or by the ear, in particular on or in the ear, and which can be fully or partially implanted. However, some devices whose main aim is not to compensate for a hearing loss, may also be regarded as hearing aid systems, for example consumer electronic devices (televisions, hi-fi systems, mobile phones, MP3 players etc.) provided they have, however, measures for compensating for an individual hearing loss.

[0007] Within the present context a traditional hearing aid can be understood as a small, battery-powered, microelectronic device designed to be worn behind or in the human ear by a hearing-impaired user. Prior to use, the hearing aid is adjusted by a hearing aid fitter according to a prescription. The prescription is based on a hearing test, resulting in a so-called audiogram, of the performance of the hearing-impaired user’s unaided hearing. The prescription is developed to reach a setting where the hearing aid will alleviate a hearing loss by amplifying sound at frequencies in those parts of the audible frequency range where the user suffers a hearing deficit. A hearing aid comprises one or more microphones, a battery, a microelectronic circuit comprising a signal processor, and an acoustic output transducer. The signal processor is preferably a digital signal processor. The hearing aid is enclosed in a casing suitable for fitting behind or in a human ear.

[0008] Within the present context a hearing aid system may comprise a single hearing aid (a so called monaural hearing aid system) or comprise two hearing aids, one for each ear of the hearing aid user (a so called binaural hearing aid system). Furthermore, the hearing aid system may comprise an external device, such as a smart phone having software applications adapted to interact with other devices of the hearing aid system. Thus, within the present context the term “hearing aid system device” may denote a hearing aid or an external device.

[0009] The mechanical design of hearing aids has developed into a number of general categories. As the name suggests, Behind-The-Ear (BTE) hearing aids are worn behind the ear. To be more precise, an electronics unit comprising a housing containing the major electronics parts thereof is worn behind the ear. An earpiece for emitting sound to the hearing aid user is worn in the ear, e.g. in the concha or the ear canal. In a traditional BTE hearing aid, a sound tube is used to convey sound from the output transducer, which in hearing aid terminology is normally referred to as the receiver, located in the housing of the electronics unit and to the ear canal. In some modern types of hearing aids, a conducting member comprising electrical conductors conveys an electric signal from the housing and to a receiver placed in the earpiece in the ear. Such hearing aids are commonly referred to as Receiver-In-The-Ear (RITE) hearing aids. In a specific type of RITE hearing aids the receiver is placed inside the ear canal. This category is sometimes referred to as Receiver-In-Canal (RIC) hearing aids.

[0010] In-The-Ear (ITE) hearing aids are designed for arrangement in the ear, normally in the funnel-shaped outer part of the ear canal. In a specific type of ITE hearing aids the hearing aid is placed substantially inside the ear canal. This category is sometimes referred to as Completely-In-Canal (CIC) hearing aids. This type of hearing aid requires an especially compact design in order to allow it to be arranged in the ear canal, while accommodating the components necessary for operation of the hearing aid.

[0011] Hearing loss of a hearing impaired person is normally frequency-dependent. This means that the hearing loss of the person varies depending on the frequency. Therefore, when compensating for hearing losses, it is advantageous to utilize frequencydependent amplification.

[0012] Additionally, the amplification is level dependent and adapted for compressing the signal in order to control the dynamic range of the output of the hearing aid. The compression can be regarded as an automatic adjustment of the gain levels for the purpose of improving the listening comfort of the user of the hearing aid and the compression may therefore be denoted Automatic Gain Control (AGC). The AGC also provides the gain values required for alleviating the hearing loss of the person using the hearing aid. Compression may be implemented in the way described in the international application W0-A1 -9934642.

[0013] Finally hearing aids normally comprise anti-feedback algorithms for continuously measuring input levels and output levels as a function of frequency for the purpose of continuously controlling acoustic feedback instability by providing cancellation signals and through lowering of the frequency dependent gain settings when necessary.

[0014] Additionally, it has been suggested to use models for the prediction of the intelligibility of speech after a transmission though a linear system. The most well-known of these models is the “articulation index”, Al, the speech intelligibility index, SII, and the “speech transmission index”, STI, but other indices exist. Additionally, determinations of speech intelligibility have been used to assess the quality of speech signals in telephone lines, see e.g. H. Fletcher and R. H. Galt “The perception of speech and its relation to telephony,” J. Acoust. Soc. Am. 22, 89-151 (1950).

[0015] The ANSI S3.5-1969 standard (revised 1997) provides methods for the calculation of the speech intelligibility index, SII. The SII makes it possible to predict the intelligible amount of the transmitted speech information, and thus, the speech intelligibility in a linear transmission system. The SII is a function of the system’s transfer function and of the acoustic input, i.e. indirectly of the speech spectrum at the output of the system. Furthermore, it is possible to take both the effects of a masking noise and the effects of a hearing aid user’s hearing loss into account in the SII. The SII is always a number between 0 (speech is not intelligible at all) and 1 (speech is fully intelligible). The SII is, in fact, an objective measure of the system’s ability to convey speech intelligibility and hereby hopefully making it possible for the listener to understand what is being said.

[0016] An increase of gain in the hearing aid will always lead to an increase in the loudness of the amplified sound, which may in some cases lead to an unpleasantly high sound level, thus creating loudness discomfort for the hearing aid user.

[0017] The loudness of the output of the hearing aid may be calculated according to a loudness model, e.g. by the method described in an article by B.C.J. Moore and B.R. Glasberg "A revision of Zwicker's loudness model", Acta Acustica Vol. 82 (1996) 335-345, which proposes a model for calculation of loudness in normal -hearing and hearing- impaired subjects. The model is designed for steady state sounds, but an extension of the model allows calculations of loudness of shorter transient-like sounds, too. Reference is made to ISO standard 226 (ISO 1987) concerning equal loudness contours.

[0018] EP-B1-1522206 discloses a hearing aid and a method of operating a hearing aid wherein speech intelligibility is improved based on frequency band gain adjustments based on real-time determinations of speech intelligibility and loudness, and which is suitable for implementation in a processor in a hearing aid.

[0019] This type of hearing aid and operation method requires the capability of increasing or decreasing the gain as a function of frequency. Additionally, the frequency dependent gain may depend on the current sound situation. For bands with high noise levels, e.g., it may be advantageous to decrease the gain, while an increase of gain can be advantageous in bands with low noise levels, in order to enhance the SIL However, such a simple strategy will not always be an optimal solution, as the SII also takes inter-band interactions, such as mutual masking, into account. A precise calculation of the SII is therefore necessary.

[0020] While such a system is generally advantageous it has been found that the amount of gain preferred by an individual hearing aid user may differ from user to user due to the preferences of the individual hearing aid user.

[0021] It is not feasible to compute a general relationship between the SII and a given change in amplification gain analytically and therefore some kind of numerical optimization routine is needed to determine this relationship in order to determine the particular amplification gain that gives the largest SII value. However, deriving an optimization routine that provides optimized speech intelligibility in real time using the limited processing resources in a hearing aid is in no way straightforward.

[0022] However, in most, if not all, of these gain adjustment methods, the gain levels are modified according to algorithms that have been predefined to reflect requirements for generalized situations, which means that these methods are not specifically optimized neither for the specific user nor for the specific sound environments the specific user is experiencing.

[0023] It is therefore a feature of the invention to provide a method of operating a hearing aid system wherein improved hearing aid performance is provided in varying specific sound environments and for specific user preferences.

[0024] It is another feature of the invention to provide a hearing aid system comprising means for providing improved hearing aid performance in varying specific sound environments and for specific user preferences.

[0025] SUMMARY OF THE INVENTION

[0026] The invention in a first aspect provides a method of operating a hearing aid system according to claim 1.

[0027] The invention in a second aspect provides a hearing aid system according to claim 15.

[0028] Further advantageous features appear from the dependent claims.

[0029] Still other features of the present invention will become apparent to those skilled in the art from the following description wherein the invention will be explained in greater detail.

[0030] BRIEF DESCRIPTION OF THE DRAWINGS

[0031] By way of example, there is shown and described a preferred embodiment of this invention. As will be realized, the invention is capable of other different embodiments, and its several details are capable of modification in various, obvious aspects all without departing from the invention. Accordingly, the drawings and descriptions will be regarded as illustrative in nature and not as restrictive. In the drawings: Fig. 1 illustrates highly schematically a method of operating a hearing aid system according to an embodiment of the invention;

[0032] Fig. 2 illustrates highly schematically a hearing aid system according to an embodiment of the invention;

[0033] Fig. 3 illustrates highly schematically a recommender system according to an embodiment of the invention; and

[0034] Fig. 4 illustrates a measurement of aided speech audibility and un-aided speech audibility for a plurality of time periods.

[0035] DETAILED DESCRIPTION

[0036] In the present context the term audibility is generally to be construed as an estimation of whether the acoustical input signal to a hearing aid or the acoustical output signal level from said hearing aid is above the hearing aid user’s hearing threshold and consequently audible. In the following audibility of the acoustical input signal may be used interchangeably with the expression un-aided audibility, which is used to represent the case where speech would be audible for the specific hearing aid user even without the hearing loss compensation provided by the hearing aid system. Similarly, audibility of the acoustical output signal may be used interchangeably with the expression aided audibility.

[0037] Unless specifically mentioned otherwise audibility will only be considered in the case where speech has been detected and therefore the expression audibility is to be construed to mean audibility of speech. Additionally, the expression audibility also covers frequency dependent audibility unless specifically mentioned otherwise.

[0038] Furthermore, it will generally be considered implicit that having determined audibility (as a percentage value) comprises having determined inaudibility since every sound is either one or the other.

[0039] In the following the expressions “at least one time period” and “a plurality of time periods” are generally considered interchangeable.

[0040] In the following the expressions “spatial filtering” and “beam forming” are likewise generally used interchangeably. The expressions recommender engine and recommender system are used interchangeably.

[0041] In the following expressions such as “user account” and “account for a hearing aid system user” are used interchangeably in order to improve readability.

[0042] Fig, 1: Method embodiment

[0043] Reference is first made to Fig. 1 which highly schematically illustrates a method 100 of operating a hearing aid system in an embodiment of the invention.

[0044] In a first step 101 of the method an acoustical input signal is received by (at least one) acoustical-electrical input transducer and converted into a digital input signal.

[0045] In a second step 102 said digital input signal is processed in order to provide a processed digital output signal adapted to alleviate a hearing deficit of a hearing aid system user.

[0046] In a third step 103 an acoustical output signal is generated based on said processed digital output signal.

[0047] In a fourth step 104 it is detected whether speech is present in the sound environment. Within the hearing aid field various speech detection methods are well known and as one example reference can be given to US 9,191,753 B2.

[0048] In a fifth step 105 audibility of at least one of the acoustical input signal and the acoustical output signal for said hearing aid system user with said hearing deficit is estimated for a single or a plurality of time periods, for which speech has been detected.

[0049] Audibility estimation

[0050] In an embodiment the duration of each of said time periods is 12 minutes (but in alternative embodiments the duration of the time periods can be selected from the interval between 2 and 15 minutes).

[0051] Within each of these time periods 11 250 digital input signal samples are obtained by having a sampling frequency of 32 kHz and using only every 1024’th input signal (i.e. an effective sampling frequency of 31.25 Hz). Obviously, the sampling frequency of the digital input signal can be varied without departing from the scope of the invention as long as the sampling frequency satisfies the requirements for frequency resolution and processing resources as will be well known for the skilled person. However, according to this specific embodiment the (audio) sampling frequency (32 kHz) of the hearing aid system has been used.

[0052] Alternatively e.g. every 512thor 256thaudio signal sample can be used (instead of every 1024th) in order to change the effective sampling frequency.

[0053] In an embodiment data is acquired, once every second, from a speech detector, a signal level estimator (such as the percentile detector described in more detail below) and a gain calculator. Based on these data, three counters, representing respectively speech detected, aided audibility and un-aided audibility are likewise updated each second, and every 6’th minute (or whatever the length of the time period) the data are read out from the counters and used to calculate the percentage of times speech was detected in said time period, and the percentage of times the detected speech was audible (i.e. the audible speech percentage) both in the aided and the un-aided case. After the read out the counters are reset and ready for the subsequent time period. Furthermore, using these calculated percentages it becomes possible to calculate the audible speech improvement percentage provided by the hearing aid system by comparing the aided audible speech percentage and the unaided audible speech percentage.

[0054] In an embodiment these calculated data i.e. the speech percentage, the aided audible speech percentage and the unaided audible speech percentage are determined independently for each hearing aid of a binaural hearing aid system and in another embodiment the data from the two hearing aids of the binaural system are subsequently combined into total measures for the calculated percentage values.

[0055] Audibility and in-audibility

[0056] Knowing the output signal level for each of a plurality of signal level samples (comprising speech) enables aided audibility (of speech) to be estimated as a percentage value simply by determining the percentage of said output signal levels that are above the corresponding hearing threshold of the specific user compared to the total number of output signal levels from said plurality of signal level samples for a given time period.

[0057] According to an embodiment input signal level samples can alternatively be used (by multiplying them with the present hearing aid gain) to determine output signal levels and hereby aided audibility as a percentage value.

[0058] In a similar manner un-aided audibility can be estimated as a percentage value simply by determining the percentage of input signal level samples (comprising speech) with a level that are above the hearing threshold of the specific user compared to the total number of signal level samples (comprising speech) for said time period.

[0059] Finally, aided in-audibility can of course be determined as the percentage of output signal level samples (comprising speech) with a level that are below the hearing threshold of the specific user compared to the total number of signal level samples (comprising speech) for said time period.

[0060] Percentiles

[0061] According to an embodiment the input or output signal level is determined as the 90 % percentile value of the considered input or output signal samples.

[0062] Generally, the percentile value is obtained by first ordering the available number of signal samples in increasing (signal level) order, then identifying (or interpolating) the signal sample having a level for which 90 % of the remaining signal samples have levels that are lower, and finally using the level of the identified or interpolated signal sample as the 90 % percentile value.

[0063] More specifically the 90 % percentile is based on absolute (average) signal samples that are updated on a sample-by-sample basis (i.e. for every 1024’th audio signal sample), whereas the 90 % percentile value itself is only updated (i.e. sampled) say every second.

[0064] However, in embodiments the 90 % percentile value (and the associated detected speech and estimated audibility may be obtained with an interval in the range between half a second and two seconds.

[0065] According to embodiments a percentile value between 85 % and 99 % may be used. However, alternatively other methods of determining a signal level than the percentile approach can be used as will be well known for the skilled person.

[0066] Frequency aspect of signal level estimation

[0067] According to an embodiment audibility is only considered within the frequency range from 1125 to 3600 Hz.

[0068] This frequency range is advantageous in being able to represent the majority of common hearing loss types and is also advantageous because it generally has a high impact on speech intelligibility compared to the remaining frequency ranges of human hearing.

[0069] Furthermore, for most fitting rationales this frequency range will always be amplified in order to compensate the hearing deficit of the specific hearing aid user, which of course is very important in the present context, as opposed to refraining from doing so in order to improve listening comfort at the cost of a (limited) reduction in speech intelligibility.

[0070] Finally, by only considering a fraction of the human hearing frequency range the power consumption required to carry out the storing of audibility data can be kept relatively low.

[0071] Alternatively, a frequency range can be selected that is within the range from 1 kHz to 4 Hkz or within the range from 800 Hz to 6 kHz, whereby at least most of the above stated advantages can still be obtained.

[0072] According to an embodiment this frequency range consists of 5 frequency bands within which audibility is estimated independently (as described above) and subsequently audibility for the total frequency range is estimated if more than 50% or more than 60% of the frequency bands are estimated to provide audible (speech) output.

[0073] Time periods

[0074] The inventors have found that the disclosed methods of estimating audibility is especially advantageous in correlating very well with user’s perception of speech being difficult to understand, while on the other hand additionally only requiring limited processing resources. One important part in achieving this is the selection of a time period for estimating the audibility from the range of 4 to 15 minutes or between 6 - 12 such as 12 minutes. However, it is also important to ensure that each time period comprises a sufficient number of audibility estimates, such that a meaningful (aided and un-aided) audible speech percentage can be obtained. In one embodiment this is achieved by estimating the audibility (and detecting speech) every second or from a range between every 100 milliseconds and every 5 seconds.

[0075] Storing data

[0076] In a sixth and final step 106 the estimated audibility, for said plurality of time periods comprising speech, is stored in a data space for further analysis.

[0077] This data space can be implemented in a plurality of ways. The data space can be part of a hearing aid memory or the data space can be part of an external device memory, such as a smart phone memory. In the latter case the smart phone can, in an embodiment, be operationally connected to an external server comprising an account for said hearing aid system user (i.e. a user account), wherein the estimated audibility data can be stored.

[0078] In an embodiment a smart phone application (i.e. a so called “app”) is configured to upload said estimated audibility data from the hearing aid system whenever there is operationally contact between said smart phone and the hearing aid system and subsequently upload the data to an external server comprising an account for said hearing aid system user (i.e. a user account), wherein the estimated audibility data can be stored.

[0079] According to another embodiment, the smart phone is used as a gateway to transfer the audibility data from the hearing aid memory, or (optionally) from two hearing aid memories in case of a binaural hearing aid system, and to a user account.

[0080] Thus, in an embodiment, said user account is configured to enable a hearing care professional or some special algorithm to monitor the data and hereby that improved hearing aid settings for the hearing aid system user can be provided as will be further discussed in the following, especially with reference to Fig. 3. Logging details

[0081] Thus, in an embodiment the estimated audibility is given by three values representing respectively the estimated percentage (over time) and as a function of time of: speech, aided audibility (of speech) and un-aided audibility (of speech). Thus the three percentage values (over time) are determined for each time period that according to this specific embodiment has a duration of 12 minutes. The three values can each be represented by a relatively limited number of bits and consequently it has been demonstrated that data for these three values covering a period of 2 weeks with full daily use and with recording the audibility’s for every 12’th minute can be stored in the hearing memory (i.e. in the hearing aid memory log) without running out of memory.

[0082] This obviously makes it attractive to upload the audibility to an internet server e.g. in the form of a user account as described above, e.g. by using a smart phone or an hearing aid system battery charger providing the same functionality as the smart phone, i.e. either by being operatively connected to an internet server or by being used as a gateway.

[0083] In an embodiment the above mentioned time periods are initiated by at least one of a predetermined scheme, a user initiated trigger and a trigger activated automatically in response to a predetermined criteria being fulfilled.

[0084] Predetermined scheme

[0085] In an embodiment said predetermined scheme is characterized by said plurality of time periods following each other without pause while the hearing aid is turned on. This is possible, at least partly, based on only considering a limited frequency range as already mentioned above.

[0086] User initiated trigger

[0087] In an embodiment a trigger action from the user or from a hearing care professional initiate the above mentioned time periods. According to a more specific embodiment the subsequent audibility estimation (i.e. following the initiation) is only activated for a limited period of time (i.e. a limited number of time periods) following the initiation. Automatic trigger

[0088] In an embodiment the initiation of the above mentioned time periods is triggered automatically in response to a predetermined criteria being fulfilled. In embodiments said predetermined criteria is the overall sound pressure level dropping below a threshold representing soft speech or identification of a preferred conversation partner (such as a spouse) or identification of speech belonging to a conversation partner whose speech has been identified by the hearing aid system user as difficult to understand or identification of a specific geographical location that has been identified by the hearing aid system user as a place where understanding of speech is difficult e.g. due to difficult reverberation effects or due to generally being a place with many people speaking simultaneously (such as an open office space or a cantina) or due to some other reason.

[0089] Aided audibility

[0090] At a first glance it may appear un-necessary to monitor the audibility of speech for users of contemporary hearing aids that are based on well proven and recognized fitting rationales (i.e. the relationship between input signal level and output signal level based on the specific users hearing loss) and advanced speech intelligibility algorithms adapted to provide the hearing aid gain that optimizes the speech intelligibility.

[0091] However, since a fitting rationale deliberately (e.g. in order to make the sound as comfortable as possible) is not designed to amplify all sounds to be audible and because even the most advanced speech intelligibility algorithms are based on some general average assumptions then the inventors have discovered that monitoring the percentage of aided audible (and hereby also aided inaudible) speech provided by the hearing aid for the specific user can be a very beneficial tool when e.g. fine tuning a hearing aid.

[0092] Un-Aided audibility

[0093] Similarly, it has been found that monitoring the percentage of un-aided audible speech provided by the hearing aid for the specific user can be a very beneficial tool especially with respect to improving listening comfort in certain situations but also when used together with the monitoring of aided audible speech. Fig, 2: Hearing aid with audibility estimation means

[0094] Reference is now given to Fig. 2, which illustrates highly schematically a hearing aid 200 (i.e. a monaural hearing aid system) with the parts needed to store for further analysis estimated audibility (i.e. at least one of aided speech audibility and un-aided speech audibility) for the (specific) user of the hearing aid 200.

[0095] At least one acoustical-electrical input transducer (not shown) is configured to receive an acoustical input signal and to provide a digital input signal 201 which is provided to a digital signal processor (not shown) configured to process the digital input signal 201 and hereby provide a processed digital output signal adapted to alleviate a hearing deficit of a hearing aid system user and an electrical-acoustical output transducer (not shown) is configured to receive the processed digital output signal and to generate an acoustical output signal.

[0096] Additionally the hearing aid 200 comprises a speech detector 202 adapted to detect speech in the sound environment based on the digital input signal 201, a processed digital output signal level calculator 203 and memory 204 comprising the hearing aid system users hearing loss. In one embodiment, the speech detector 202 is part of the digital signal processor. In another embodiment the speech detector and the digital signal processor configured to provide the processed digital output signal are not integrated in a single chip.

[0097] At the beginning of each time period each of the speech counter 206, aided speech audibility counter 207 and un-aided speech audibility counter are all initialized. In case speech is detected (for a considered input signal sample) the three counters are updated. First a speech counter 206 is increased by one. Secondly, the calculated output signal level is compared with the hearing loss by the aided speech audibility detector 205-a and in case the calculated output signal level is above the hearing loss an aided speech audibility counter 207 is increased by one. Thirdly the input signal level is compared with the hearing loss by the un-aided speech audibility detector 205-b and in case the input signal level is above the hearing loss an un-aided speech audibility counter 208 is increased by one.

[0098] At the end of said time period the values of said counters 206, 207 and 208 are read out and stored in a data space 209. In an embodiment the total number of considered input signal samples for said time period is used to express the amount of speech, aided speech audibility and un-aided speech audibility as percentage values.

[0099] For the subsequent time period the same procedure is carried out and the new values of the counter 206, 207 and 208 are stored together with the previously stored values of said counter. Hereby speech percentage, aided speech audibility percentage and unaided speech audibility percentage for each time period is obtained as a function of time.

[0100] Positive emotions

[0101] According to one specifically advantageous embodiment at least said aided speech audibility percentage (or some other relative measure instead of percentage such as e.g. fractions, index or decimals) is presented for the hearing aid system user.

[0102] Hereby, hearing aid system user satisfaction, perceived hearing aid benefit and the hearing aid system user’s ability to understand speech-in-noise abilities will be improved for at least some hearing aid system users.

[0103] This type of effect has been proved e.g in the article: “Focusing on Positive Listening Experiences Improves Speech Intelligibility in Experienced Hearing Aid Users” by Lelic D. et al, published in: Trends Hear, 2024 April 2024.

[0104] Negativity bias is a psychological phenomenon where negative experiences are perceptually more salient than positive experiences. This means it will (at least for some hearing aid system users) require only one difficult listening experience with a hearing aid system to overshadow all the positive ones. This could lead hearing aid system users to believe that a hearing aid system is not beneficial for them. In the worst case, the disappointed / dissatisfied hearing aid system user may end up returning the hearing aid system for credit. However, one way to mitigate negativity bias has been found to be to ask the hearing aid system user to actively focus on positive listening experiences while wearing the hearing aid system. The reasoning behind is that by internalizing positive experiences, the overall negative effect of difficult experiences is minimized leading to improved hearing aid system outcomes. Compelling evidence suggests that focusing on positive listening experiences, improves hearing aid satisfaction, perceived hearing aid benefit and even the ability to understand speech-in- noise when wearing a hearing aid system. This effect appears to be especially pronounced when the hearing aid system user writes down the positive experiences. Consequently, the method of operating a hearing aid system (and such a hearing aid system) according to the present invention enables hearing aid system users (by being presented with the measured speech audibility scores) to identify (i.e. become aware of) listening situations where audibility is good and hereby focus on - and appreciate these particular situations.

[0105] Additionally or alternatively the present invention likewise enables hearing aid system users to identify listening situations, where audibility is improved, based on estimated audibility data and based on identifying data (such as the number and type of hearing aid system signal processing algorithms, or a special hearing aid system program or a hearing aid system parameter fine tuning, or at least one of a previously logged geographical location, a sound environment classification and a presence of a specific speaker being detected) and using these identified listening situations to focus on - and appreciate these situations.

[0106] Furthermore, the hearing aid system itself can use said stored estimated audibility data and said stored identified data to modify or recommend a hearing aid system setting and hereby further enable the hearing aid system users ability to take advantage of the positive emotions.

[0107] Thus, at least three different embodiments of the present invention can help a hearing aid system user focus on positive listening experiences whereby at least some users will improve their ability to understand speech in noise generally, which of course for most users is the most important reason for buying a hearing aid system.

[0108] Positive reinforcement, use case 1:

[0109] The combination of identifying a specific listening situation (based on e.g. detecting a specific location, a specific speaker, or some other recognizable social event) with an estimated percentage of the audibility can direct the hearing aid system user’s attention, (e.g. when looking back at the day before going to sleep or at some other time), to at least one specific listening situation that provided a good speed audibility, which (as already explained above) will improve the ability to understand speech in nosie

[0110] Positive reinforcement, use case 2:

[0111] For at least some hearing aid system users, considering the estimated speech audibility for similar listening situations over time can also evoke a positive re-inforcement effect because various hearing aid system fine-tuning mechanisms based on evaluating previous speech audibility estimations can optimize the provided speech audibility for a recurring at least similar situation.

[0112] Examples of such similar listening situations comprises a specific location, some specific (at least one) person or some specific type of recurring social event (that may take place at different locations and sometimes with at least partly different participants but nevertheless be similar in some manner).

[0113] In an embodiment the hearing aid system is adapted to monitor a plurality of identified listening situations an automatically provide the estimated audibility measure for at least the listening situations which has shown the most significant improvement.

[0114] Positive reinforcement, use case 3:

[0115] As a final example the performance of a specific hearing aid system algorithm, such as e.g. a beam forming algorithm or a deep neural network based noise reduction, can be monitored, thus in way similar to use case 2, explained above, can help the user better appreciate the improvements provided due to e.g. fine tunings carried out by the hearing aid system.

[0116] The optimum audible speech measure

[0117] However, the present invention additionally or alternatively provides an improved method of hearing aid system finetuning wherein the user for a specific listening situation evaluates how good the perception of speech was in that specific listening situation and based hereon adapts the hearing aid system to provide a particular optimum audibility measure for said specific listening situation.

[0118] This is a particularly novel and inventive approach because it is based on taking into account that it is not an optimum solution to seek to maximize audibility generally because especially in e.g. cocktail party situations it is not desirable to make all speech audibility - on the contrary only a minor part of the speech is actually relevant for the hearing aid system user.

[0119] Additional Benefits

[0120] Positive emotions can lead to higher conversion rates for the hearing clinic. Due to the hearing aid system user becoming a satisfied user faster, a hearing care professional (i.e. a HCP) would need to spend less time on the same costumer (due to fewer followups), and consequently will have more time for new customers.

[0121] Positive emotions is both relevant for first-time hearing aid system users as well as for experienced hearing aid system users while adjusting to the sound of a new hearing aid system. People with a more cautious attitude, and some of the sceptics could be more positive about buying the hearing aid system user when they have contemplated about their positive listening experiences, and this will be based on the fact that at least for some / most hearing aid system users the ability to understand speech in noise will have improved.

[0122] Fig, 4: Graph of estimated audibility

[0123] Reference can, as one example be made to Fig. 4, which illustrates highly schematically a graph 400 comprising speech percentage 402, aided speech audibility percentage 401 and un-aided speech audibility percentage 403 for a plurality of time periods obtained as a function of time. Thus the x-axis represents time and the y-axis represents percentage values.

[0124] Fig 3: Recommender system

[0125] Reference is now given to Fig. 3, which illustrates highly schematically a hearing aid recommender system 300 (which in the following may simply be denoted recommender system). The recommender system 300 comprises data spaces 301-1, 301-2, . . ,301-N holding for a plurality (N) of hearing aid users at least measures of estimated speech, aided audibility and un-aided audibility as a function of time (i.e. for a plurality of time periods, which in the following may also be denoted a time interval). These data are preferably received directly from the hearing aids, e.g. by using a smart phone as already described above. However, alternatively the data may be provided from a hearing care professional connected to the recommender system over the internet.

[0126] Storing additional data together with Audibility

[0127] Additionally, the recommender system 300 comprises data spaces 303, that for each of said plurality of hearing aid system users hold additional information, based on user consent and optionally in anonymized form, in the form of at least one of:

[0128] - type of hearing aid (as is well known many different hearing aid types exist that differ in many different ways including just to name some whether the type is e.g. behind-the-ear or in-the-ear or some other form factor, whether the earpiece is of the custom made or so called instant-fit type, and whether the hearing aid type is designed for mild or severe hearing losses);

[0129] - audiogram;

[0130] - fitting rationale used;

[0131] - demographic data such as, age, gender, spoken language, nationality, occupation, education level, marital status, family size etc., and

[0132] - hearing aid fine tunings carried out by a hearing care professional or the user, thus this data may provide information about the number of fine tunings carried out, which is valuable since it can be construed as a measure of how satisfied the hearing aid user is with the hearing aid system.

[0133] Finally, the recommender system 300 comprises a recommender system 302 that is configured to analyse the information held in the data spaces 301 and 303 and in response hereto identify at least one of a specific hearing aid type, specific hearing loss, specific demographic characteristic, specific fitting rationale and a specific type of hearing aid fine tuning having a less than optimum amount of at least one of aided audible speech and un-aided audible speech.

[0134] How to identify less than optimal situations

[0135] In an embodiment said identification is made simply by combining the information held in the data spaces 301 and 303 and hereby providing reference tables with averaged audibility scores (i.e. the average of the amount of aided or un-aided audibility as a function of a selected time interval).

[0136] In an embodiment the selected time interval can be anything from a single time period (such as e.g. 12 minutes), a single day, 14 days, a month or more. Obviously it requires multiple read-outs (i.e. upload of data) from the hearing aid system to the data spaces 301-1,. . ,301-N in order to obtain the longer time intervals.

[0137] Having these averaged audibility scores a specific user’s estimated audibility can easily be determined to be within the expected range or not. According to one embodiment a specific hearing aid user’s estimated audibility is identified (to be too low) if the estimated audibility is lower than say 90 % of the average of a plurality of other hearing aid users estimated audibility. However, not only specific hearing aid users with a low audibility score can be identified. The same is true for a specific type of hearing loss or a specific hearing aid type or some specific demographic characteristic etc.

[0138] According to another embodiment the threshold for determining when an estimated audibility is considered to be too low is based on fitting the available audibility scores to a normal distribution and identifying the audibility scores that are say more than two standard deviations lower than the mean audibility score.

[0139] Additionally, the threshold for determining when an estimated audibility is considered to be too low can of course deviate from the above mentioned 90 % and so can it from the number of standard deviations below the mean of a fitted normal distribution.

[0140] How to react to an identified low audibility score

[0141] In an embodiment the recommender system 302 continuously monitors the collected audibility scores and in case a specific user, or a specific user characteristic such as age or martial status, or a specific hearing aid type or a hearing aid fitting rationale has a too low audibility score, an action is triggered by the recommender system 302.

[0142] In an embodiment the recommender system simply alerts the associated hearing care professional for a specific hearing aid user if a too low audibility score is detected. In an embodiment the recommender system alerts the relevant hearing aid system manufacturer if a too low audibility score is detected for a specific hearing aid type or for a specific hearing aid fitting rationale or for a specific demographic hearing aid user characteristic.

[0143] In an embodiment the recommender system 300 can, if a too low audibility score is detected for e.g. a specific hearing aid user, control the hearing aid system of said specific user to record a sound bite when a too low audibility score is detected. However, this embodiment can also be applied if a too low audibility score is detected for a specific hearing aid type, fitting rational, hearing loss or any of the other data types comprised in the data spaces 303.

[0144] According to an alternative embodiment the recommender system can control an app paired with the specific hearing aid system in order to facilitate the recording of the sound bite. According to one embodiment the app itself can record the sound bite.

[0145] It is noted that unless otherwise noted the actions triggered by the recommender system according to the various embodiments described above can be combined.

[0146] A quantitative measure based on estimated audibility

[0147] Thus, in an embodiment, the estimated audibility (of speech) is generally used to provide a quantitative measure of the quality of the processing of the digital input signal based on the specific circumstances a specific (hearing aid system) user experiences.

[0148] This can be very helpful by enabling e.g. a hearing care professional or some machine learning algorithm to suggest fine tunings of the hearing aid system for said specific user.

[0149] In an (other) embodiment the estimated audibility is collected for a plurality users together with additional information such as e.g. demographic user data, whereby a quantitative measure of hearing aid system performance for a specific type of users can be obtained an used to enable a hearing care professional or some machine learning algorithm to suggest fine tunings for said specific type of hearing aid system users. In a similar manner the estimated audibility can be used as a quantitative measure of the performance of a specific hearing aid system type by analysing the estimated audibility for plurality of different users wearing the same type of hearing aid system in order to identify possible improvements of the general hearing aid system set-up for said type of hearing aid system.

[0150] Quantitative measure and additional data

[0151] In an embodiment the above mentioned use of the estimated audibility to provide a quantitative measure of the hearing aid system performance is combined with identifying at least one of:

[0152] - the number and type of hearing aid system signal processing algorithms that are active in at least one of the time periods used to estimate the audibility, and

[0153] - at least one of a special hearing aid system program and a parameter fine tuning, initiated by said hearing aid system user, and being active in said at least one time period, and

[0154] - at least one of a previously logged geographical location, sound environment classification and a presence of a specific speaker being detected in said at least one time period.

[0155] The above mentioned signal processing algorithms at least include speech intelligibility enhancement and noise reduction algorithms as well as the use of either frequency domain signal processing or so called time-domain (or low delay) signal processing based on a time domain adaptive filter, such as described e.g. in WO-A1-2019211187.

[0156] Hearing aid modification based on a quantitative measure

[0157] In an embodiment the above mentioned stored estimated audibility data and stored identified data is used to modify or recommend a hearing aid system setting in the form of at least one of:

[0158] - modifying said, at least one, specific hearing aid system signal processing algorithm; and

[0159] - modifying said special hearing aid system program or parameter fine tuning; and - providing, at least one, special hearing aid program or parameter fine tuning adapted to only be active in response to, at least one of, said previously logged geographical location, sound environment classification and presence of a specific speaker being detected.

[0160] Hearing aid modification approaches

[0161] In an embodiment said step of using said stored estimated audibility data and said stored identified data to modify or recommend a modification of the hearing aid system settings comprises the further step of

[0162] - providing said stored data as input to an automatic optimization algorithm adapted to modify hearing aid settings controlling the processing of the digital input signal; or

[0163] - providing said stored data to a hearing care professional.

[0164] Spatial filtering

[0165] Narrow Beam output

[0166] In an embodiment the processing of the digital input signal comprises spatial filtering in order to provide a processed digital output signal (adapted to alleviate a hearing deficit of a hearing aid system user) in the form of a directional signal representing sound from a direction said hearing aid system user is assumed to primarily pay attention to. In an embodiment said direction the hearing aid system user is assumed to primarily pay attention to is the look direction.

[0167] In an embodiment said directional signal has a narrow width of say + / - ten degrees around said direction or in the range of + / - five degrees to + / - fifteen degrees. In an embodiment the width of the directional signal is defined as the angle of deviation from the central angle (i.e. the direction) wherein the sensitivity compared to the sensitivity at the central angle has decreased by 3 dB. However, as will be obvious for the skilled person the width of a directional signal can be defined in many different ways.

[0168] Using spatial filtering to provide a directional signal with a narrow width is especially advantageous for the present invention because it can improve the benefit of estimating audibility in order to fine tune hearing aid system settings because low audibility situations that are due to people talking from a direction not being the look direction (i.e. from another table in the restaurant or from another room at home or in an office) can be removed from the provided audibility estimates, whereby the provided audibility estimates, more specifically low audibility estimates better reflect at situation where the provided audibility is not as intended and should be improved.

[0169] Narrow beams for analysis only

[0170] However, some hearing aid system users may not prefer narrow directionality such as described above and therefore, in an embodiment, the step of estimating, for at least one time period comprising speech, audibility of at least one of the acoustical input signal and the acoustical output signal for said hearing aid system user with said hearing deficit is replaced by the steps of:

[0171] - estimating, for at least one time period comprising speech, audibility of a second processed digital output signal provided using spatial filtering in order to provide a directional signal representing sound from a direction said hearing aid system user is assumed to primarily pay attention to, wherein said second processed digital output signal (i.e. said directional signal) has a more narrow width than said processed digital output signal used to generate said acoustical output signal.

[0172] In an alternative or additional embodiment audibility is estimated for a third processed digital output signal provided using spatial filtering in order to provide a narrow directional signal representing sound from a direction said hearing aid system user is assumed to not primarily pay attention to.

[0173] Thus the second and third processed digital output signal distinguishes the processed digital output signal based on which the acoustical output signal is generated in not being used to generate the acoustical output signal and in providing a more narrow directional signal and / or directional signal not pointing in (i.e. receiving sound from) the look direction of the hearing aid system user.

[0174] The second and third processed digital output signals may be provided together or independent on each other.

[0175] Hearing aid modification based on audibility and preferred speaker

[0176] In an embodiment the hearing aid system is adapted to carry out a method of operating the hearing aid system comprising the steps of identifying (i.e. detecting) a preferred conversation partner such as a spouse, another family member, a close colleague at work, a private friend or some other conversation partner that the hearing aid system user often interacts with and typically has a difficult time understanding and in response to such a detection increase the sound pressure level (i.e. by increasing the hearing aid gain) provided by the hearing aid system if it additionally is detected that the estimated aided audibility is below a first threshold level.

[0177] In an embodiment said first threshold level is 90 % aided audibility (for speech) or between 85% and 95% aided audibility.

[0178] It is noted that methods for identifying a specific person speaking are known, reference can e.g. be given to the article “Improving speaker discrimation of target speech extraction with time domain speakerbeam” by Marc Delecroix et al. and published 23. January 2020 on arxiv.org. Herein it is disclosed how a previously acquired utterance of the target speaker can subsequently be used to identify said target speaker.

[0179] Thus in an embodiment the first threshold level is pre-determined, but according to another embodiment the first threshold level is adaptive such that the hearing aid system can impact the first threshold level e.g. by the user activating an input mechanism, such as a button, when not satisfied with the provided speech intelligibility from the conversation partner and in response hereto the threshold level can be increased. However, the adaptive threshold level can also be decreased in order to e.g. improve listening comfort.

[0180] Logging data based on audibility

[0181] In an embodiment the hearing aid system is adapted to carry out a method of operating the hearing aid system comprising the steps of logging at least one of a geographical location, a sound environment classification and a presence of a specific speaker for a time period (or at least one time period) initiated by a user initiated trigger if the estimated aided audibility for said time period is below a second threshold.

[0182] In an embodiment said second threshold level is 95 % aided audibility (for speech) or between 85% and 98% aided audibility. This approach is advantageous in logging information that can assist a hearing care professional or some machine learning algorithm such as a neural network or a deep neural network (DNN) to identify hearing situations and / or circumstances based on both the hearing aid user’s input (i.e. the users perception of the hearing situation is reflected in whether or not the user triggers the logging) and an objective performance measure in the form of the audibility experienced by the hearing aid system user.

[0183] In an embodiment this method of operating the hearing aid system comprises the further step of not logging the above mentioned data if the estimated aided audibility for said time period is also below a third threshold that is significantly lower than the second threshold and adapted to represent the case where the level of the detected speech is so low that it most likely was not of interest for the user. In an embodiment this third threshold is selected from an interval between 25 % and 1% aided audible speech.

[0184] Thus if the two embodiments comprising the above mentioned first and second thresholds respectively are combined then the first threshold value will be selected to be lower than the second threshold, whereby the audibility needs to be lower in order to trigger an (automatic) increase in gain, compared to just logging a specific sound environment or a geographical location.

[0185] Request user input based on logged data

[0186] In an embodiment the hearing aid system is adapted to carry out a method of operating the hearing aid system comprising the steps of identifying at least one of a geographical location, a sound environment classification and a specific speaker presence that previously has been logged and in response hereto requesting the user to rate the perceived speech intelligibility in response to said identification.

[0187] Hereby it is ensured that a specific hearing situation and / or specific circumstances that at least once has been identified as difficult, when encountered again will trigger that the hearing aid system user is asked whether the situation is still considered problematic with respect to understanding speech such that appropriate action can be taken, such as increasing the sound pressure level.

[0188] In an (alternative compared to the previous) embodiment the hearing aid system is adapted to carry out a method of operating the hearing aid system comprising the steps of identifying at least one of a geographical location, a sound environment classification and a specific speaker presence that previously has been logged and in response hereto executing at least one of: adapting automatically the sound pressure level of the generated acoustical output signal in response to said identification; and estimating, for at least one time period comprising speech, audibility of the acoustical output signal for said hearing aid system user with said hearing deficit.

[0189] Hereby it is ensured that a specific hearing situation and / or specific circumstances that at least once has been identified as difficult, when encountered again will trigger that the hearing aid system is adapted accordingly e.g. by increasing the gain of the hearing aid system and / or will trigger that the audibility is estimated again in order to determine whether the hearing aid gain (i.e. the sound pressure level of the generated acoustical output signal) shall also be increased next time the specific hearing situation and / or specific circumstances are encountered.

Claims

CLAIMS1. A method of operating a hearing aid system, comprising the steps of:- receiving at least one acoustical input signal and converting it into at least one digital input signal;- processing said at least one digital input signal to provide a processed digital output signal adapted to alleviate a hearing deficit of a hearing aid system user;- generating an acoustical output signal based on said processed digital output signal,- detecting whether speech is present in the sound environment;- estimating, for at least one time period comprising speech, audibility of at least one of the acoustical input signal and the acoustical output signal for said hearing aid system user with said hearing deficit, and- storing for further analysis, in a data space, said estimated audibility for said at least one time period comprising speech.

2. The method according to claim 1, comprising the further step of increasing the sound pressure level of the generated acoustical output signal in response to:- said estimated audibility of the acoustical output signal being below a first threshold, and- said detected speech being identified to belong to a preferred conversation partner of the hearing aid system user or being identified to belong to a conversation partner whose speech has been identified by the hearing aid system user as difficult to understand.

3. The method according to claim 1 or 2, comprising the further step of logging at least one of a geographical location, a sound environment classification and a presence of a specific speaker in response to:- said hearing aid system user operating a hearing aid system user interface to indicate that perceived speech intelligibility is not satisfactory; and- said estimated audibility being below a second threshold.

4. The method according to claim 3, comprising the further steps of:- identifying at least one of a geographical location, a sound environment classification and a specific speaker presence that previously has been logged and in response hereto- requesting the user to rate the perceived speech intelligibility in response to said identification.

5. The method according to any of the preceding claims comprising the step of initiating said at least one time period based on:- a predetermined scheme; or- a trigger activated automatically in response to a predetermined criteria being fulfilled; or- a trigger provided by the hearing aid system user or a hearing care professional.

6. The method according to any of the preceding claims, wherein said audibility is only estimated based on the frequency interval between 1 and 4 kHz or between 800 Hz and 6 kHz.

7. The method according to any of the preceding claims, wherein said step of estimating the audibility of the acoustical input signal comprises:- multiplying a percentile, in the range between 85% and 99%, of the digital input signal with a gain adapted to alleviate the hearing deficit of the hearing aid user; and- comparing the result of said multiplication with a hearing loss threshold of the hearing aid system user; or wherein said step of estimating the audibility of the acoustical output signal comprises:- comparing a percentile in the range between 85% and 99%, of the digital output signal with a hearing loss threshold of the hearing aid system user.

8. The method according to any of the preceding claims, wherein the step of processing said at least one digital input signal to provide a processed digital output signal adapted to alleviate a hearing deficit of a hearing aid system user comprises the further step of:- using spatial filtering to provide a processed digital output signal in the form of a directional signal.

9. The method according to any of the preceding claims, wherein the step of detecting whether speech is present in the sound environment comprises the further step of using a second processed digital output signal to determine whether speech is present, wherein said second processed digital output signal is provided by using spatial filtering to provide a directional signal and wherein said second processed digital output signal is not used to generate the acoustical output signal.

10. The method according to any of the preceding claims comprising the further step of- using the estimated audibility to provide a quantitative measure of the quality of the processing of the digital input signal.

11. The method according to claim 10 comprising the further steps of- identifying data in the form of at least one of- - the number and type of hearing aid system signal processing algorithms that are active in said at least one time period comprising speech, and- - a special hearing aid system program or a parameter fine tuning, initiated by said hearing aid system user, and being active in said at least one time period comprising speech, and- - at least one of a previously logged geographical location, a sound environment classification and a presence of a specific speaker being detected in said at least one time period comprising speech; and- storing for further analysis said identified data together with said estimated audibility for each of said at least one time period comprising speech.

12. The method according to claim 11 comprising the further steps of- using said stored estimated audibility data and said stored identified data to modify or recommend a hearing aid system setting in the form of at least one of- - modifying said, at least one, specific hearing aid system signal processing algorithm; and- - modifying said special hearing aid system program or parameter fine tuning;and- - providing, at least one, special hearing aid program or parameter fine tuning adapted to only be active in response to, at least one of, said previously logged geographical location, sound environment classification and presence of a specific speaker being detected.

13. The method according to claim 12 wherein said step of using said stored estimated audibility data and said stored identified data to modify or recommend a modification of the hearing aid system settings comprises the further step of- providing said stored data as input to an automatic optimization algorithm adapted to modify hearing aid settings controlling the processing of the digital input signal; or- providing said stored data to a hearing care professional.

14. The method according to any of the preceding claims, comprising the further step of- transmitting the estimated audibility to an external server and storing the estimated audibility in said external server, wherein said external server is further configured to receive and store, for each of a plurality of hearing aid system users, at least one of- type of hearing aid used;- audiogram;- fitting rationale used;- demographic data such as, age, gender, spoken language, nationality, occupation, education level, marital status, family size etc., and- hearing aid fine tunings carried out by a hearing care professional or the user.

15. The method according to any of the preceding claims, comprising the further step(s) of- presenting for said hearing aid system user a ratio of respectively inaudible and audible speech based on said estimated audibility, for at least one time period comprising speech.

16. The method according to claim 15, wherein said at least one time period is selected from a group of time periods consisting of or comprising:- an hour, a day, a week, a month, a year,- a specific time limited occasion such as sharing a meal or just meeting with specific friends and / or family and / or colleagues at some specified or unspecified location, and- a specific time limited occasion at some specific location such as at home or outside home such as at a specific restaurant or at some other often visited location such as a workplace cantina or a workplace office or some preferred spot in a park or a forest or at some other specific location.

17. The method according to any of the preceding claims, wherein the duration of each of said time periods is 12 minutes or selected from the interval between 2 and 15 minutes.

18. The method according to claim 17, wherein within each of these time periods signal samples are obtained with an effective sampling frequency of 31.25 Hz, or an effective sampling frequency in the range between 20 and 40 Hz.

19. A hearing aid system (200) comprising:- an acoustical-electrical input transducer (201) configured to receive an acoustical input signal and to provide a digital input signal;- a digital signal processor configured to process the digital input signal and hereby provide a processed digital output signal adapted to alleviate a hearing deficit of a hearing aid system user;- an electrical-acoustical output transducer configured to receive the processed digital output signal and to generate an acoustical output signal;- a speech detector (202) adapted to detect speech in the sound environment based on the digital input signal;- a data space (209) configured to store data for further analysis; wherein the digital signal processor is further configured to:- estimating, for at least one time period, in response to speech being detected, the audibility of at least one of the acoustical input signal and the acoustical outputsignal for said hearing aid system user with said hearing deficit; and- storing said estimated audibility in said data space (209).

20. The hearing aid system according to claim 19, further comprising at least one of:- a user interface adapted to enable said hearing aid system user to initiate estimation of audibility; and- the digital signal processor being adapted to optionally estimate audibility according to a predetermined scheme that is independent of said user interface.

Images