Dynamic Virtual Hearing Modeling

Abstract

Techniques for generating, updating, and / or using a virtual hearing model associated with a recipient of a hearing prosthesis are presented herein. The virtual hearing model of the recipient is generated and updated based on psychoacoustic data associated with the recipient and, in some cases, based on psychoacoustic data collected from one or more selected hearing prosthesis recipient populations. The recipient-specific virtual hearing model can be used in real time to determine one or more settings of the hearing prosthesis.

Description

Technical Field

[0001] The present invention relates generally to hearing prostheses. Background Art

[0002] Hearing loss is a sensory disorder that is generally of two types, conductive and / or sensorineural. Conductive hearing loss occurs when the normal mechanical pathways of the outer and / or middle ear are blocked (e.g., by damage to the ossicular chain or the ear canal). Sensorineural hearing loss occurs when the inner ear or the neural pathways from the inner ear to the brain are damaged.

[0003] Because the hair cells in the cochlea are not damaged, individuals with conductive hearing loss typically have some form of residual hearing. As such, individuals with conductive hearing loss typically receive hearing prostheses that generate cochlear fluid movement. Such hearing prostheses include, for example, acoustic hearing aids, bone conduction devices, and direct acoustic stimulators.

[0004] However, in many severely deaf people, the cause of their deafness is sensorineural hearing loss. People with some forms of sensorineural hearing loss cannot obtain adequate benefits from hearing prostheses that generate mechanical movement of the cochlear fluid. These individuals can benefit from implantable hearing prostheses that stimulate the nerve cells of the recipient's auditory system in other ways (e.g., electrically, optically, etc.). Cochlear implants are usually recommended when sensorineural hearing loss is attributed to the loss or destruction of cochlear hair cells, which convert acoustic signals into nerve impulses. Auditory brainstem stimulators are another type of stimulating hearing prosthesis that may also be recommended when the recipient experiences sensorineural hearing loss due to damage to the auditory nerve. Summary of the invention

[0005] In one aspect, a method is provided. The method includes: receiving a request for updated sound processing settings from a hearing prosthesis configured to be worn by a recipient, wherein the request includes listening situation data representing an intended listening situation for the hearing prosthesis; determining selected sound processing settings for use by the hearing prosthesis in the intended listening situation based on a recipient-specific virtual hearing model that takes into account how the hearing prosthesis operates and contributes to the recipient's perception; and sending the selected sound processing settings to the hearing prosthesis.

[0006] In another aspect, a method is provided. The method includes: generating a virtual hearing model based on recipient-specific psychoacoustic data, wherein the virtual hearing model is a holistic model of the hearing system of a recipient of a hearing prosthesis, and wherein the virtual hearing model takes into account the operation of each of the recipient's outer ear system, middle ear system, and inner ear system in a bilateral manner, along with the recipient's auditory cortex and brain's hearing cognition, and how the hearing prosthesis operates and contributes to the recipient's perception; using the virtual hearing model to determine selected sound processing settings for use with the hearing prosthesis; and instantiating the selected sound processing settings at the hearing prosthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The embodiments of the present invention are described herein with reference to the accompanying drawings, wherein

[0008] Figure 1A is a schematic diagram illustrating a cochlear implant according to certain embodiments presented herein;

[0009] Figure 1B yes Figure 1A A block diagram of a cochlear implant;

[0010] Figure 2 is a schematic diagram illustrating the operation of a portion of the techniques presented herein for generating, updating and / or using a recipient-specific virtual hearing model according to certain embodiments presented herein;

[0011] Figure 3 is a flow chart of a method according to certain embodiments presented herein;

[0012] Figure 4 is a block diagram of a computing system configured to implement aspects of the proposed technology according to certain embodiments presented herein;

[0013] Figure 5 is a functional block diagram illustrating a bone conduction device according to certain embodiments presented herein; and

[0014] Figure 6 is a flow chart of a method according to certain embodiments presented herein. DETAILED DESCRIPTION

[0015] Techniques for generating, updating, and / or using a virtual hearing model associated with a recipient of a hearing prosthesis are presented herein. The virtual hearing model of the recipient is generated and updated based on psychoacoustic data associated with the recipient and, in some cases, based on psychoacoustic data collected from one or more selected groups of hearing prosthesis recipients. The recipient-specific virtual hearing model may be used in real time to determine one or more settings of the hearing prosthesis.

[0016] For ease of description only, the technology presented herein is primarily described herein with reference to an exemplary hearing / hearing prosthesis (i.e., a cochlear implant). However, it should be appreciated that the technology presented herein can also be used with a variety of other types of devices, including other hearing prostheses. For example, the technology presented herein can be implemented in, for example, acoustic hearing aids, auditory brainstem stimulators, bone conduction devices, middle ear hearing prostheses, direct acoustic stimulators, bimodal hearing prostheses, bilateral hearing prostheses, etc. As such, the description of the present invention with respect to a cochlear implant should not be construed as limiting the scope of the technology presented herein.

[0017] Figure 1A is a schematic diagram of an exemplary cochlear implant 100 configured to implement aspects of the technology presented herein, and Figure 1B is a block diagram of a cochlear implant 100. Figure 1A and Figure 1B Give a description.

[0018] As shown, the recipient has an outer ear 101, a middle ear 103, and an inner ear 105. The elements of the outer ear 101, the middle ear 103, and the inner ear 105 are described below, and then the cochlear implant 100 is described.

[0019] In a functioning human hearing anatomy, the outer ear 101 includes the pinna 121 and the ear canal 109. Sound waves or sound pressure 107 are collected by the pinna 121 and routed into and through the ear canal 109. The eardrum 111, which vibrates in response to the sound waves 107, is disposed across the distal end of the ear canal 109. This vibration is coupled to the oval window or ovate window 123 through three bones of the middle ear 103, which are collectively referred to as the ossicles 113 and include the malleus 115, the incus 117, and the stapes 119. The ossicles 113 of the middle ear 103 serve to filter and amplify the sound waves 107, thereby causing the oval window 123 to vibrate. Such vibrations set up waves of fluid motion within the cochlea 133. This fluid motion in turn activates the hair cells (not shown) that are lined up inside the cochlea 133. Activation of the hair cells causes the transmission of appropriate nerve impulses via the spiral ganglion cells and the auditory nerve 125 to the brain (not shown), where they are perceived as sound.

[0020] The cochlear implant 100 includes an external component 102 and an internal / implantable component 104. The external component 102 is attached directly or indirectly to the body of the recipient and typically includes an external coil 106 and typically includes a magnet fixed relative to the external coil 106 (not shown in FIG. 1 ). The external component 102 also includes one or more input elements / devices 134 for receiving input signals at the sound processing unit 112. In this example, the one or more input devices 134 include a plurality of microphones 108 (e.g., microphones positioned at the pinna 110 of the recipient, a telecoil, etc.), which are configured to capture / receive input acoustic / sound signals (sounds); one or more auxiliary input devices 129 (e.g., a telecoil, one or more audio ports (such as direct audio input (DAI)), a data port (such as a universal serial bus (USB) port, a cable port, etc.), and a wireless transmitter / receiver (transceiver) 135), each of which is located in, on, or near the sound processing unit 112.

[0021] The sound processing unit 112 also includes, for example, at least one battery 127, a radio frequency (RF) transceiver 131, and a processing block 150. The processing block 150 includes a plurality of elements, including a sound processing module 152 and a psychoacoustic monitoring system 154. As further described below, in certain embodiments, the psychoacoustic monitoring system 154 includes an environment classifier 156. Each of the sound processing module 152 and the psychoacoustic monitoring system 154 may be formed by one or more processors (e.g., one or more digital signal processors (DSPs), one or more uC cores, etc.), firmware, software, etc., arranged to perform the operations described herein. That is, the sound processing module 152 and the psychoacoustic monitoring system 154 may each be implemented as a firmware element, partially or completely implemented in digital logic gates in one or more application specific integrated circuits (ASICs), partially or completely implemented in software, etc.

[0022] As further described below, in some examples, the psychoacoustic monitoring system 154 is configured to monitor or analyze sound signals received at one or more input devices 134. More specifically, the psychoacoustic monitoring system 154 can be configured to capture, extract, determine, or otherwise obtain recipient-specific psychoacoustic data from the sound signals for use in generating or updating a recipient-specific virtual hearing model of the recipient. Additionally or alternatively, as also described below, the psychoacoustic monitoring system 154 can be configured to initiate a request for updated settings (such as updated sound processing settings) of the cochlear implant 100. The request for updated settings can include so-called listening situation data, which represents the expected listening situation of the recipient of the cochlear implant 100. The listening situation data is generated by the psychoacoustic monitoring system 154 from the sound signals received at one or more input devices 134. Further details of the psychoacoustic monitoring system 154 onboard (i.e., integrated therein) the cochlear implant 100 are provided below.

[0023] As noted, in certain embodiments, the psychoacoustic monitoring system 154 may include an environment classification module (environment classifier) ​​156 that is configured to evaluate / analyze the received sound signal (sound) and determine the sound category / class / sound environment. That is, the environment classifier 156 is configured to use the received sound to "classify" the surrounding sound environment and / or sound into one or more sound categories (i.e., determine the input signal type). The sound category or environment may include, but is not limited to, "speech" (e.g., the sound signal mainly includes a speech signal), "noise" (e.g., the sound signal mainly includes a noise signal), "speech+noise" (e.g., there is both speech and noise in the sound signal), "wind" (e.g., the sound signal mainly includes a wind signal), "music" (e.g., the sound signal mainly includes a music signal), and "quiet" (e.g., the sound signal includes minimal speech or noise signals. The environment classifier 156 may also estimate the signal-to-noise ratio (SNR) of the sound.

[0024] return Figure 1A and Figure 1B In an exemplary embodiment of the present invention, the implantable component 104 includes an implant body (main module) 114, a lead region 116, and an intracochlear stimulation assembly 118, all of which are configured to be implanted beneath the skin / tissue (tissue) 145 of a recipient. The implant body 114 typically includes an airtight housing 115 in which the RF interface circuitry 124 and the stimulator unit 120 are disposed. The implant body 114 also includes an internal / implantable coil 122, which is typically external to the housing 115 but is connected to the stimulator unit 120 via an airtight feedthrough ( Figure 1B ) is connected to the RF interface circuit system 124.

[0025] As noted, the stimulation assembly 118 is configured to be at least partially implanted in the recipient's cochlea 133. The stimulation assembly 118 includes a plurality of longitudinally spaced intracochlear electrical stimulation contacts (electrodes) 126 that collectively form a contact or electrode array 128 for delivering electrical stimulation (electrical current) to the recipient's cochlea. The stimulation assembly 118 extends through an opening (e.g., cochleostomy, round window, etc.) in the recipient's cochlea and has a proximal end that is connected via the lead region 116 and a sealed feedthrough ( Figure 1B The lead region 116 includes a plurality of conductors (wires) that electrically couple the electrodes 126 to the stimulator unit 120.

[0026] As noted, the cochlear implant 100 includes an external coil 106 and an implantable coil 122. The coils 106 and 122 are typically wire antenna coils, each of which includes multiple turns of electrically insulated single or multiple strands of platinum or gold wire. Typically, a magnet is fixed relative to each of the external coil 106 and the implantable coil 122. The magnet being fixed relative to the external coil 106 and the implantable coil 122 facilitates operational alignment of the external coil with the implantable coil. This operational alignment of the coils 106 and 122 enables the external component 102 to transmit data, and possibly power, to the implantable component 104 via a tightly coupled wireless link formed between the external coil 106 and the implantable coil 122. In some examples, the tightly coupled wireless link is a radio frequency (RF) link. However, various other types of energy transfer, such as infrared (IR) transfer, electromagnetic transfer, capacitive transfer, and inductive transfer, may be used to transfer power and / or data from the external component to the implantable component, thus, Figure 1B Only one example arrangement is shown.

[0027] As noted above, the processing block 150 includes a sound processing module 152. The sound processing module 152 (e.g., one or more processing elements implementing firmware, software, etc.) is configured to generally convert an input sound signal into a stimulation control signal 136 for stimulating a first ear of a recipient (i.e., the sound processing module 152 is configured to perform sound processing on an input sound signal received at one or more input devices 134 to generate a signal 136 representing electrical stimulation for delivery to the recipient). The input sound signal that is processed and converted into a stimulation control signal may be an audio signal received via the microphone 108 or any of the other input devices 134.

[0028] exist Figure 1BIn an embodiment of the present invention, the stimulation control signal 136 is provided to the RF transceiver 131, which transmits the stimulation control signal 136 (e.g., in an encoded manner) transcutaneously to the implantable component 104 via the external coil 106 and the implantable coil 122. That is, the stimulation control signal 136 is received at the RF interface circuit system 124 via the implantable coil 122 and provided to the stimulator unit 120. The stimulator unit 120 is configured to utilize the stimulation control signal 136 to generate an electrical stimulation signal (e.g., a current signal) for delivery to the recipient's cochlea via one or more stimulation contacts 126. In this way, the cochlear implant 100 electrically stimulates the recipient's auditory nerve cells, thereby bypassing missing or defective hair cells that normally convert acoustic vibrations into neural activity in a manner that causes the recipient to perceive one or more components of an input audio signal.

[0029] Figure 1A and Figure 1B An arrangement is illustrated in which cochlear implant 100 includes external components. However, it should be appreciated that embodiments of the present invention may be implemented in cochlear implants having alternative arrangements. For example, elements of sound processing unit 112 (e.g., such as processing block 150, power source 102, etc.) may be implanted in a recipient.

[0030] It should also be appreciated that the various components referenced herein (e.g., microphone 108, auxiliary input 129, processing block 150, etc.) can be distributed across more than one prosthesis (e.g., two cochlear implants 100), and indeed, across more than one type of device (e.g., a cochlear implant 100 and a consumer electronic device or remote control for the cochlear implant 100).

[0031] As indicated, the sound processing module 152 is configured to convert the input sound signal into a stimulation control signal 136 for stimulating the first ear of the recipient. The sound processing module 152 processes the sound signal according to various operating parameters indicated by one of a plurality of selectable settings or operating modes or operating modes. The various selectable settings or operating modes can be provided for the program in the form of an executable program or parameter set. The setting can adapt to any of a variety of specific configurations that affect the operation of the cochlear implant. For example, the setting can include different digital signal and sound processing algorithms, processes and / or operating parameters for different algorithms, other types of executable programs (such as system configurations, user interfaces, etc.), or operating parameters for such programs. In some examples, the selectable settings will be stored in the memory of the cochlear implant 100 and are related to different optimal settings for different listening situations or environments (i.e., noisy environments or quiet environments, windy environments, etc.) encountered by the recipient.

[0032] Additionally, because the dynamic range for electrical stimulation is relatively narrow and varies between different recipients and between different stimulation contacts, the programs used in the sound processing module 152 can be individually customized to optimize the perception presented to a specific recipient (i.e., customize the characteristics of the electrical stimulation for the recipient). For example, many speech processing strategies rely on a set of customized stimulation settings that provide a threshold level (T level) and comfort level (C level) for stimulation of each frequency band to a specific recipient. Once these stimulation settings are established, the sound processor can process and convert the received acoustic signals into stimulation data in an optimal manner for use by the stimulator unit 120 when delivering stimulation signals to the recipient.

[0033] Thus, it is apparent that a typical cochlear implant has many parameters that determine the sound processing operation of the device. The personalized programs, commands, data, settings, parameters, instructions, modes and / or other information that define the specific characteristics of the cochlear implant 100 for processing electrical input signals and generating electrical stimulation data therefrom are generally referred to collectively as the recipient's "MAP", or more generally, the "sound processing settings" of the cochlear implant or hearing prosthesis. As further described below, proposed herein are techniques for selecting and / or dynamically (potentially in real time) adjusting the sound processing settings of a hearing prosthesis such as cochlear implant 100 based on a recipient-specific (personalized) "virtual hearing model".

[0034] As used herein, a recipient-specific or personalized "virtual hearing model" is an advanced customized computational model that represents the complete or overall operation of the hearing system of a hearing prosthesis recipient. That is, the recipient-specific virtual hearing model is capable of receiving representations of sounds as input and then estimating how the recipient "perceives" those represented sounds (i.e., what the recipient might hear if those sounds were input into the recipient's actual ears). As further described below, the recipient-specific virtual hearing model can be used in advanced computer simulations to determine, potentially in real time, the optimal sound processing settings (e.g., algorithms, programs, etc.) for a hearing prosthesis, thereby improving the recipient's auditory perception. The recipient-specific virtual hearing model can be used to plan customized rehabilitation exercises for the recipient.

[0035] According to the embodiments presented herein, the recipient-specific virtual hearing model is a holistic model of the recipient's hearing system, meaning that the model covers (takes into account) each of the outer ear system, the middle ear system, and the inner ear system, as well as certain elements of auditory cognition in the auditory cortex and the brain in a bilateral manner (i.e., takes into account both ears of the recipient). The recipient-specific virtual hearing model also takes into account how the hearing prosthesis (e.g., cochlear implant 100) operates and contributes to the recipient's perception. As further described below, the recipient-specific virtual hearing model is generated based on recipient-specific data / information (sometimes referred to herein as recipient-specific psychoacoustic data) and possibly also based on data collected from one or more selected hearing prosthesis recipient populations.

[0036] Figure 2 is a schematic diagram illustrating aspects of the techniques presented herein for generating and / or updating and using a recipient-specific virtual hearing model according to certain embodiments presented herein. More specifically, Figure 2 Four (4) aspects of the technology described herein are shown, namely: data collection 260, recipient-specific virtual hearing model 262, simulated virtual listening situation 264, and simulated hearing profile 266. For ease of description, the following reference Figure 1A and Figure 1B Each of these aspects is further described with respect to the cochlear implant 100 .

[0037] Referring first to data collection 260, this generally refers to a stage or portion of the technique in which recipient-specific (personalized) data is collected for subsequent use in generating or updating a recipient-specific virtual hearing model 262. More specifically, most hearing prosthesis recipients begin their hearing rehabilitation journey with some form of medical evaluation and diagnosis by an audiologist, clinician, physician, or other medical professional (collectively referred to herein as a "clinician"). For individuals diagnosed with mild to severe hearing loss, they may be implanted with a cochlear implant or other hearing aid. Individuals with less severe hearing loss may use hearing aids instead.

[0038] During the preliminary testing phase, as well as the preoperative phase and the postoperative phase (if necessary), several assessments and / or measurements are performed on the recipient. These assessments and / or measurements can take several different forms, such as regular health checks, fitting sessions, image scans, hearing performance tests, etc. Each of these different assessments and / or measurements will generate a variety of recipient-specific "psychoacoustic data". Thus, psychoacoustic data is data related to or characterizing how a specific recipient (possibly using a hearing prosthesis) perceives sound signals (sounds). Each piece of psychoacoustic data is related to the operation / function, structure, shape, etc. of one or more of the recipient's outer ear, middle ear, and / or inner ear, and thus may include physical characteristics of the recipient's ears, physiological measurements, neural responses, hearing perception scores (e.g., speech in noise perception), etc. If the recipient has or uses a hearing prosthesis, the psychoacoustic data can characterize how the recipient perceives sound with the aid of the hearing prosthesis. Recipient-specific psychoacoustic data may include, for example, physical measurements (e.g., optical measurements) or other physical characteristics of one or more ears of the recipient, histograms, audiograms, performance tests, age, disease state, etc.

[0039] According to the embodiments presented herein, data may be collected from a number of different sources in a number of different ways and locations. Figure 2 25 and 26. The recipient-specific psychoacoustic data is represented by arrow 258 in FIG. 25. For example, psychoacoustic data 258 may be collected by psychoacoustic monitoring system 154 in cochlear implant 100, as described elsewhere herein. Alternatively, psychoacoustic data 258 may be collected via a fitting system or other computing device in a clinic, hospital, recipient's home, etc. Figure 2 An example computing device 261 (e.g., a computer) is illustrated that is configured to collect / capture / obtain psychoacoustic data 258 in accordance with certain embodiments presented herein. As noted, it should be appreciated that computing device 261 is merely illustrative, and in practice, psychoacoustic data 258 of a recipient may be collected from a multitude of different devices, locations, etc.

[0040] According to embodiments presented herein, recipient-specific psychoacoustic data 258 is collected (e.g., captured by computing device 261, cochlear implant 100, etc.) and stored at computing system 263, which includes one or more computing devices. Figure 2 As represented by arrow 266 in , one or more computing devices 265 may wirelessly communicate with computing device 261 , cochlear implant 100 , or any other psychoacoustic data collection device.

[0041] In some embodiments, computing system 263 may be a remote (e.g., cloud-based) system, and computing device 265 may be, for example, a server. In other embodiments, computing system 263 may be a local system, and computing device 265 may be, for example, a server, a computer, a mobile computing device, etc. In still other embodiments, computing system 263 may include both a local computing device (e.g., a mobile computing device carried by the recipient) and a remote device (e.g., a server located at a central processing location).

[0042] In some embodiments, cochlear implant 100 may have the ability to communicate directly (e.g., via a communication network, via a short-range wireless connection, etc.) with computing system 263, which may be a local device (e.g., a mobile phone) or a remote (e.g., cloud-based) system. In such embodiments, the request for updated sound processing settings is received directly from cochlear implant 100. However, in other embodiments, cochlear implant 100 may rely on a local device (such as a mobile phone) to communicate with computing system 263.

[0043] As noted, the computing system 263 (e.g., computing device 265) is configured to store the recipient-specific psychoacoustic data 258. The computing system 263 is also configured to use the recipient-specific psychoacoustic data 258 to generate / construct the recipient-specific virtual hearing model 262. As noted above, the recipient-specific virtual hearing model 262 is an advanced customized computing model generated based on the recipient-specific psychoacoustic data 258 to represent how the recipient's hearing system operates in a bilateral manner to convert sound signals into sound perceptions. That is, the recipient-specific virtual hearing model 262 is capable of receiving representations of sounds as input and then estimating how the recipient might perceive those represented sounds (i.e., what the recipient might hear if those sounds were input to the recipient's actual hearing system).

[0044] A recipient-specific virtual hearing model 262 is generated and executed at a computing system 263 by a machine-trained expert system 268 (i.e., the recipient-specific virtual hearing model 262 is generated and / or updated using an expert system that implements a machine learning algorithm and / or an artificial intelligence algorithm). In some embodiments, artificial intelligence (AI) may also be used to generate or update the recipient-specific virtual hearing model 262. Since the computing system 263 is separate from the cochlear implant 100, there may be a large amount of processing power available to execute models based on a large set of parameters. Thus, the processing power of the computing system 263 is sufficient to perform advanced modeling and computer simulation of how a recipient may perceive various sounds using the machine-trained expert system 268 and the recipient-specific virtual hearing model 262.

[0045] With all of the collected recipient-specific psychoacoustic data 258 (e.g., physical characteristics, physiological measurements, neural responses, etc.), the recipient-specific virtual hearing model 262 can be in the form of a holistic model of the recipient's hearing system, which covers all of the outer ear system, the middle ear system, and the inner ear system, as well as the auditory cortex and some elements of auditory cognition in the brain. In some embodiments, in addition to the recipient-specific psychoacoustic data 258, the recipient-specific virtual hearing model 262 can be generated at least in part based on data collected from other recipients. For example, the recipient-specific virtual hearing model 262 can be generated or updated based on data obtained from recipients with similar types of hearing loss, with the same disease and / or pathology, with similar hearing profiles, with similar demographics, etc.

[0046] According to the embodiments presented herein, recipient-specific psychoacoustic data 258 is collected when the recipient begins his / her rehabilitation journey and undergoes implant surgery (i.e., a hearing model is generated based on the recipient's pre-operative data and post-operative data). In cochlear implants, the recipient typically undergoes an initial fitting or "on" session some time period after surgery. This is the first point in time when the cochlear implant is activated and the recipient is able to perceive sound using the implant. Following the initial fitting / on session, the recipient's hearing profile, which is a type of recipient-specific psychoacoustic data 258, is also uploaded to the computing system 263. At this point, the machine-trained expert system 268 generates a recipient-specific virtual hearing model 262.

[0047] The recipient-specific virtual hearing model 262 typically covers the recipient's entire hearing system, which includes the left ear and the right ear. This bilateral formation allows the determination of the effects of various settings on the recipient's ability to localize sounds. The recipient-specific virtual hearing model proposed herein can also cover the following situations: (1) the recipient is a unilateral recipient (e.g., one ear has a hearing aid, bone conduction, cochlear implant or other hearing prosthesis, and the other ear has normal hearing); (2) the recipient is a bimodal recipient (e.g., each ear has a different type of hearing prosthesis, such as one ear has a cochlear implant and the other ear has a hearing aid; or (3) the recipient is a bilateral recipient (e.g., both ears have the same type of hearing prosthesis, such as two cochlear implants), as desired.

[0048] As noted, the recipient-specific virtual hearing model 262 is constructed using the recipient-specific psychoacoustic data 258, which includes data related to how the cochlear implant 100 aids the recipient's perception (i.e., the estimated perception is the perception of the recipient using the cochlear implant). Thus, according to the embodiments presented herein, the recipient-specific virtual hearing model 262 can be used in a computer simulation to select, determine, and / or dynamically adjust the sound processing settings used by the cochlear implant 100 in a manner that improves the recipient's hearing perception performance.

[0049] More specifically, in accordance with the embodiments presented herein, the recipient-specific virtual hearing model 262 can be used to simulate the operation of the recipient's hearing system in a number of different simulated / virtual "listening situations" (i.e., the virtual hearing model is tested / simulated in virtual listening situations). As used herein, a virtual "listening situation" is a set of specific audio situations that a recipient of a hearing prosthesis (e.g., a recipient of a cochlear implant 100) may experience. The set of audio situations that form a virtual listening situation goes beyond simple environmental classifications (e.g., "speech," "noise," etc.). That is, in accordance with the embodiments presented herein, the simulation is performed at a greater granularity and with more detail than simply different "sound environments" (e.g., noise, speech, etc.).

[0050] For example, a virtual listening situation according to embodiments presented herein may take into account recipient-specific factors, such as a person's cognitive load, age / aging, genetic attributes (e.g., related to hearing), emotional state, etc. In some embodiments, a virtual listening situation may take into account external factors (non-verbal cues such as the pitch of the voice (if there is speech), gestures, body language, etc.). In some embodiments, a virtual listening situation may include determining the "type" of the person producing the speech. For example, he / she may listen more intently when he / she is interacting with a work supervisor than when he / she is interacting with a peer. In another example, he / she may listen intently when he / she is in a phone conversation because other cues related to the message / information (e.g., gestures, body language, etc.) are not available. Other listening situations may vary greatly and may depend on how the generated information (simulation results) is used (e.g., listening to something just to relax, compared to listening to something to learn).

[0051] Figure 2An exemplary simulated virtual listening situation 264 is shown. In accordance with the techniques presented herein, a computing system 263 includes a profile simulator 278 that simulates the operation of a recipient-specific virtual hearing model 262 (at several different times) in several different virtual listening situations. By testing the recipient-specific virtual hearing model 262 in one or more simulated virtual listening situations, a machine-trained expert system 268 is able to learn the settings and / or adjustments of the cochlear implant 100 that are likely to induce the best (optimal) hearing perception in the recipient-specific virtual hearing model 262 (and thus for the actual recipient).

[0052] That is, as noted above, the recipient-specific virtual hearing model 262 inherently incorporates the operation of the cochlear implant 100 (and / or another hearing prosthesis, if present). Thus, for each simulation iteration, the recipient-specific virtual hearing model 262 also simulates how the recipient's cochlear implant 100 might perform and affect the recipient's hearing perception if the recipient were to encounter a real listening situation that matches the virtual listening situation. During these simulations, the performance of the cochlear implant 100 is simulated using the virtual / simulated sound processing settings.

[0053] In particular, according to the embodiments presented herein, the machine-trained expert system 268 includes a machine learning algorithm that is trained to learn what settings and / or adjustments to the operation of the cochlear implant 100 will result in the best hearing perception on the recipient-specific virtual hearing model 262 in a given listening situation. The machine learning algorithm of the machine-trained expert system 268 is configured to simulate the operation of the recipient-specific virtual hearing model 262 in a plurality of different listening situations (e.g., shopping malls, restaurants, concert halls, classrooms, offices, special spatial sound source collections, etc.) at different sound processing settings for each listening situation, without interrupting the recipient's hearing or routine. Thus, in each simulation iteration of the recipient-specific virtual hearing model 262, the simulated sound processing settings for the cochlear implant 100 can be changed / adjusted to determine whether the cochlear implant 100, and therefore the recipient-specific virtual hearing model 262, generally provides a better or worse sound perception to the recipient in the relevant virtual listening situation. As described further below, in some examples, the simulated scenario may be based on a real-time environment that the recipient is currently experiencing or a scenario that the recipient has not yet experienced but is predicted to experience.

[0054] Over time, the simulation of the recipient-specific virtual hearing model 262 in different virtual listening situations generates a “simulated hearing profile” 266 for the recipient. As used herein, a simulated hearing profile 266 is a collection of simulation outputs resulting from execution of the recipient-specific virtual hearing model 262. The recipient's simulated hearing profile 266 can be used to generate settings (e.g., MAP) that can be instantiated at (e.g., sent to, installed at, and activated at) a hearing prosthesis of the recipient.

[0055] For example, assume a scenario in which a system (e.g., a smart home system that the recipient is using at home) detects (via a sensor) that the recipient enters a laundry room to operate a washing machine. The system initiates a connection with the cloud to obtain a simulation profile (e.g., an operating profile of the recipient's prosthesis when operating a washing machine in the laundry room), so that the settings determined to be optimal (e.g., threshold (T) levels, comfort (C) levels, etc.) are sent to the prosthesis and used to update the T level and C level at one or more certain frequency bands. In another example scenario, the recipient enters a library (e.g., a quiet environment) and sits at a table to read, where he / she sits closer to a wall or closer to a corner of the room (e.g., reverberation or reverberation). Based on the geographic floor location, the system (e.g., the recipient's smartphone or GPS sensor) sends information to the cloud. The cloud then searches for a simulation profile that has been simulated in a "quiet" environment (because a library may be a "quiet" sound environment) and combined with reverberation suppression information (due to the proximity of one or more walls).

[0056] In certain embodiments, the machine-trained expert system 268 includes a rating subsystem that enables the machine-trained expert system 268 to learn what benchmarks should be achieved when the system is creating and / or running different simulated listening situations using the recipient-specific virtual hearing model 262. These benchmarks can be based on data obtained from a larger population of recipients, for example. For example, data generated based on simulating a virtual hearing model associated with many other recipients can be compared and correlated with the effects of various sound processing settings on other similar recipients in the population being analyzed to better understand how to best change the hearing prosthesis used by a given recipient.

[0057] In general, the recipient-specific virtual hearing model 262 may be updated over time based on the results of the simulation. Additionally, the recipient-specific virtual hearing model 262 may be updated again over time as new psychoacoustic data is received throughout the lifecycle of the recipient, for example, from the psychoacoustic monitoring system 154, the computing system 261, and / or other sources. This new recipient-specific psychoacoustic data 258 may come, for example, from fitting session data, data from a post-operative MRI or other imaging scan, data from regular health checkups, physiological measurements at the beginning of each week / month, background objective measurements taken from the device in operation, device setting changes made by the recipient, device usage logs, etc. If new psychoacoustic data is received, the recipient-specific virtual hearing model 262 may be re-simulated in different listening situations (e.g., periodically), and the simulated hearing profile 266 may be updated accordingly.

[0058] As noted above, the computing system 263 (and more specifically, the machine-trained expert system 268) operates in conjunction with the psychoacoustic monitoring system 154 in the cochlear implant 100. The psychoacoustic monitoring system 154 is typically configured to (e.g., continuously) monitor the recipient's real-time surroundings and provide the machine-trained expert system 268 with an indication of the recipient's "expected listening situations." As used herein, a recipient's "expected listening situations" are a set of specific audio situations that the recipient has encountered or is likely to encounter in the near future. As noted above, similar to virtual listening situations, the set of audio situations that form expected listening situations go beyond simple environmental classifications (e.g., "speech," "noise," etc.) and also include details such as recipient-specific factors, external factors, and the like.

[0059] According to certain embodiments presented herein, the recipient's expected listening situation is determined based on the sound signals received at one or more input devices 134 of the cochlear implant 100. In particular, the cochlear implant 100 (e.g., the psychoacoustic monitoring system 154) is configured to provide "listening situation data" to the computing system 263 (more specifically, the machine-trained expert system 268), which represents the expected listening situation of the hearing prosthesis. The listening situation data may include, for example, the classification of the sound environment (e.g., "noise", "speech", etc.), SNR, average speech level and average noise level, approximate spatial distribution of speech sources and noise sources, dynamic range and spectral characteristics of various components of the auditory scene, other characteristics of the surrounding noise environment (e.g., background music distribution and level), etc.

[0060] In some embodiments, when the characteristics of the recipient's expected listening situation are found to closely match the simulated listening situation, the machine-trained expert system 268 can select the relevant sound processing settings (e.g., processing algorithms, T levels and C levels, noise maskers, activation / deactivation of specific operating features, gain settings on specific electrodes or electrode groups, delay timing alignment between acoustic and electroacoustic paths, power control / consumption / speed of chip processors, etc.) that are determined to be optimal in the matched simulated listening situation, and push them to the recipient's cochlear implant 100 for instantiation and use. In some examples, the machine-trained expert system can even recommend changes outside the prosthesis itself. For example, for a recipient suffering from tinnitus, in addition to applying a noise masker, the expert system can also recommend that the recipient turn on a device / speaker to broadcast or stream soft natural sounds to soothe the tinnitus.

[0061] For example, in accordance with the embodiments presented herein, the psychoacoustic monitoring system 154 in the cochlear implant 100 can be configured to determine that the recipient is experiencing or may experience a certain listening situation. The psychoacoustic monitoring system 154 can provide data representing the expected listening situation to the machine-trained expert system 268, which can then select a set of sound processing settings (determined based on the recipient-specific virtual hearing model 262 analysis in different simulation environments) and provide these selected sound processing settings to the cochlear implant 100 for instantiation and use (e.g., for use by the sound processing module 152, etc.). For example, a machine learning algorithm in the machine-trained expert system 268 can determine the most likely matching virtual listening situation (based on the data received from the cochlear implant 100). Using the matching virtual listening situation, the machine learning algorithm can select the best sound processing settings for the cochlear implant 100 in the expected listening situation. Other examples of such real-time determination of hearing prosthesis settings are described in more detail below.

[0062] Since the recipient-specific virtual hearing model 262 represents the recipient's natural auditory system, from a diagnostic perspective, the clinician can use the model to predict and understand the potential physical capabilities and limitations of the recipient's auditory perception. In practical terms, by viewing the recipient's virtual hearing model 262, which is constantly updated as the recipient ages, deficiencies in the auditory system can potentially be noted well in advance. In this way, diagnostic, rehabilitation and other exercises can be planned accordingly. For example, the impact of a specific rehabilitation strategy or change in sound processing settings can be simulated before application, and its possible impact on hearing performance can be predicted.

[0063] In certain embodiments, the proposed system may have the advantage of maintaining, comparing, and / or even deriving (through the use of artificial intelligence) what the recipient's virtual hearing model 262 may look like in the future (e.g., in one year, five years, ten years, etc.). For people who are prone to ear discomfort (e.g., ear barotrauma) due to pressure changes, the recipient-specific virtual hearing model 262 may take such items into account and simulate environmental changes so that the recipient and / or clinician understand the situation so that customized precautions can be taken if the recipient were to travel or move to a location where he / she would experience such pressure changes. Modeling may be based on similar trends observed by other recipients in the group with similar or similar simulated hearing profiles.

[0064] Another application of the recipient-specific virtual hearing model 262 can be extended and applied to the area of ​​rehabilitation. Most existing rehabilitation programs require both the recipient and his / her clinician to spend a lot of time conducting several trial and error sessions in order to find the best rehabilitation for the recipient. Based on these subjective sessions, the recipient may or may not achieve the best rehabilitation. According to the embodiments presented herein, the recipient-specific virtual hearing model 262 can provide another tool that the clinician (or the entire clinic) can use to identify the recipient's personalized optimal rehabilitation program / plan (e.g., advanced modeling and computer simulation to select customized rehabilitation exercises for the recipient).

[0065] For example, the recipient-specific virtual hearing model 262 can be used to select customized rehabilitation exercises for the recipient. In some embodiments, a large number of models for different individuals in a group can be constructed by supplementing information (such as age, occupation, geographic location, living habits, etc.). These models can also be divided into different user groups, such as a user group showing normal hearing in a specific age range, a user group containing characteristics of tinnitus, a user group showing characteristics or effects after taking certain medications, etc. For all these user groups, an individual model can be used and the characteristics of the model can be compared with those of the user groups.

[0066] In such an example, the model could indicate that due to hearing loss, the individual is likely to experience high anxiety levels, which can lead to poor concentration, inability to remember conversations, or even inability to hear the other party. In this case, the system might select exercises (particularly those focused on sound) to stimulate the person's cognitive functions or select games to help the individual train to remember more, focus better, think faster, etc.

[0067] As described above, according to the embodiments presented herein, the recipient-specific virtual hearing model may be used in real-time to dynamically select / adjust (optimize) the settings of the recipient's hearing prosthesis. Figure 3is a flow chart illustrating an example method 370 for dynamically or real-time use of a recipient-specific virtual hearing model according to embodiments presented herein. Figure 1A and Figure 1B Cochlear implants 100 and Figure 2 Computing Systems 263 Figure 3 However, it should be appreciated that Figure 3 The method can be implemented by using other hearing prostheses, computing systems, etc.

[0068] More specifically, method 370 begins at 372, where computing system 263 receives a request for updated sound processing settings from cochlear implant 100. The request received from cochlear implant 100 includes listening situation data representing an expected listening scenario for the cochlear implant. According to the embodiments presented herein, the "expected" listening scenario may be a current / present listening scenario of cochlear implant 100 or an estimated / predicted future listening scenario.

[0069] In general, the expected listening situation of the cochlear implant 100 is determined by the psychoacoustic monitoring system 154 at least in part based on the sound signals received at the cochlear implant. More specifically, in one example, a system can be provided in which structures (e.g., stations) are defined as basic features or patterns that indicate different listening situations. For example, probability or reference scores may be defined in the system. In practical terms, information related to these features can be collected, monitored and tracked in real time. By analyzing the continuously collected surrounding related features (features related to the listening situation and external features such as the recipient's location), the system will establish a probability reference and / or probability score. When the probability score (based on all relevant combined features) is close to the reference score, the system will know that the recipient is in the expected listening situation.

[0070] For example, in some embodiments, cochlear implant 100 is configured to sample and classify the sound environment (e.g., an environment classifier generates sound classification information / data for determining an intended listening situation), as well as determine other sound parameters, such as average speech level and average noise level, approximate spatial distribution of speech sources and noise sources, dynamic range and spectral characteristics of various components of the auditory scene, other characteristics of the ambient noise environment (e.g., background music distribution and level), etc.

[0071] In some embodiments, cochlear implant 100 automatically generates a request for updated sound processing settings (i.e., without recipient involvement). In response to detecting a number of different conditions (e.g., detecting a new sound environment or specific sound parameters at certain points in time, etc.), cochlear implant 100 can send an automated request for updated sound processing settings. In other embodiments, cochlear implant 100 generates a request for updated sound processing settings in response to one or more user inputs received at cochlear implant 100 or another device (e.g., a mobile phone, remote control, or other mobile computing device in communication with cochlear implant 100).

[0072] return Figure 2 In the example of , in response to a request from cochlear implant 100, at 374, computing system 263 determines selected sound processing settings for use by cochlear implant 100 in the expected listening situation based on virtual hearing model 263 representing the operation of the recipient's hearing system. At 376, the selected sound processing settings are sent to cochlear implant 100 for instantiation and subsequent use by cochlear implant 100. That is, once the selected sound processing settings are received at cochlear implant 100, cochlear implant 100 installs the selected sound processing settings and begins processing sound signals using those settings.

[0073] The computing system 263 can use the virtual hearing model 262 to determine the selected sound processing settings in several different ways. For example, as noted above, the computing system 263 is configured to simulate the virtual hearing model 262 in different virtual listening situations (i.e., predict how the hearing system of the recipient of the cochlear implant 100 perceives the sound in different scenarios with different acoustic properties, different noise levels, etc.). In some embodiments, these simulations can be run in advance to cover a variety of different listening situations. In these embodiments, when a request for updated sound processing settings is received from the cochlear implant 100, the computing system 263 (more specifically, the machine-trained expert system 268) determines which of the pre-executed simulated virtual listening situations matches the expected listening situation most closely. The machine-trained expert system 268 selects the simulated sound processing settings that perform best in the most closely matched simulated virtual listening situation as the selected sound processing settings. That is, the best sound processing settings for each simulated listening situation are stored in the computing system 263. Following receipt of a request for updated sound processing from the cochlear implant 100, the machine-trained expert system 268 matches the expected listening situation with the simulated listening situation and then selects, as the selected sound processing parameters, the sound processing settings that are determined to be optimal under the relevant simulated listening situation (for the recipient-specific virtual hearing model 262, and thus for the recipient).

[0074] The above describes an example in which a machine-trained expert system 268 uses data indicating an expected listening situation to identify similar / analogous simulated listening situations and sound processing parameters associated therewith. In other embodiments, the data indicating the expected listening situation can be used in real time to establish or select a listening environment for simulation, thereby determining selected sound processing parameters. That is, in these embodiments, the machine-trained expert system 268 uses the data included in the request from the cochlear implant 100 to create, establish, select, etc. a simulated listening situation that matches the expected listening situation of the recipient of the cochlear implant. The machine-trained expert system 268 then runs a real-time simulation for the recipient-specific virtual hearing model 262 and therefore for the recipient in the selected simulated listening situation to determine the optimal sound processing settings for the cochlear implant 100 in the expected listening situation. As noted, after the analysis is completed, the selected sound processing settings are provided to the cochlear implant 100.

[0075] As noted above, the computing system 263 may use the recipient-specific virtual hearing model 262 in a number of different ways to determine selected sound processing settings for the cochlear implant 100. As such, it should be appreciated that the above two techniques for determining selected sound processing settings are merely illustrative, and that the embodiments presented herein may use other techniques to select sound processing settings for the cochlear implant 100 or another hearing prosthesis.

[0076] To facilitate further understanding of the presented technology, several examples of real-time applicability of recipient-specific virtual hearing models to the selection of sound processing settings are provided below. For ease of illustration, reference is again made to Figure 1A and Figure 1B Cochlear implants 100 and Figure 2 These examples are described with reference to the computing system 263 of FIG. However, it should be understood that similar techniques may be implemented with other hearing prostheses, computing systems, etc.

[0077] In one example, a recipient of cochlear implant 100 attends a cocktail party (i.e., the intended listening environment). Cochlear implant 100 is configured to sample and classify the sound environment (e.g., the environment classifier determines that the recipient is in a "speech+noise" environment), as well as determine other sound parameters, such as average speech level and average noise level, approximate spatial distribution of speech sources and noise sources, dynamic range and spectral characteristics of various components of the auditory scene, other characteristics of the ambient noise environment (e.g., background music distribution and level), etc. This information, sometimes referred to herein as listening situation data, represents the intended listening situation (i.e., cocktail party) for cochlear implant 100. As noted, the listening situation data can be transmitted to a machine-trained expert system 268 in computing system 263. Machine-trained expert system 268 is configured to then run a simulation of the recipient's hearing perception under the intended listening situation (i.e., cocktail party) and determine the optimal sound processing settings for that particular listening situation. These settings are then immediately communicated to the recipient in real time to adjust the cochlear implant 100 in an optimal manner.

[0078] According to embodiments herein, once a recipient has encountered an expected listening situation, the relevant sound processing settings selected for use therein by the machine-trained expert system 268 may be added to a database associated with the recipient. Thus, the next time the recipient is in this or a similar listening situation, these settings may be reapplied without performing a simulation and / or performing only a validation process. Further, such expected listening situations and relevant sound processing settings may be added to a large global database to help fine-tune the adaptation process and / or be used by other recipients when they are in similar environments.

[0079] In another illustrative example, a recipient of cochlear implant 100 enters a concert hall to listen to a performance. A concert hall is a particularly unusual listening environment for the recipient in that it may include significant reverberation, significant harmonics, and information and noise sources in various directions. According to the embodiments presented herein, psychoacoustic monitoring system 154 in cochlear implant 100 is configured to sample the auditory environment to collect listening situation data. In some examples, the listening situation data may be supplemented with auxiliary environmental data from alternative sources (such as a database of sound locations) obtained from one or more auxiliary sources (e.g., other computing systems) at computing system 263, information about the performance and / or a specific venue, occupancy information, etc. Similarly, using the listening situation data and in some examples using the auxiliary environmental data, the machine-trained expert system 268 is configured to simulate the recipient's hearing perception in the expected listening situation (i.e., a concert hall) and determine the optimal sound processing settings for this particular listening situation. These settings are then immediately communicated to the recipient in real time to adjust the cochlear implant 100 so as to improve the recipient's perception and enjoyment of the concert.

[0080] In another example, the recipient may be taking a jungle or nature walk, where the auditory environment experienced by the recipient may be very different from the environment experienced by the recipient in daily life (e.g., there are fewer auditory reflections and background 'noise' is not actually noise and forms a certain part of the experience in the jungle). According to the embodiments presented herein, the psychoacoustic monitoring system 154 in the cochlear implant 100 is configured to sample the auditory environment to collect listening situation data indicating that the recipient is in this particular listening situation. Likewise, using the listening situation data, the machine-trained expert system 268 is configured to run a simulation of the recipient's hearing perception under the expected listening situation (i.e., nature walk) and determine the optimal sound processing settings for the particular listening situation. These settings are then immediately communicated to the recipient in real time to adjust the cochlear implant 100 so as to improve the recipient's perception and enjoyment while on the nature walk.

[0081] The above specific examples are merely illustrative to show that the techniques presented herein can be used in any of several different situations to determine updated sound processing settings for a cochlear implant 100 or another hearing prosthesis. The above examples also illustrate that the techniques presented herein can function somewhat like advanced classifiers, where sound processing can be adjusted based on a specific listening situation, with adaptability and granularity far greater than that of simply using an on-board environmental classifier. Thus, building on the previous examples, in some embodiments, a machine-trained expert system 268 can continuously run simulations based on the listening situation experienced by the recipient using a recipient-specific virtual hearing model 262. The machine-trained expert system 268 can then adjust the psychoacoustic monitoring system 154 and / or environmental classifier in the cochlear implant 100 of a specific recipient and push this to the recipient's sound processor. This can occur periodically or even continuously in real time based on the listening situation experienced by the recipient.

[0082] In other embodiments, the psychoacoustic monitoring system 154 in the cochlear implant 100 may itself include a machine learning system that executes a recipient-specific virtual hearing model (e.g., implemented on an embedded AI chip). For example, a remote machine-trained expert system 268 may push / install a copy of the recipient-specific virtual hearing model 262 at the cochlear implant 100. The cochlear implant 100 may be a complete virtual hearing model 262 or a modified version of the virtual hearing model that is optimized to run in a low-power environment. In these embodiments, a copy of the recipient-specific virtual hearing model 262 may be used in a manner similar to that described above with reference to the recipient-specific virtual hearing model 262 running in the computing system 263 to select sound processing settings for the cochlear implant 100 (e.g., running a simulation in a virtual listening situation to determine the optimal settings for the expected listening situation via a machine learning algorithm). Over time, the machine-trained expert system 268 can push changes made to the recipient-specific virtual hearing model 262 to the psychoacoustic monitoring system 154 to control and change how the embedded machine learning system learns and adjusts. In this way, the embedded machine learning system is regularly trained and updated by the more powerful cloud-based machine learning system.

[0083] In other words, in these examples, low-power machine learning and / or artificial intelligence algorithms are embedded in cochlear implant 100 (e.g., as part of psychoacoustic monitoring system 154). A more powerful machine-trained expert system 268 in computing system 263 (e.g., on a mobile phone, computer, server, etc.) is used to train the recipient's embedded system based on the recipient's listening situation and one or more simulated hearing profiles. On a regular basis, the embedded system, including a copy of the recipient's specific virtual hearing model 262, can be updated by the machine-trained expert system 268, thereby providing improved performance without requiring the processing level required for cochlear implant 100 to train the more powerful machine-trained expert system 268 in computing system 263.

[0084] As noted above, aspects of the techniques presented herein may be implemented on a local or remote computing system comprising one or more computing devices. Figure 4 is a functional block diagram of an example computing system 463 configured to implement various aspects of the techniques presented herein. Figure 4 In the example of , computing system 463 includes a single computing device 465. It should be appreciated that this implementation is merely illustrative, and that computing systems in accordance with embodiments presented herein may be formed by one or more different computing devices. Thus, it should be appreciated that in some embodiments, Figure 4 The components shown may be implemented across multiple computing devices.

[0085] exist Figure 4 , computing device 465 includes a plurality of interfaces / ports 489(1) to 489(N), a memory 490, and one or more processors 491. Figure 4 Not shown, but computing device 465 may also include a user interface, display screen, etc., depending on the type of computing device used to implement the techniques presented herein. Interfaces 489(1)-489(N) may include, for example, any combination of a network port (e.g., an Ethernet port), a wireless network interface, a Universal Serial Bus (USB) port, an Institute of Electrical and Electronics Engineers (IEEE) 1394 interface, a PS / 2 port, etc. Interfaces 289(1)-289(N) may be configured to transmit / receive signals via a wired connection or a wireless connection (e.g., telemetry, Bluetooth, etc.).

[0086] The memory 490 includes the machine learning logic 468, the profile simulator logic 478, and one or more simulated hearing profiles 466. The memory 490 may include a read-only memory (ROM), a random access memory (RAM), a disk storage media device, an optical storage media device, a flash memory device, an electrical memory storage device, an optical memory storage device, or other physical / tangible memory storage devices. The one or more processors 491 are, for example, microprocessors or microcontrollers that execute instructions of the machine learning logic 468 and the profile simulator logic 478. Thus, in general, the memory 490 may include one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (by one or more processors 491), the computer-executable instructions are operable to perform all or part of the proposed techniques. For example, the machine learning logic 468 may be executed by one or more processors 491 to, for example, perform the techniques described above with reference to a machine-trained expert system such as the machine-trained expert system 268. Profile emulator logic 478 may be executed by one or more processors 491 to, for example, perform the techniques described above with reference to a profile emulator such as profile emulator 278 .

[0087] For ease of description only, the techniques presented herein are primarily described herein with reference to an illustrative hearing / hearing prosthesis (i.e., a cochlear implant). However, it should be appreciated that the techniques presented herein may also be used with a variety of other types of devices, including other hearing prostheses. For example, the techniques presented herein may be implemented in, for example, acoustic hearing aids, auditory brainstem stimulators, bone conduction devices, middle ear hearing prostheses, direct acoustic stimulators, bimodal hearing prostheses, bilateral hearing prostheses, and the like. Specifically, Figure 5is a functional block diagram of one exemplary arrangement of a bone conduction device 500 configured to implement the embodiments presented herein.

[0088] Bone conduction device 500 includes microphone array 540 , electronics module 512 , transducer 520 , user interface 524 , and power source 527 .

[0089] The microphone array 540 includes a first microphone 508(1) and a second microphone 508(2) configured to convert received sound signals (sound) into microphone signals 544(1) and 544(2). The microphone signals 544(1) and 544(2) are provided to the electronic module 512. In general, the electronic module 512 is configured to convert the microphone signals 544(1) and 544(2) into one or more transducer drive signals 518, which activate the transducer 520. More specifically, the electronic module 512 includes, among other components, at least one processor 550, a memory 532, and a transducer drive component 534.

[0090] The memory 532 includes sound processing logic 552 and psychoacoustic monitoring logic 554. The memory 532 may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical memory storage devices, optical memory storage devices, or other physical / tangible memory storage devices. The at least one processor 550 is, for example, a microprocessor or microcontroller that executes instructions of the sound processing logic 552 and psychoacoustic monitoring logic 554. Thus, in general, the memory 532 may include one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (by the at least one processor 550), the computer-executable instructions are operable to perform various aspects of the techniques presented herein.

[0091] Transducer 520 illustrates an example of a stimulator unit that receives one or more transducer drive signals 518 and generates stimulation (vibrations) for delivery to the recipient's skull via a cutaneous or transcutaneous anchor system (not shown) coupled to the bone conduction device 500. The delivered vibrations result in movement of cochlear fluid in the contralateral functioning ear of the recipient, thereby activating the auditory hair cells in the functioning ear.

[0092] Similar to cochlear implants, bone conduction devices and / or other hearing prostheses operate according to several sound processing settings. Figure 5In an embodiment, a virtual hearing model of a recipient of a bone conduction device 500 in a computing system (e.g., system 263, system 463, etc.) may be generated based on recipient-specific psychoacoustic data captured from the bone conduction device 500 or from another device (e.g., via execution of psychoacoustic monitoring logic 554 by at least one processor 550).

[0093] Similar to the above-described embodiments, simulations may be performed using the virtual hearing model in a number of different virtual listening situations to generate a simulated hearing profile for the recipient of the bone conduction device 500. Subsequently, the psychoacoustic monitoring logic 554 may, when executed by the at least one processor 550, generate a request to the computing system for an updated set of sound processing settings for use by the bone conduction device 500 in the expected listening situation. The computing system may then determine, based on the virtual hearing model representing the operation of the recipient's hearing system, selected sound processing settings for use by the bone conduction device in the expected listening situation; and send the selected sound processing settings to the bone conduction device 500 for instantiation.

[0094] Figure 6 692 is a flow chart of a method 692 according to an embodiment presented herein. The method 692 begins at 693, where a virtual hearing model is generated based on recipient-specific psychoacoustic data. Once generated, the virtual hearing model is an overall model of the hearing system of the hearing prosthesis recipient, and the virtual hearing model bilaterally considers the operation of each system in the recipient's outer ear system, middle ear system, and inner ear system, along with the recipient's auditory cortex and brain's hearing cognition, and how the hearing prosthesis operates and contributes to the recipient's perception. At 694, the virtual hearing model is used to determine a selected sound processing setting for use by the hearing prosthesis. At 695, the selected sound processing setting is instantiated at the hearing prosthesis (e.g., sent to the hearing prosthesis, installed at the hearing prosthesis, and activated at the hearing prosthesis).

[0095] It should be appreciated that the embodiments described above are not mutually exclusive and that the various embodiments may be combined in various ways and arrangements.

[0096] Since the specific preferred embodiments disclosed herein are intended to serve as illustrations and not limitations of several aspects of the present invention, the scope of the invention described and claimed herein is not limited by these embodiments. Any equivalent embodiments are intended to be within the scope of the present invention. In fact, from the foregoing description, various modifications of the present invention except those shown and described herein will become apparent to those skilled in the art. Such modifications are also intended to fall within the scope of the appended claims.

Claims

1. A method for configuring sound processing settings, comprising: At the computing system: receiving a request for updated sound processing settings from a hearing prosthesis configured to be worn by a recipient, wherein the request includes listening situation data representing an expected listening situation for the hearing prosthesis; determining, based on the recipient-specific virtual hearing model, selected sound processing settings for use with the hearing prosthesis in the intended listening situation; and sending the selected sound processing settings to the hearing prosthesis, The recipient-specific virtual hearing model is capable of receiving representations of sounds as input and then estimating how the recipient perceives those represented sounds.

2. The method of claim 1 , wherein the recipient-specific virtual hearing model is a holistic model of the recipient's auditory system that takes into account the operation of each of the recipient's outer ear system, middle ear system, and inner ear system in a bilateral manner, together with the recipient's auditory cognition in the auditory cortex and brain, and how the hearing prosthesis operates and contributes to the recipient's perception.

3. The method of claim 1 , wherein determining the selected sound processing settings for use by the hearing prosthesis in the expected listening situation comprises: iteratively performing simulation on the recipient-specific virtual hearing model in at least one virtual listening situation matching the expected listening situation; adjusting the simulated operation of the hearing prosthesis in the recipient-specific virtual hearing model at each iteration to account for different sound processing settings of the hearing prosthesis; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

4. The method according to claim 3, further comprising: prior to receiving the request from the hearing prosthesis, iteratively performing simulation on the recipient-specific virtual hearing model in a plurality of different virtual listening situations, wherein at least one virtual listening situation of the plurality of different virtual listening situations includes the at least one virtual listening situation that matches the expected listening situation; upon receiving the request from the hearing prosthesis, identifying the at least one virtual listening situation that matches the expected listening situation; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings previously estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

5. The method according to claim 3, further comprising: after receiving the request from the hearing prosthesis, generating the at least one virtual listening situation matching the expected listening situation based on the listening situation data; iteratively performing simulation on the recipient-specific virtual hearing model in the at least one virtual listening situation matching the expected listening situation; adjusting the simulated operation of the hearing prosthesis in the recipient-specific virtual hearing model at each iteration to account for different sound processing settings of the hearing prosthesis; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

6. The method of claim 3, wherein adjusting the simulation operation of the hearing prosthesis in the recipient-specific virtual hearing model at each iteration comprises: The simulated operation of the hearing prosthesis is adjusted in the recipient-specific virtual hearing model using a machine learning algorithm.

7. The method of claim 3, wherein adjusting the simulation operation of the hearing prosthesis in the recipient-specific virtual hearing model at each iteration comprises: The simulated operation of the hearing prosthesis is adjusted in the recipient-specific virtual hearing model using an artificial intelligence algorithm.

8. The method according to claim 1, further comprising: The recipient-specific virtual hearing model is generated based on recipient-specific psychoacoustic data.

9. The method according to claim 8, further comprising: At least a portion of the recipient-specific psychoacoustic data is received from a psychoacoustic monitoring system in the hearing prosthesis.

10. The method according to claim 8, further comprising: receiving additional recipient-specific psychoacoustic data over time; as well as The recipient-specific virtual hearing model is dynamically updated based on the additional recipient-specific psychoacoustic data.

11. The method according to claim 8, further comprising: The recipient-specific virtual hearing model is generated based on psychoacoustic data collected from one or more selected groups of hearing prosthesis recipients.

12. The method of claim 1, wherein the computing system is a mobile computing device.

13. A method for configuring sound processing settings, comprising: generating a virtual hearing model based on the recipient-specific psychoacoustic data, wherein the virtual hearing model is a holistic model of the auditory system of a recipient of the hearing prosthesis, and wherein the virtual hearing model takes into account the operation of each of the recipient's outer ear system, middle ear system, and inner ear system in a bilateral manner, along with hearing cognition in the recipient's auditory cortex and brain, and how the hearing prosthesis operates and contributes to the recipient's perception; using the virtual hearing model to determine selected sound processing settings for use with the hearing prosthesis; as well as The selected sound processing settings are instantiated at the hearing prosthesis.

14. The method of claim 13, wherein using the virtual hearing model to determine selected sound processing settings for use with the hearing prosthesis comprises: receiving a request for updated sound processing settings from the hearing prosthesis, wherein the request includes listening situation data representing an expected listening situation for the hearing prosthesis; iteratively performing simulation on the virtual hearing model in at least one virtual listening situation matching the expected listening situation; adjusting the simulated operation of the hearing prosthesis in the recipient-specific virtual hearing model at each iteration to account for different sound processing settings of the hearing prosthesis; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

15. The method according to claim 14, further comprising: prior to receiving the request from the hearing prosthesis, iteratively performing simulation on the virtual hearing model in a plurality of different virtual listening situations, wherein at least one virtual listening situation of the plurality of different virtual listening situations includes the at least one virtual listening situation that matches the expected listening situation; upon receiving the request from the hearing prosthesis, identifying the at least one virtual listening situation that matches the expected listening situation; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings previously estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

16. The method according to claim 14, further comprising: after receiving the request from the hearing prosthesis, generating the at least one virtual listening situation matching the expected listening situation based on the listening situation data; iteratively performing simulation on the virtual hearing model in the at least one virtual listening situation matching the expected listening situation; adjusting the simulated operation of the hearing prosthesis in the virtual hearing model at each iteration to account for different sound processing settings of the hearing prosthesis; as well as A set of sound processing settings is determined as the selected sound processing settings for use with the hearing prosthesis, the set of sound processing settings estimated to provide a selected hearing perception to the recipient in the at least one virtual listening situation matching the expected listening situation.

17. The method according to claim 13, further comprising: At least a portion of the recipient-specific psychoacoustic data is received from a psychoacoustic monitoring system in the hearing prosthesis.

18. The method according to claim 13, further comprising: receiving additional recipient-specific psychoacoustic data over time; as well as The virtual hearing model is dynamically updated based on the additional recipient-specific psychoacoustic data.

19. The method according to claim 13, further comprising: The recipient-specific virtual hearing model is generated based on psychoacoustic data collected from one or more selected groups of hearing prosthesis recipients.

20. The method of claim 13, wherein instantiating the selected sound processing setting at the hearing prosthesis comprises: The selected sound processing settings are transmitted to the hearing prosthesis.

21. The method of claim 13, further comprising: installing a copy of the virtual hearing model at the hearing prosthesis; as well as At the hearing prosthesis, the copy of the virtual hearing model is used to determine selected sound processing settings for use with the hearing prosthesis.

Images