Stimulus control

Abstract

This disclosure relates to stimulation control. This document presents techniques for controlling stimulation provided by an implantable device. Stimulation control can be performed at an external device and can utilize machine learning (e.g., artificial intelligence (AI)). The techniques provide a cochlear implant stimulation strategy that utilizes computational models of the healthy hearing system and the implanted hearing system to more closely simulate natural acoustic hearing.

Description

[0001] This application is a divisional application of Chinese invention patent application with international filing date of April 5, 2024, national application number 202480024878.0, and invention title "Stimulus Control". Technical Field

[0002] Various aspects of the present invention generally relate to controlling stimuli delivered by electronic devices. Background Technology

[0003] In recent decades, medical devices have provided a wide range of therapeutic benefits to recipients. Medical devices can include internal or implantable components / devices, external or wearable components / devices, or combinations thereof (e.g., devices having an external component that communicates with the implantable component). Medical devices include, for example, conventional hearing aids, and partially or fully implantable hearing prostheses (e.g., bone conduction devices, mechanical stimulators, cochlear implants). wait Pacemakers, defibrillators, functional electrical stimulation devices, fully implantable visual prostheses, vagus nerve stimulators, spinal cord stimulators and other medical devices have been successful for many years in performing life-saving and / or lifestyle improvement functions and / or recipient monitoring.

[0004] Over the years, the types of medical devices and the range of functions they perform have increased. For example, many medical devices, sometimes referred to as “implantable medical devices,” now typically include one or more instruments, devices, sensors, processors, controllers, or other functional mechanical or electrical components that are permanently or temporarily implanted into a recipient’s body. These functional devices are typically used to diagnose, prevent, monitor, treat, or manage diseases / injuries or their symptoms, or to study, replace, or modify anatomical structures or physiological processes. Many of these functional devices utilize power and / or data received from an external device that is part of or operates in conjunction with the implantable component. Summary of the Invention

[0005] In one aspect, a method is provided. The method includes: receiving a signal associated with a physiological function at an implantable medical device system; determining, based on the signal, information about a stimulation signal for stimulating the physiological function by a machine learning model, wherein the machine learning model is trained based on modeling physiological effects from the stimulation; and controlling the stimulation of a recipient of the implantable medical device system based on the determined information.

[0006] In another aspect, one or more non-transitory computer-readable storage media are provided, including instructions. When executed by one or more processors, the instructions cause the one or more processors to: receive signals associated with physiological function at an implantable medical device system; determine information about stimulation signals for stimulating the physiological function based on the signals by a machine learning model, wherein the machine learning model is trained based on modeling the physiological effects from the stimulation; and control the stimulation of a recipient of the implantable medical device system based on the determined information.

[0007] In another aspect, an implantable medical device system is provided. The implantable medical device system includes: a memory for storing data; and one or more processors, wherein the one or more processors are configured to: receive signals associated with physiological functions; determine, based on the signals, information for stimulating the physiological functions by a machine learning model, wherein the machine learning model is trained based on modeling the physiological effects from the stimulation; and control the stimulation of a recipient of the implantable medical device system based on the determined information.

[0008] In another aspect, an alternative method is provided. This includes: a machine learning model using at least one processor determining information about a stimulus signal for stimulating the physiological function based on signals associated with that function; modeling a physiological effect from the stimulus signal via the at least one processor; and updating the machine learning model via the at least one processor based on a difference between the modeled physiological effect and a reference physiological effect representing normal physiological function. Attached Figure Description

[0009] Embodiments of the present invention are described herein in conjunction with the accompanying drawings, in which: Figure 1A This is a schematic diagram illustrating various aspects of a cochlear implant system that can be used to implement the techniques presented herein; Figure 1B It is wearing Figure 1A A side view of the recipient of the sound processing unit of the cochlear implant system; Figure 1C yes Figure 1A A schematic diagram of the components of a cochlear implant system; Figure 1D yes Figure 1A A block diagram of a cochlear implant system; Figure 1E This is a schematic diagram illustrating a computing device that can be used to implement various aspects of the techniques presented herein; Figure 2This is a functional block diagram illustrating an exemplary audio signal processing path for a cochlear implant system that can utilize various aspects of the techniques presented herein. Figure 3 This is a functional block diagram illustrating a method for training a hearing machine learning (ML) model according to certain embodiments; Figure 4A An exemplary spectrogram for rising complex tones is shown; Figure 4B Showing targets Figure 4A An exemplary neurograph of the voltage of inner hair cells in rising complex tones; Figure 4C Showing targets Figure 4A An exemplary electroneurogram of the fine structure of the auditory nerve in rising complex tones; Figure 5 This is a functional block diagram illustrating a method for training a sound processor machine learning (ML) model to control stimuli according to certain embodiments; Figure 6 This is a schematic diagram of an exemplary neural network that can be used to implement various aspects of the techniques presented in this article; Figure 7 This is a functional block diagram illustrating a method for training a machine learning (ML) model for a sound processor using feature extraction applied to an input audio signal, according to certain embodiments. Figure 8 This is a functional block diagram illustrating a method for training a machine learning (ML) model for a sound processor using another feature extraction applied to an input audio signal, according to certain embodiments; Figure 9 This is a functional block diagram illustrating a method for training a machine learning (ML) model for a sound processor using a neurograph of an input audio signal, according to certain embodiments. Figure 10 This is a functional block diagram illustrating a method for training a sound processor machine learning (ML) model to reduce noise according to certain embodiments; Figure 11A This is a functional block diagram illustrating a method for training a sound processor machine learning (ML) model using a neuroelectrograph of an input audio signal for focusing on multipolar stimuli, according to certain embodiments. Figure 11B This is a functional block diagram illustrating another method, according to certain embodiments, for training a sound processor machine learning (ML) model using a neuroelectrograph of an input audio signal to focus on multipolar stimuli. Figure 12 This is a flowchart illustrating an exemplary process of controlling stimuli according to certain embodiments; and Figure 13This is a flowchart illustrating an exemplary process for training a machine learning model to control stimuli, according to certain embodiments. Detailed Implementation

[0010] This article presents techniques for controlling stimulation provided by electronic devices, such as implantable medical devices. Stimulation control can be performed at an external device and can utilize machine learning (e.g., artificial intelligence (AI)). In some aspects, the techniques provide a stimulation strategy that utilizes computational models of healthy and implanted hearing systems to more closely simulate natural acoustic hearing.

[0011] The exemplary techniques presented in this paper minimize the differences between normal hearing and electro-hearing computational models to deliver sound information more consistent with normal hearing and improve outcomes for hearing device users.

[0012] It should be understood that there are several different types of medical devices in which / utilizing them can implement the techniques presented herein. For ease of description only, the techniques presented herein are described primarily with reference to a specific medical device in the form of a cochlear implant. However, it should be understood that the techniques presented herein can also be implemented, in part or in whole, by any of several different types of medical devices, particularly any other type of device for delivering electrical stimulation signals to a recipient. For example, the techniques can be implemented in devices that deliver electrical stimulation to the auditory nerve, middle ear, vestibular system, retina and / or brain, as well as other areas of the body. As used herein, the term “hearing device” should be interpreted broadly as any device that delivers sound signals to a user in any form, including acoustic stimulation, mechanical stimulation, electrical stimulation, optical stimulation, etc. In this way, hearing devices can be devices for use by people with hearing loss (e.g., hearing aids, middle ear prostheses, bone conduction devices, direct acoustic stimulators, electroacoustic hearing prostheses, auditory brainstem stimulators, bimodal hearing prostheses, bilateral hearing prostheses, dedicated tinnitus treatment devices, tinnitus treatment device systems, combinations or variations thereof) or devices for use by people with normal hearing (e.g., consumer devices that provide audio streaming, consumer headphones, headphones and other listening devices).

[0013] Figure 1A-1E An exemplary cochlear implant system 102 is shown that can be used to implement aspects of the techniques presented herein. The cochlear implant system 102 includes an external component 104 and an implantable component 112. Figure 1A-1E In the examples, the implantable component is sometimes referred to as a "cochlear implant". Figure 1A The image shows a cochlear implant 112 implanted in the user's head 154, while... Figure 1B This is a schematic diagram of an external component 104 worn on the user's head 154. Figure 1C This is another schematic diagram of the cochlear implant system 102, and Figure 1D Further details of the cochlear implant system 102 are shown. For ease of description, they will generally be described together. Figure 1A-1E .

[0014] The cochlear implant system 102 includes an external component 104 configured to be directly or indirectly attached to a user's body, and an implantable component (or implant) 112 configured to be implanted into the user. Figure 1A-1E In the example, external component 104 includes a sound processing unit 106, while cochlear implant 112 includes an implantable coil 114, an implant body 134, and an elongated stimulation component 116 configured for implantation in a user's cochlea.

[0015] exist Figure 1A-1E In the example, sound processing unit 106 is an off-ear (OTE) sound processing unit, sometimes referred to herein as an OTE component, configured to send data and power to implantable component 112. Generally, an OTE sound processing unit is a component having a generally cylindrical housing 111 and configured to be magnetically coupled to a user's head (e.g., including an integrated external magnet 150 configured to be magnetically coupled to an implantable magnet 152 in implantable component 112). OTE sound processing unit 106 also includes an integrated (head component) coil 108 configured to be inductively coupled to implantable coil 114.

[0016] It should be understood that the OTE sound processing unit 106 is merely an illustration of an external device that can operate in conjunction with the implantable component 112. For example, in alternative examples, the external component may include a behind-the-ear (BTE) sound processing unit or a micro-BTE sound processing unit and a separate external coil assembly. Generally, the BTE sound processing unit includes a housing shaped to be worn on the user's outer ear and connected via a cable to the separate external coil assembly, wherein the external coil assembly is configured to be magnetically and inductively coupled to the implantable coil 114. It should also be understood that alternative external components may be located in the user's ear canal and worn on the body. wait .

[0017] As described above, the cochlear implant system 102 includes a sound processing unit 106 and a cochlear implant 112. However, as further described below, the cochlear implant 112 can operate independently of the sound processing unit 106 for at least a period of time to stimulate the user. For example, the cochlear implant 112 can operate in a first general mode (sometimes referred to as "external hearing mode"), in which the sound processing unit 106 captures a sound signal, which is then used as the basis for delivering a stimulation signal to the user. The cochlear implant 112 can also operate in a second general mode (sometimes referred to as "invisible hearing" mode), in which the sound processing unit 106 cannot provide a sound signal to the cochlear implant 112 (e.g., the sound processing unit 106 is absent, the sound processing unit 106 is powered off, or the sound processing unit 106 malfunctions). wait Therefore, in invisible hearing mode, the cochlear implant 112 captures sound signals itself via an implantable sound sensor and then uses these sound signals as the basis for delivering stimulation signals to the user. In some examples, the external device is still able to deliver power to the implant in invisible hearing mode. In such examples, the external device can implement the techniques presented herein to calculate an optimal power level using information from the cochlear implant 112 retrieved or stored on the external device (e.g., stimulation parameters). Further details regarding the operation of the cochlear implant 112 in external hearing mode are provided below, followed by details regarding the operation of the cochlear implant 112 in invisible hearing mode. It should be understood that the references to external hearing mode and invisible hearing mode are merely illustrative, and the cochlear implant 112 can also operate in alternating modes.

[0018] exist Figure 1A and 1C In the diagram, the cochlear implant system 102 is shown with an external computing device 110 configured to implement various aspects of the presented technology. The computing device 110 (in...) Figure 1E (As shown in more detail below) includes, for example, personal computers, server computers, handheld devices, laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics (e.g., smartphones), network PCs, minicomputers, mainframe computers, tablet computers, remote control units, distributed computing environments including any of the above systems or devices, and so on. Computing device 110 may be a single virtual or physical device operating in a networked environment via a communication link to one or more remote devices, such as an implantable medical device or an implantable medical device system.

[0019] In its most basic configuration, the computing device 110 includes at least one processing unit 183 and a memory 184. The processing unit 183 includes one or more hardware or software processors (e.g., a central processing unit) that can receive and execute instructions. The processing unit 183 can communicate with and control the performance of other components of the computing device 110.

[0020] Memory 184 is one or more computer-readable storage media based on software or hardware, operable to store information accessible by processing unit 183. Among other things, memory 184 may store instructions and other data that can be executed by processing unit 183 to implement an application or to enable the operations described herein. Memory 184 may be volatile memory (e.g., RAM), non-volatile memory (e.g., ROM), or a combination thereof. Memory 184 may include temporary or non-temporary memory. Memory 184 may also include one or more removable or non-removable storage devices. In examples, memory 184 may include RAM, ROM, EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, optical disc storage, magnetic storage, solid-state storage, or any other memory medium that can be used to store information for later access. In examples, memory 184 includes modulated data signals (e.g., signals whose one or more characteristics are set or changed in a manner that encodes information in the signal), such as carrier waves or other transmission mechanisms, and includes any information delivery medium. By way of example and not limitation, memory 184 may include wired media (e.g., a wired network or direct wired connection), and wireless media (e.g., acoustic, RF, infrared, and other wireless media) or combinations thereof. In some embodiments, memory 184 includes stimulus control logic 185 (having a stimulus generator model 192), which, when executed, enables processing unit 183 to perform aspects of the presented technology.

[0021] In the illustrated example, computing device 110 also includes a network adapter 186, one or more input devices 187, and one or more output devices 188. Computing device 110 may include other components such as a system bus, component interfaces, a graphics system, a power supply (e.g., a battery), and other components.

[0022] Network adapter 186 is a component of computing device 110 that provides network access (e.g., access to at least one network 189). Network adapter 186 can provide wired or wireless network access and can support one or more of various communication technologies and protocols, such as Ethernet, cellular, Bluetooth, near-field communication, and RF (radio frequency), etc. Network adapter 186 may include one or more antennas and associated components configured to communicate wirelessly according to one or more wireless communication technologies and protocols. In some examples, one or more antennas may be shared with charging coil 121 and / or external coil 108.

[0023] One or more input devices 187 are means by which the computing device 110 receives input from a user. One or more input devices 187 may include physically actuated user interface elements (e.g., buttons, switches, or dial pads), touchscreens, keyboards, mice, pens, and voice input devices, as well as other input devices.

[0024] One or more output devices 188 are means through which the computing device 110 can provide output to a user. Output devices 188 may include, for example, a display 190 and one or more speakers 191, as well as other output devices.

[0025] It should be understood that Figure 1E The arrangement of the computing device or system 110 shown is merely illustrative, and the aspects of the techniques presented herein can be implemented in many different types of systems / devices. For example, the computing device 110 may be a laptop computer, a tablet computer, a mobile phone, or a surgical system. wait .

[0026] The OTE sound processing unit 106 includes one or more input devices configured to receive input signals (e.g., sound or data signals). The one or more input devices include one or more sound input devices 118 (e.g., one or more external microphones, audio input ports, pickup coils). wait One or more auxiliary input devices 128 (e.g., audio ports, such as Direct Audio Input (DAI), data ports, such as Universal Serial Bus (USB) ports, cable ports). wait The input devices include a wireless transmitter / receiver (transceiver) 120 (e.g., for communicating with an external computing device 110). However, it should be understood that one or more input devices may include additional types of input devices and / or fewer input devices (e.g., the short-range wireless transceiver 120 and / or one or more auxiliary input devices 128 may be omitted).

[0027] The OTE sound processing unit 106 also includes an external coil 108, a charging coil 121, a tightly coupled transmitter / receiver (RF transceiver) 122 (sometimes referred to as a radio frequency (RF) transceiver 122), at least one rechargeable battery 132, and an external sound processing module 124. The external sound processing module 124 may include, for example, one or more processors and a memory device (memory) including sound processing logic. The memory device may also include stimulus control logic 185, which, when executed, enables one or more processors to perform aspects of the presented technology. The memory device may include any or more of the following: non-volatile memory (NVM), ferroelectric random access memory (FRAM), read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, flash memory, electrical, optical, or other physical / tangible memory storage devices. The one or more processors are, for example, microprocessors or microcontrollers that execute instructions stored in the memory device for the sound processing logic and stimulus control logic 185 (having a stimulus generator model 192).

[0028] The implantable component 112 includes an implant body (main module) 134, a lead area 136, and an intracochlear stimulation assembly 116, all configured to be implanted under the user's skin / tissue (tissue) 115. The implant body 134 generally includes an hermetically sealed housing 138, which may include at least one battery 125, an RF interface circuitry 140, and a stimulator unit 142. The implant body 134 also includes an internal / implantable coil 114, which is generally outside the housing 138 but connected via an hermetically sealed feedthrough (…). Figure 1D (Not shown) is connected to the RF interface circuit system 140.

[0029] As mentioned, the stimulation component 116 is configured to be at least partially implanted in the user's cochlea. The stimulation component 116 includes a plurality of longitudinally spaced intracochlear electrical stimulation contacts (electrodes) 144, which together form a contact or electrode array 146 for delivering electrical stimulation (current) to the user's cochlea.

[0030] Stimulation component 116 extends through an opening in the user's cochlea (e.g., cochlear window, round window) wait ), and has via lead area 136 and airtight feedthrough ( Figure 1D (Not shown) is connected to the proximal end of the stimulator unit 142. The lead region 136 includes a plurality of conductors (wires) that electrically couple the electrode 144 to the stimulator unit 142. The implantable component 112 also includes an electrode external to the cochlea, sometimes referred to as the external cochlear electrode (ECE) 139.

[0031] As described, the cochlear implant system 102 includes an external coil 108 and an implantable coil 114. An external magnet 150 is fixed relative to the external coil 108, and an implantable magnet 152 is fixed relative to the implantable coil 114. The magnets fixed relative to the external coil 108 and the implantable coil 114 facilitate operative alignment of the external coil 108 and the implantable coil 114. This operative alignment of the coils enables the external component 104 to transmit power and optional data to the implantable component 112 via a tightly coupled wireless link 148 formed between the external coil 108 and the implantable coil 114. In some examples, the tightly coupled wireless link 148 is a radio frequency (RF) link. However, various other types of power transfer (e.g., infrared (IR), electromagnetic, capacitive, and inductive transfer) can be used to transfer power and / or data from the external component to the implantable component, and therefore, Figure 1D Only one exemplary arrangement is shown.

[0032] As described above, the sound processing unit 106 includes an external sound processing module 124. The external sound processing module 124 is configured to convert received input signals (received at one or more locations in the input device) into output signals for stimulating a user's first ear (i.e., the external sound processing module 124 is configured to perform sound processing on the input signals received at the sound processing unit 106). In other words, one or more processors in the external sound processing module 124 are configured to execute sound processing logic in memory to convert the received input signals into output signals representing electrical stimulation to be delivered to the user. The external sound processing module 124 may preferably use machine learning (e.g., artificial intelligence (AI)) to further control the stimulation provided by the implant 112, according to the techniques presented herein.

[0033] As stated, Figure 1D An embodiment is shown in which an external sound processing module 124 in the sound processing unit 106 generates an output signal. In an alternative embodiment, the sound processing unit 106 may send less processed information (e.g., audio data) to the implantable component 112, and sound processing operations (e.g., sound-to-output signal conversion) may be performed by a processor within the implantable component 112.

[0034] Return to Figure 1DIn a specific example, the output signal is provided to an RF transceiver 122, which transmits the output signal (e.g., in an encoded manner) percutaneously to the implantable component 112 via an external coil 108 and an implantable coil 114. That is, the output signal is received at the RF interface circuitry 140 via the implantable coil 114 and provided to the stimulator unit 142. The stimulator unit 142 is configured to use the output signal to generate an electrical stimulation signal (e.g., a current signal) for delivery to the user's cochlea. In this way, the cochlear implant system 102 electrically stimulates the user's auditory nerve cells, thereby bypassing the missing or defective hair cells that normally translate acoustic vibrations into neural activity, so that the user perceives one or more components of the received sound signal.

[0035] As detailed above, in external hearing mode, the cochlear implant 112 receives processed sound signals from the sound processing unit 106. However, in invisible hearing mode, the cochlear implant 112 is configured to capture and process sound signals for electrical stimulation of the user's auditory nerve cells. Specifically, as Figure 1D As shown, the cochlear implant 112 includes a plurality of implantable sound sensors 160 and an implantable sound processing module 158. Similar to the external sound processing module 124, the implantable sound processing module 158 may include, for example, one or more processors and a memory device (memory) including sound processing logic. The memory device may include any or more of the following: non-volatile memory (NVM), ferroelectric random access memory (FRAM), read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, flash memory, electrical, optical, or other physical / tangible memory storage devices. The one or more processors are, for example, microprocessors or microcontrollers that execute instructions for the sound processing logic stored in the memory device.

[0036] In the invisible hearing mode, the implantable sound sensor 160 is configured to detect / capture signals (e.g., acoustic sound signals, vibrations). waitThe signal is provided to the implantable sound processing module 158. The implantable sound processing module 158 is configured to convert the received input signal (received at one or more of the implantable sound sensors 160) into an output signal for stimulating the user's first ear (i.e., the processing module 158 is configured to perform sound processing operations). In other words, one or more processors in the implantable sound processing module 158 are configured to execute sound processing logic in memory to convert the received input signal into an output signal 156 provided to the stimulator unit 142. The stimulator unit 142 is configured to use the output signal 156 to generate an electrical stimulation signal (e.g., a current signal) for delivery to the user's cochlea, thereby bypassing missing or defective hair cells that typically translate acoustic vibrations into neural activity.

[0037] It should be understood that the above descriptions of the so-called external hearing mode and the so-called invisible hearing mode are merely illustrative, and the cochlear implant system 102 may operate differently in different embodiments. For example, in an alternative embodiment of the external hearing mode, the cochlear implant 112 may use signals captured by the sound input device 118 and the implantable sound sensor 160 to generate stimulation signals for delivery to the user.

[0038] In at least one embodiment, during operation of a hearing device system including a cochlear implant, as referenced below... Figure 2 As further detailed, the sound processing module 124 is configured to convert output signals received from input devices (e.g., one or more sound input devices 118 and / or one or more auxiliary input devices 128) into a set of output signals representing electrical stimulation.

[0039] refer to Figure 2 This diagram illustrates a functional block diagram of an exemplary sound / audio signal processing path for an auditory prosthesis (e.g., cochlear implant system 102) that can utilize various aspects of the techniques presented herein to implement the techniques described herein. Figure 2 The various sound processing operations discussed can be performed via sound processing logic provided for any combination of external or internal components of the cochlear implant system. (Reference) Figure 2 The various features illustrated in the diagram are discussed as follows: Figure 1A-1D Various features of the cochlear implant system 102 are indicated.

[0040] refer to Figure 2 Consider a sensory / environmental signal or audio signal processing path 251 that can be provided via the sound processing module 124 of the external component 104 and / or via the sound processing module 158 of the implantable component 112. Figure 2In the example, the input device may include two audio input devices, namely a first microphone 218A and a second microphone 218B, and at least one auxiliary input device 228 (e.g., an audio input port, a cable port, a pickup coil). wait If not in electrical form, the input device can convert the received / input sound signal into an electrical signal 253 (referred to herein as an electroacoustic or sensory signal), which represents the sound / sensory signal received at the input device. The electroacoustic / sensory signal 253 may include an electrosensory signal 253A from microphone 218A, an electrosensory signal 253B from microphone 218B, and an electrosensory signal 253C from auxiliary input 228.

[0041] exist Figure 2 In this audio signal processing path, the functional operations enabled (i.e., the operations of one or more processors when executing sound processing logic) are typically represented by modules 254, 256, 258, 260, and 262, which together constitute audio signal processing path 251. Therefore, audio signal processing path 251 may include a pre-filter bank processing module 254, a filter bank module 256, a post-filter bank processing module 258, a channel selection module 260, and a mapping module 262, each of which is described in more detail below. Stimulus generator model 192 can be used at any part of the signal processing path or in lieu of any part to generate stimulus signals (e.g., stimulus pulses, analog stimuli, etc.). Furthermore, the stimulus generator model can receive audio signals processed before or at any point during the signal processing path. The processed signal can provide various characteristics as described below. For example, the stimulus generator model can receive audio signals processed by filter bank module 256 as described below.

[0042] Consider an operational example of providing an electroacoustic signal 253 generated by an input device to a pre-filter bank processing module 254. The pre-filter bank processing module 254 is configured to combine the electroacoustic signal 253 received from the input device as needed, and to prepare / enhance these signals for subsequent processing. Operations performed by the pre-filter bank processing module 254 may include, for example, microphone directional operation, noise reduction operation, input mixing / combining operation, input selection / reduction operation, dynamic range control operation, and / or other types of signal enhancement operation. The operations at the pre-filter bank processing module 254 generate a pre-filter bank output signal 255, which, as further described below, forms the basis for additional sound processing operations. The pre-filter bank output signal 255 represents the signal received at a given point in time at the sound input device (e.g., mixed, selected). wait The combination of input signals (e.g., mixing, selection) wait ).

[0043] In operation, the pre-filter bank output signal 255 generated by the pre-filter bank processing module 254 is provided to the filter bank module 256. The filter bank module 256 generates a suitable set of bandwidth-limited channels or frequency partitions, each frequency partition including the spectral components of the received sound / sensory signal. That is, the filter bank module 256 includes multiple bandpass filters that separate the pre-filter bank output signal 255 into multiple components / channels, each component / channel carrying a frequency sub-band of the original signal (i.e., the frequency components of the received sound / sensory signal).

[0044] The channels created by filter bank module 256 are sometimes referred to herein as sound processing or bandpass filtering channels, and the audio signal components within each sound processing channel are sometimes referred to herein as bandpass filtered signals or channelized signals. The bandpass filtered or channelized signals created by filter bank module 256 are processed (e.g., modified / adjusted) as they pass through audio signal processing path 251. Therefore, bandpass filtered or channelized signals are referred to differently at different stages of audio signal processing path 251. However, it will be understood that references to bandpass filtered signals or channelized signals herein can refer to the spectral components of the audio signal received at any point within audio signal processing path 251 (e.g., preprocessing, processing, selection). wait ).

[0045] At the output of filter bank module 256, the channelized signal is referred to herein as the preprocessed signal or filter bank channel 257. The number “n” of filter bank channels 257 generated by filter bank module 256 can depend on a number of different factors, including but not limited to implant design, number of active electrodes, coding strategy and / or (multiple) recipient preferences. In some arrangements, twenty-two (22) channelized signals are created, and audio signal processing path 251 is considered to include 22 channels.

[0046] Filter bank channel 257 is provided to post-filter bank processing module 258. Post-filter bank processing module 258 is configured to perform a number of sound processing operations on target filter bank channel 257. These sound processing operations include, for example, channelized gain adjustment for hearing loss compensation in one or more channels (e.g., performed via loudness growth function (LGF) processing) (e.g., gain adjustment of one or more discrete frequency ranges of the sound signal, also referred to herein as filter channels), noise reduction operations, and speech enhancement operations. wait After performing the sound processing operation, the post-filter bank processing module 258 outputs multiple processed channelized signals 259.

[0047] exist Figure 2In its specific arrangement, the audio signal processing path 251 includes a channel selection module 260. The channel selection module 260 is configured to perform a channel selection process to select, according to one or more selection rules, which of the "n" channels should be used in hearing compensation. The signal selected at the channel selection module 260... Figure 2 The signal is indicated by arrow 261 and is referred to herein as the selected channelization signal, or more simply as the selection signal.

[0048] exist Figure 2 In one embodiment, the channel selection module 260 selects a subset "m" of the "n" processed channelized signals 259 used to generate electrical stimulation for delivery to the recipient (i.e., the number of sound processing channels is reduced from "n" channels to "m" channels). In a specific example, "m" maximum amplitude channels (maximums) are generated from the "n" available combined channel signals, where "n" and "m" are programmable during the initial fitting and / or operation of the prosthesis. In one instance, this specific example may be associated with an advanced combinatorial encoder (ACE), which is typically a stimulus coding strategy such as Optimized Pitch and Speech (OPAL). It should be understood that different channel selection methods can be used and are not limited to maximum selection. It should also be understood that in some embodiments, the channel selection module 260 may be omitted. For example, some arrangements may use sequential alternating sampling (CIS), CIS-based, or other non-channel selection sound coding strategies.

[0049] Figure 2 The audio signal processing path 251 of the example illustrated in the figure may further include a mapping module 262, which can generate an output signal 263. In one embodiment, the mapping module 262 may be configured (e.g., via stimulus generator model 192) to map a selected signal 261 (or a processed channelized signal 259 in embodiments excluding channel selection) such that the output signal 263 corresponds to a set of stimulus control signals (e.g., stimulus commands) representing attributes of an electrical stimulation signal to be delivered to a receiver to induce perception of at least a portion of the received sound signal. For example, this channel mapping may include threshold and comfort level mapping, dynamic range adjustment (e.g., compression), and volume adjustment. wait Furthermore, it can cover the selection of various sequential and / or simultaneous stimulation strategies.

[0050] In one embodiment, a set of stimulation control signals (stimulation commands) 263 representing electrical stimulation signals can be encoded for transdermal transmission to the implantable component (e.g., via an RF link). Therefore, the mapping module 262 can also be referred to as a channel mapping and encoding module, and operates as an output block configured to convert multiple channelized signals into multiple stimulation control signals. The implantable component, via the stimulator unit 142, can generate stimulation (current) signals for delivery to the recipient via the stimulation component 116 based on the multiple stimulation control signals.

[0051] In one embodiment, for example, if the channel selection module 260 is omitted from the audio signal processing path 251, the mapping module 262 can perform a mapping operation involving mapping a channel envelope to a current level that can be mixed with a stream received from one or more sources. Typically, the channel envelope is a "time envelope" extracted from each frequency band (channel) and used to modulate a train of pulses delivered to the implanted electrode. Therefore, the amplitude of the current pulse can be extracted from the channel envelope, where the channel envelope corresponds to the amplitude of the signal in a given frequency channel.

[0052] Therefore, the audio signal processing path 251 typically operates to convert the received sound signal into an output signal 263, which can be used to deliver a stimulus to the recipient in a manner that evokes the perception of the sound signal.

[0053] As described above, cochlear implants electrically stimulate the auditory nerve, bypassing damaged sensory receptors and evoking neural activation patterns that represent acoustic sounds. Although cochlear implants restore hearing for people with severe to complete deafness, many cochlear implant recipients still struggle with complex listening conditions, such as speech perception and music perception in noise.

[0054] These difficulties stem from the limited acoustic information transmitted by cochlear implants. Typically, cochlear implants extract envelopes corresponding to the frequency bands of each implanted electrode, and those envelopes are used to modulate a fixed-rate biphasic pulse train sent to the electrodes. Using only a limited number of frequency bands for the temporal envelopes reduces the temporal and spectral resolution of the acoustic sound. Furthermore, cochlear implants can use techniques such as frequency decomposition to provide computational efficiency, but these techniques only produce a coarse approximation of the sound. Additionally, the current delivered to the electrodes diffuses through the conductive fluid of the cochlea, thus limiting channel independence. Therefore, the neural activation patterns induced by cochlear implants are only coarse approximations of those induced by acoustic hearing.

[0055] According to an exemplary embodiment, stimulation control is provided by an implantable device. Stimulation control can be performed at an external device and can utilize machine learning (e.g., artificial intelligence (AI)). An exemplary embodiment provides a cochlear implant stimulation strategy that utilizes computational models of the healthy hearing system and the implanted hearing system to more closely simulate natural acoustic hearing.

[0056] Advances in electrode design and stimulation techniques (e.g., cochlear periaxial electrode arrays and focused multipolar stimulation) have enabled finer temporal and spectral resolution in the neural activation patterns generated by cochlear implant stimulation. Furthermore, sophisticated models of the auditory system can predict the spike responses of auditory nerve fibers by modeling the middle ear, traveling waves along the basilar membrane, inner hair cell transduction, auditory synapses, and spike behavior of the auditory nerves. Additionally, computational models of electrohearing can predict the auditory nerve response to electrical stimulation by modeling electrode properties, electrode placement, current diffusion within the cochlea, neural activation, and the temporal characteristics of auditory neurons (e.g., reluctance, adaptation, facilitation, and compliance). Computational models of electrohearing can also be personalized for individual cochlear implant recipients by considering unique patterns of neural health along the cochlea, ossification or fibrosis within the cochlea, and / or patient-specific etiological factors.

[0057] Exemplary embodiments minimize the discrepancies between normal hearing and electro-hearing computational models to deliver sound information more consistent with normal hearing and improve outcomes for cochlear implant recipients. This type of strategy was previously infeasible due to the massive computational demands of auditory models, which made real-time applications impossible and increased power consumption. The auditory model incorporates many components not found in spectrogram-based cochlear implant stimulation strategies, including episodic enhancement, fundamental frequency modulation, and traveling wave dynamics. Episodic enhancement and fundamental frequency modulation have been shown to improve speech perception in cochlear implant recipients when applied independently to pulse trains. Exemplary embodiments encode episodic enhancement and fundamental frequency modulation, whose interaction is consistent with the human auditory system.

[0058] Furthermore, the exemplary embodiment pre-compensates for the temporal characteristics of neurons (e.g., reluctance and adaptation) so that portions of the sound stimulus not encoded by neurons in the normal auditory system are not encoded by the cochlear implant processor. This saves power by removing redundant pulses and reduces unnecessary channel interactions.

[0059] In exemplary embodiments, deep neural networks (DNNs) or other machine learning models can be trained to generate stimulation patterns that minimize the differences between acoustic electroencephalography (EEG) and electrically evoked neural excitation patterns. In embodiments, the computational model may include a model for the auditory periphery (e.g., the activity of spiral ganglion neurons in the auditory nerve). However, in other embodiments, the computational model may model more central processes (e.g., the auditory brainstem, hypothalamus, or auditory midbrain).

[0060] In some embodiments, the processing power of a neural network is utilized to deliver higher-resolution auditory information tailored to the individual characteristics of the recipient. In an exemplary embodiment, a neural network is deployed to deliver electroauditory stimulation, the neural network having been trained based on both the recipient's characteristic manner of using electroauditory and a standard reference model of "normal" hearing. Furthermore, exemplary embodiments of the invention can be adapted to a wide range of electrostimulation modalities, including focused multipolar and other sensory electrostimulation therapies, which require consideration of the recipient-specific electrical model with reference to a normal response model to non-electrical stimulation.

[0061] refer to Figure 3 This diagram illustrates a functional block diagram of a method 300 for training a hearing machine learning (ML) model for use with some of the techniques presented herein. Initially, a training set of audio signals or audio samples is provided to the hearing computational model 310. The hearing computational model can be a computational model of a normal-hearing cochlea. These types of models are often computationally expensive and impractical to implement on hearing aids or cochlear implant sound processors.

[0062] The hearing computational model 310 processes audio signals and generates signals represented as an electroneurogram (e.g., as shown in the image). Figure 3 The output of the normal hearing (NH) electroneurogram 320 shown indicates the firing or activation patterns of neurons in the auditory nerve of normal hearing. (See reference...) Figures 4A-4C , Figure 4A A spectrogram 400 for a rising complex tone is shown. The spectrogram plots time along the X-axis, tone frequency along a first Y-axis, and the power level (dB) of the tone along a second relative Y-axis, where the power level is indicated by shading. Figure 4B Electroneurogram 410 shows the inner hair cell voltage for rising complex tones. Electroneurogram 410 plots time along the X-axis, the characteristic frequency of the tone along the first Y-axis, and the inner hair cell voltage (in millivolts) along the second relative Y-axis, where the voltage is indicated by shading. Figure 4CA neurograph 420 is shown illustrating the fine structure of the auditory nerves for rising complex tones. The neurograph 420 plots time along the X-axis, the characteristic frequencies of the tone along a first Y-axis, and the amount of spikes along a second relative Y-axis, where the number of spikes is indicated by shading. The hearing computation model 310 can generate an NH neurograph 320 in the form of neurographs 410 and / or 420 for use with the exemplary embodiments described below. The hearing computation model can employ any conventional or other model of the auditory system capable of predicting spike responses of auditory nerve fibers by modeling the middle ear, traveling waves along the basilar membrane, inner hair cell transduction, auditory nerve synapses, and auditory nerve spike behavior. While spectrograph 400 is typically used in conventional cochlear implant processors, neurographs 410 and 420 can be used with the exemplary embodiments and provide additional detail beyond spectrograph 400.

[0063] The hearing machine learning (ML) model 350 can be trained to perform equivalent functions as the hearing computation model 310, and generate neurographs from audio signals (e.g., as...). Figure 3 The ML electroencephalogram 360 shown is illustrated. The hearing ML model 350 can include any conventional or other machine learning model (e.g., mathematical / statistical model; classifier; decision tree; random forest; feedforward, recursive, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural network). wait ) to generate an electroneurogram. For example, the hearing ML model 350 may include a neural network that is basically similar to the one described below (e.g., Figure 6 ) neural network.

[0064] During the training phase, both the hearing computational model 310 and the hearing ML model 350 are provided with a training set of audio signals or audio samples. The hearing ML model processes the audio signals and generates a representation as an electroneurogram (e.g., as shown in the image). Figure 3 The output of the ML electroneurogram 360 shown indicates the firing or activation patterns of neurons in the auditory nerve. The NH electroneurogram 320 is compared with the ML electroneurogram 360 by a cost function 330, which provides the differences between these electroneurograms to train the hearing ML model 350. The cost function can be any conventional or other cost or error function (e.g., mean absolute error, L1 norm (e.g., the sum of absolute differences of vector components), L2 norm (or Euclidean distance), or a weighted average applied to the difference or error). wait For example, the cost function could be the mean absolute error between the data values ​​of NH electroencephalogram 320 and ML electroencephalogram 360 (e.g., the sum of the absolute values ​​of the errors (or differences) divided by the sample size). waitData values ​​from 320 and 360-degree electroencephalograms can correspond to the same dimensions used to apply the cost function (e.g., the same neurons, the same sampling frequency). wait ). via any conventional or other training technique (e.g., backpropagation). wait The weights of the hearing ML model 350 are adjusted to minimize the cost function of quantizing the difference (or error) between the ML electroencephalogram 360 and the NH electroencephalogram 320. Training of the hearing ML model 350 is complete once the difference between the ML electroencephalogram 360 and the NH electroencephalogram 320 converges (e.g., the difference remains constant or within a threshold range over a specific time period or a specific number of training iterations). The hearing ML model 350 can be used to generate reference electroencephalograms representing normal hearing through exemplary embodiments described below.

[0065] refer to Figure 5 This diagram illustrates a functional block diagram of a method 500 for training a sound processor machine learning (ML) model 540 to control stimuli according to certain embodiments. A training set of audio signals or audio samples is provided to a hearing model 510. The audio samples may include speech, music, broadband stimuli, and / or ambient or any other sound. The hearing model 510 may include a hearing computation model 310, or a previously trained hearing machine learning (ML) model 350, to generate a reference electroneurogram 520 representing normal hearing in substantially the same manner as described above. The audio signals may include microphone signals, the output of a beamformer combining multiple microphone signals, and / or audio signals from a telephone or other audio accessory. Furthermore, various preprocessing techniques (e.g., automatic gain control (AGC), noise reduction, etc.) may be applied. wait The hearing model 510 processes audio signals and generates a neural electrogram (e.g., as shown in the image). Figure 5 The output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510 can generate the reference electroneurogram 520 (e.g., in the form of electroneurogram 410 and / or electroneurogram 420). The audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0066] A sound processor machine learning (ML) model 540 can be trained to generate stimulus information (e.g., impulse information, analog information, etc.) that provides a stimulus electrogram 560 similar to a reference electrogram 520 representing normal hearing. The sound processor ML model 540 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursive, convolutional, convolutional recursive, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate a neural electrogram. For example, the sound processor ML model 540 can employ the following (e.g., Figure 6 ) neural network.

[0067] During the training phase, a training set of audio signals or audio samples is also provided to the sound processor ML model 540. The sound processor ML model 540 processes the audio signals and generates information (e.g., pulse information, analog information, etc.) to be provided to the electrical stimulation model 550. This information may include control or characteristics of the stimulation signal (e.g., pulses), which can be used by the stimulator unit 142 to generate the stimulation signal for the implant 112. For example, the information may indicate the current level and / or other characteristics of the electrodes 144 of the implant 112 at corresponding times (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity). wait ).

[0068] The electrical stimulation model 550 can be any conventional or other computational or machine learning model (e.g., finite element model, biophysical model, phenomenological model, neural network) of the neural response to stimulation signals delivered by the cochlear implant. wait The electrical stimulation model 550 addresses electric field effects (e.g., current diffusion). wait ) and neural interfaces (e.g., neural thresholds for activating neurons, refractory times) wait Modeling. This model is preferably specific to a particular cochlear implant recipient and includes information about the location and type of electrodes in the recipient's cochlea, neural health patterns along the cochlea, fibrosis and ossification within the cochlea, and details about the shape and size of the cochlea and the location of target neural groups. These patient-specific details can be obtained through imaging (e.g., clinical CT scans). wait Electrophysiological measurements (e.g., electroevoked compound action potentials, EEG, electrocochleography) wait ), psychophysical measurements (e.g., detection thresholds, amplitude modulation detection thresholds, masking tuning curves) wait The location of the electrodes can be determined using radiographic imaging (CT, X-ray, etc.) or other surgical applications that use implant telemetry (impedance) to track and estimate the electrode location during implantation.

[0069] For example, the electrical stimulation model 550 can be implemented by a neural network and trained using information and a training set of corresponding known neural responses (e.g., as described below for...). Figure 6 The above) is used to generate neural responses in response to input electroneurograms (e.g., firing or activation patterns of neurons in the auditory nerve). waitThe electrical stimulation model 550 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on information generated by the sound processor machine learning (ML) model 540. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 5 The stimulus electroneurogram 560 is shown in the figure. A cost function 530 compares the reference electroneurogram 520 with the stimulus electroneurogram 560, providing the sound processor ML model 540 with the differences between these electroneurograms. The cost function can be any conventional or other cost or error function (e.g., mean absolute error, L1 norm (e.g., the sum of absolute differences of vector components), L2 norm (or Euclidean distance), or a weighted average applied to the difference or error). wait For example, the cost function could be the mean absolute error between the data values ​​of the reference electroneurogram 520 and the stimulated electroneurogram 560 (e.g., the sum of the absolute values ​​of the errors (or differences) divided by the sample size). wait Data values ​​from electroneurograms 520 and 560 can correspond to the same dimensions used to apply the cost function (e.g., the same neurons, the same sampling frequency). wait (For example, via counterpropagation) wait The weights of the sound processor ML model 540 are adjusted to minimize the cost function of the difference (or error) between the quantized reference electroencephalogram (EEG) 520 and the stimulation EEG 560. Training is complete once the difference between the reference EEG and the stimulation EEG converges (e.g., the difference remains constant or within a threshold range over a specific time period or number of training iterations), and the sound processor ML model 540 can be used in a sound processor of an exemplary embodiment (e.g., its stimulation generator model 192) to process audio signals and generate information to control the stimulator unit 142 to generate and apply stimulation signals for stimulation. Therefore, during the training phase, the sound processor ML model 540 is trained such that the stimulation EEG generated from the information produced by the sound processor ML model is a close approximation of the reference EEG representing normal hearing. The stimulation ML learning model 540 can be deployed to the device by providing the weights of the trained model.

[0070] For example, the sound processor ML model 540 can employ a neural network. Figure 6An exemplary neural network 600 is illustrated. The neural network 600 may include an input layer 610, one or more intermediate layers 620 (e.g., including any hidden layers), and an output layer 630. Each layer includes one or more neurons 650, wherein input layer neurons receive input (e.g., an audio signal or audio sample) and may be associated with weight values. Neurons in the intermediate and output layers are connected to one or more neurons in the preceding layer and receive the outputs of the connected neurons in the preceding layer as inputs. Each connection is associated with weight values, and each neuron produces an output based on a weighted combination of its inputs. The outputs of neurons may also be based on certain types of neural networks (e.g., recursive neural networks). wait The bias value of ).

[0071] Weight (and bias) values ​​can be adjusted based on various training techniques. For example, a neural network can be machine-learned using a training dataset as input and a corresponding known or reference output, where the neural network attempts to produce the provided output and adjusts the weight (and bias) values ​​using the error between the output (e.g., the difference between the produced output and the known output) (e.g., via backpropagation or other training techniques).

[0072] The reference output corresponds to a reference electroneurogram (EMG) 520 representing normal hearing, relative to the sound processor machine learning (ML) model 540. In this case, the weights of the sound processor ML model 540 are adjusted to provide appropriate information to the electrical stimulation model 550 to generate a stimulated EMG 560 that matches or approximates the reference EMG 520 representing normal hearing. As described above, the difference between the reference EMG and the stimulated EMG is provided to adjust the weights of the sound processor ML model.

[0073] In some embodiments, as described below, feature vectors can be extracted from training set input data and used as input for training, while their known or reference corresponding outputs can be used as known or reference outputs for training. Feature vectors can include any suitable features of the training set input data. For example, features of an audio signal can include fundamental frequency or other frequencies, pitch, amplitude or intensity, spectrogram (magnitude and / or phase), and Mel-frequency cepstral coefficients. wait .

[0074] The output layer of a neural network indicates the output obtained from the input data (e.g., impulse information). wait Output layer neurons can also indicate the probability of the output.

[0075] In some embodiments, the signal processing path is divided into independent modules to improve efficiency. (See reference) Figure 7This diagram illustrates a functional block diagram of a method 700 for training a sound processor machine learning (ML) model using feature extraction applied to an input audio signal, according to certain embodiments. An exemplary feature extraction process includes a Fast Fourier Transform (FFT) operating over a sliding time window. The FFT essentially transforms the audio signal from the time domain to the frequency domain. A training set of audio signals or audio samples is provided to a hearing model 510. Audio samples may include speech, music, broadband stimuli, and / or ambient or any other sound. The hearing model may include a hearing computation model 310, or a previously trained hearing machine learning (ML) model 350, to generate a reference electroneurogram 520 representing normal hearing in substantially the same manner as described above. The audio signal may include a microphone signal, the output of a beamformer combining multiple microphone signals, and / or an audio signal from a telephone or other audio accessory. Furthermore, various preprocessing techniques (e.g., automatic gain control (AGC), noise reduction, etc.) may be applied. wait The hearing model 510 processes audio signals and generates a neural electrogram (e.g., as shown in the image). Figure 7 The output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510 can generate the reference electroneurogram 520 in the form of electroneurograms 410 and / or 420 as described above. The audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0076] A sound processor machine learning (ML) model 720 can be trained to generate stimulus signals that provide an electroneurogram similar to a reference electroneurogram 520 representing normal hearing. The sound processor ML model 720 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursion, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate an electroneurogram. For example, the sound processor ML model 720 can employ the methods described above (e.g., Figure 6 ) neural network.

[0077] During the training phase, a training set of audio signals or audio samples is also provided to the feature extraction module 710 to extract features from the audio signals. Features can include any desired features or attributes (e.g., spectrograms, features related to normal hearing). waitIn an embodiment, the feature extraction module may perform a Fast Fourier Transform (FFT) on the audio signal to extract features from it. The FFT essentially transforms the audio signal from the time domain to the frequency domain. The extracted features may include the FFT output, magnitude, phase, frequency, Mel-frequency cepstral coefficients (MFCC), and / or any other signal features. The extracted features are provided to the sound processor ML model 720. The sound processor ML model 720 processes the extracted features and generates information to be provided to the electrical stimulation model 550. For example, as described above, the information may indicate the current level and / or other characteristics of the electrodes 144 of the implant 112 at corresponding times (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity). wait The electrical stimulation model can be any conventional or other computational or machine learning model of the neural response to stimulation signals delivered by the cochlear implant (e.g., finite element model, biophysical model, phenomenological model, neural network). wait And it is basically similar to the above-mentioned electrical stimulation model.

[0078] The electrical stimulation model 550 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on information produced by the sound processor machine learning (ML) model 720. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 7 The stimulation electroneurogram 730 is shown in the diagram. A cost function 530 compares the reference electroneurogram 520 with the stimulation electroneurogram 730, which is determined in essentially the same manner as described above and provides the sound processor machine learning (ML) model 720 with the differences between these electroneurograms. This is done in essentially the same manner as described above (e.g., via backpropagation). wait The weights of the sound processor ML model 720 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroencephalogram 520 and the stimulation electroencephalogram 730.

[0079] Once the difference between the reference electroneurogram 520 and the stimulation electroneurogram 730 converges (e.g., remains constant or within a threshold range over a specific time period or number of training iterations), training is complete, and the feature extraction module 710 and the sound processor ML model 720 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to extract and process audio signal features and generate information to control the stimulator unit 142 to generate and apply stimulation signals for stimulation.

[0080] Therefore, during the training phase, the reference output for the sound processor machine learning (ML) model 720 corresponds to a reference electroneurogram 520 representing normal hearing. In this case, the weights of the sound processor ML model 720 are adjusted to provide appropriate information to the electrical stimulation model 550 to generate a stimulation electroneurogram 730 of normal hearing that matches or approximates the reference electroneurogram 520. As described above, the difference between the reference electroneurogram and the stimulation electroneurogram is provided to adjust the weights of the sound processor ML model 720. Thus, the sound processor ML model 720 is trained such that the stimulation electroneurogram generated from the information produced by the sound processor ML model 720 is a close approximation of the reference electroneurogram representing normal hearing. The stimulation ML learning model 720 can be deployed to the device by providing the weights of the trained model.

[0081] In some embodiments, the filter bank process can act as a feature extractor and be applied to the audio signal. (Reference) Figure 8 This diagram illustrates a functional block diagram of a method 800 for training a sound processor machine learning (ML) model using another feature extraction applied to an input audio signal, according to certain embodiments. An exemplary feature extraction process includes a filter bank process. A set of training audio signals or audio samples is provided to a hearing model 510. Audio samples may include speech, music, broadband stimuli, and / or ambient or any other sound. The hearing model may include a hearing computation model 310, or a previously trained hearing machine learning (ML) model 350, to generate a reference electroneurogram 520 representing normal hearing in substantially the same manner as described above. The audio signals may include microphone signals, the output of a beamformer combining multiple microphone signals, and / or audio signals from a telephone or other audio accessory. Furthermore, various preprocessing techniques (e.g., automatic gain control (AGC), noise reduction, etc.) may be applied. wait The hearing model 510 processes audio signals and generates a neural electrogram (e.g., as shown in the image). Figure 8 The output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510 can generate the reference electroneurogram 520 in the form of electroneurograms 410 and / or 420 as described above. The audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0082] A sound processor machine learning (ML) model 820 can be trained to generate stimulus signals that provide an electroneurogram similar to a reference electroneurogram 520 representing normal hearing. The sound processor ML model 820 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursion, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate an electroneurogram. For example, the sound processor ML model 820 can employ the methods described above (e.g., Figure 6 ) neural network.

[0083] During the training phase, a training set of audio signals or audio samples is also provided to the filter bank unit 810 to extract features from the audio signals. In an embodiment, the filter bank unit generates a suitable set of bandwidth-limited channels or frequency partitions, each including spectral components of the received audio. Various features can be extracted from the resulting channels or partitions and provided to the sound processor machine learning (ML) model 820 (e.g., the output of the filter bank processing, frequency partitions or channels, frequency, phase, amount of partitions or channels, amplitude, etc.). The filter bank unit can perform any conventional or other filter bank process (e.g., corresponding to...). Figure 2 Filter bank module 256 wait The number of filters in the filter bank can be equal to the number of electrical stimulation channels in the cochlear implant. The extracted features are provided to the sound processor ML model 820. The sound processor ML model 820 processes the extracted features and generates information to be provided to the electrical stimulation model 550. For example, as described above, the information may indicate the current level and / or other characteristics of the electrodes 144 of the implant 112 at corresponding times (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity). wait The electrical stimulation model 550 can be any conventional or other computational or machine learning model of the neural response to stimulation signals delivered by the cochlear implant (e.g., finite element model, biophysical model, phenomenological model, neural network). wait And it is basically similar to the above-mentioned electrical stimulation model.

[0084] The electrical stimulation model 550 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on information produced by the sound processor machine learning (ML) model 820. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 8The stimulation electroneurogram 830 is shown in the diagram. A cost function 530 compares the reference electroneurogram 520 with the stimulation electroneurogram 830, which is determined in essentially the same manner as described above and provides the sound processor ML model 820 with the differences between these electroneurograms (e.g., via backpropagation). wait The weights of the sound processor ML model 820 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroencephalogram 520 and the stimulation electroencephalogram 830.

[0085] Once the difference between the reference electroneurogram 520 and the stimulation electroneurogram 830 converges (e.g., the difference remains constant or within a threshold range within a specific time period or a specific number of training iterations), training is complete, and the filter bank module 810 and the sound processor ML model 820 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to extract and process audio signal features and generate information to control the stimulator unit 142 to generate and apply stimulation signals for stimulation.

[0086] Therefore, during the training phase, the reference output for the sound processor machine learning (ML) model 820 corresponds to a reference electroneurogram 520 representing normal hearing. In this case, the weights of the sound processor ML model 820 are adjusted to provide appropriate information to the electrical stimulation model 550 to generate a stimulation electroneurogram 830 of normal hearing that matches or approximates the reference electroneurogram 520. As described above, the difference between the reference electroneurogram and the stimulation electroneurogram is provided to adjust the weights of the sound processor ML model 820. Thus, the sound processor ML model 820 is trained such that the stimulation electroneurogram generated from the information produced by the sound processor ML model 820 is a close approximation of the reference electroneurogram representing normal hearing. The stimulation ML learning model 820 can be deployed to the device by providing the weights of the trained model.

[0087] In an embodiment, the stimulation generator model 192 can process the electroneurogram representing normal hearing generated from the hearing model to generate information for the electrical stimulation model. In this case, the stimulation generator model 192 is trained to eliminate the effects of electrical stimulation on the recipient (e.g., modeled or introduced by the electrical stimulation model) to produce an electroneurogram that matches or approximates normal hearing.

[0088] refer to Figure 9This diagram illustrates a functional block diagram of a method 900 for training a sound processor machine learning (ML) model using an electroneurogram of an input audio signal, according to certain embodiments. A set of training audio signals or audio samples is provided to a hearing model 510. The audio samples may include speech, music, broadband stimuli, and / or ambient or any other sound. The hearing model may include a hearing computation model 310, or a previously trained hearing machine learning (ML) model 350, to generate a reference electroneurogram 520 representing normal hearing in substantially the same manner as described above. The audio signals may include microphone signals, the output of a beamformer combining multiple microphone signals, and / or audio signals from a telephone or other audio accessory. Furthermore, various preprocessing techniques (e.g., automatic gain control (AGC), noise reduction, etc.) may be applied. wait The hearing model 510 processes audio signals and generates a neural electrogram (e.g., as shown in the image). Figure 9 The output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510 can generate the reference electroneurogram 520 in the form of electroneurograms 410 and / or 420 as described above. The audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0089] A sound processor machine learning (ML) model 920 can be trained to generate stimulus signals that provide an electroneurogram similar to a reference electroneurogram 520 representing normal hearing. The sound processor ML model 920 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursion, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate an electroneurogram. For example, the sound processor ML model 920 can employ the methods described above (e.g., Figure 6 ) neural network.

[0090] During the training phase, a reference electroneurogram 520 from the hearing model 510 is also provided to the sound processor ML model 920. The sound processor ML model 920 processes the reference electroneurogram and generates information to be provided to the electrical stimulation model 550. For example, as described above, the information may indicate the current level and / or other characteristics of the electrodes 144 of the implant 112 at corresponding times (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity). wait The electrical stimulation model 550 can be any conventional or other computational or machine learning model (e.g., finite element model, neural network) of the neural response to stimulation signals delivered by the cochlear implant. waitAnd it is basically similar to the above-mentioned electrical stimulation model.

[0091] The electrical stimulation model 550 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on information produced by the sound processor machine learning (ML) model 920. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 9 The stimulation electroneurogram 930 is shown in the diagram. A cost function 530 compares the reference electroneurogram 520 with the stimulation electroneurogram 930, which is determined in essentially the same manner as described above and provides the sound processor ML model 920 with the differences between these electroneurograms (e.g., via backpropagation). wait The weights of the sound processor ML model 920 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroneurogram 520 and the stimulation electroneurogram 930.

[0092] Once the difference between the reference electroneurogram 520 and the stimulation electroneurogram 930 converges (e.g., the difference remains constant or within a threshold range over a specific time period or number of training iterations), training is complete, and the hearing model 510 and the sound processor ML model 920 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to process audio signals and generate information in substantially the same manner as described above. This information controls the stimulator unit 142 to generate and apply stimulation signals for stimulation.

[0093] Therefore, during the training phase, the reference output for the sound processor machine learning (ML) model 920 corresponds to a reference electroneurogram 520 representing normal hearing. In this case, the weights of the sound processor ML model 920 are adjusted to provide appropriate information to the electrical stimulation model 550 to produce a stimulation electroneurogram 930 of normal hearing that matches or approximates the reference electroneurogram 520. As described above, the difference between the reference electroneurogram and the stimulation electroneurogram is provided to adjust the weights of the sound processor ML model 920. Thus, the sound processor ML model 920 is trained such that the stimulation electroneurogram generated from the information produced by the sound processor ML model 920 is a close approximation of the reference electroneurogram representing normal hearing. In other words, the sound processor ML model 920 is trained to eliminate the effects of electrical stimulation on the recipient (e.g., modeled or introduced by the electrical stimulation model) to produce an electroneurogram that matches or approximates normal hearing. The stimulation ML learning model 920 can be deployed to the device by providing the weights of the trained model.

[0094] In some embodiments, in addition to simulating the behavior of a normal hearing cochlea, noise reduction capability is also provided. In this case, during training, a clean audio signal is provided to the hearing model to generate a reference electroencephalogram (EEG) representing normal hearing, while a noisy frequency signal is provided (e.g., noise is added to the clean audio signal) to generate an EEG for training the stimulus generator model. The stimulus generator model is trained to minimize the difference between the clean reference EEG and the resulting stimulus EEG from the noisy signal. The stimulus generator model thus performs noise reduction and eliminates the effects of electrical stimulation on the recipient (modeled or introduced by the electrical stimulation generator model).

[0095] refer to Figure 10 The diagram illustrates a functional block diagram of a method 1000 for training a sound processor machine learning (ML) model to reduce noise according to certain embodiments. A clean training audio signal or a set of audio samples is provided to a first hearing model 510A. The audio samples may include speech, music, broadband stimuli and / or ambient or any other sound. The clean audio signal set is also provided to a mixer 1010, which introduces noise to produce a noisy signal. The noise may include any type of noise (e.g., from the surrounding environment, synthetic noise, noisy human voice noise, reverberation or other convolutional noise, echo, cafe / restaurant noise). wait The noisy signal is provided to a second hearing model 510B. Hearing models 510A and 510B may include a hearing computation model 310 or a previously trained hearing machine learning (ML) model 350 to generate a neural electrogram representing normal hearing in substantially the same manner as described above. The clean audio signal (before the introduction of noise) may include microphone signals, the output of a beamformer combining multiple microphone signals, and / or audio signals from a telephone or other audio accessory. Hearing model 510A processes the clean audio signal and produces a neural electrogram (e.g., as shown in the image). Figure 10 The output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. Hearing model 510A can generate the reference electroneurogram 520 in the form of electroneurograms 410 and / or 420 as described above. Similarly, hearing model 510B processes noisy frequency signals and generates an electroneurogram (e.g., as shown in the reference electroneurogram 520). Figure 10 The output of the noisy electroneurogram 1015 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510B can generate the noisy electroneurogram 1015 in the form of electroneurograms 410 and / or 420 as described above. As mentioned above, the audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0096] A sound processor machine learning (ML) model 1020 can be trained to generate stimulus signals that provide an electroneurogram similar to a reference electroneurogram 520 representing normal hearing. The sound processor ML model 1020 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursion, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate an electroneurogram. For example, the sound processor ML model 1020 can employ the methods described above (e.g., Figure 6 ) neural network.

[0097] During the training phase, a noisy electroneurogram 1015 from the hearing model 510B is provided to the sound processor ML model 1020. The sound processor ML model 1020 processes the noisy electroneurogram and generates information to be provided to the electrical stimulation model 550. For example, as described above, this information may indicate the current level and / or other characteristics of the electrodes 144 of the implant 112 at corresponding times (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity). wait The electrical stimulation model 550 can be any conventional or other computational or machine learning model (e.g., finite element model, neural network) of the neural response to stimulation signals delivered by the cochlear implant. wait And it is basically similar to the above-mentioned electrical stimulation model.

[0098] The electrical stimulation model 550 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on information produced by the sound processor machine learning (ML) model 1020. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 10 The stimulation electroneurogram 1030 is shown in the figure. A cost function 530 compares the reference electroneurogram 520 (generated from a clean audio signal) with the stimulation electroneurogram 1030 (generated from a noisy frequency signal). The cost function is determined in essentially the same manner as described above and provides the sound processor ML model 1020 with the differences between these electroneurograms. (e.g., via backpropagation) wait The weights of the sound processor ML model 1020 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroencephalogram 520 and the stimulation electroencephalogram 1030.

[0099] Once the difference between the reference electroneurogram 520 and the stimulation electroneurogram 1030 converges (e.g., the difference remains constant or within a threshold range over a specific time period or number of training iterations), training is complete, and the hearing model 510A and the sound processor ML model 1020 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to process audio signals and generate information in substantially the same manner as described above. This information controls the stimulator unit 142 to generate and apply stimulation signals for stimulation.

[0100] Therefore, during the training phase, the reference output for the sound processor machine learning (ML) model 1020 corresponds to a reference electroneurogram 520 representing normal hearing. In this case, the weights of the sound processor ML model 1020 are adjusted to provide appropriate information to the electrical stimulation model 550 to produce a stimulation electroneurogram 1030 of normal hearing that matches or approximates the reference electroneurogram 520. As described above, the difference between the reference electroneurogram and the stimulation electroneurogram is provided to adjust the weights of the sound processor ML model 1020. Thus, the sound processor ML model 1020 is trained such that the stimulation electroneurogram generated from the information generated by the sound processor ML model 1020 based on the noisy frequency signal is a close approximation of the reference electroneurogram representing normal hearing from a clean audio signal. In other words, the sound processor ML model 1020 is trained to eliminate the effects of noise and electrical stimulation on the receiver (e.g., modeled or introduced by the electrical stimulation model) to produce a neuroneurogram of normal hearing that matches or approximates a clean audio signal. The stimulation ML learning model 1020 can be deployed to the device by providing the weights of the trained model.

[0101] Although commercial cochlear implants typically use monopolar stimulation, exemplary embodiments can be used with any stimulation modality. The spatial resolution of electrical stimulation can be controlled, for example, by using different electrode configurations for a given stimulation channel to activate neural cell regions of different widths. For example, monopolar stimulation is an electrode configuration in which, for a given stimulation channel, current is “pulled out” via one of the intracochlear electrodes 144, but the current is diverted by electrodes outside the cochlea (sometimes referred to as external cochlear electrodes (ECE) 139). Figure 1D "Injection." Monopolar stimulation typically exhibits a large degree of current diffusion (i.e., a broad stimulation pattern) and therefore has low spatial resolution. Other types of electrode configurations, such as bipolar, tripolar, and focused multipolar (FMP) stimulation, are also known as "phased array" stimulation. waitTypically, the size of the stimulated neural population is reduced by "pulling out" current via one or more of the cochlear electrodes 144, while simultaneously "infusing" current via one or more other adjacent cochlear electrodes. Bipolar, tripolar, focused multipolar, and other electrode configurations that both pull out and infuse current via the cochlear electrodes are generally and collectively referred to herein as "focused" stimulation. Compared to monopolar stimulation, focused stimulation typically exhibits less current diffusion (i.e., a narrower stimulation pattern) and therefore higher spatial resolution. Similarly, other electrode configurations, such as bipolar patterns, virtual channels, wide channels, and defocused multipolar, also exhibit this characteristic. wait The size of the stimulated nerve group is usually increased by “pulling out” current through multiple adjacent cochlear electrodes.

[0102] Exemplary embodiments may utilize more advanced stimulation models focusing on multipolar stimulation. Given multipolar information derived from channel amplitude (for stimulation channels) by a sound processor machine learning (ML) model, the electrical stimulation model computes an electroneurogram. The stimulation generator model 192 is trained to eliminate the effects of electrical stimulation on the recipient (e.g., modeled or introduced by the electrical stimulation model) to produce an electroneurogram that matches or approximates normal hearing.

[0103] refer to Figure 11A This diagram illustrates a functional block diagram of a method 1100 for training a sound processor machine learning (ML) model for focused multipolar stimulation using an electroneurogram of an input audio signal, according to certain embodiments. The focused multipolar stimulation parameters can be individualized using electrical measurements from the cochlea (e.g., transimpedance matrix or electrically evoked compound action potentials). A set of training audio signals or audio samples is provided to a hearing model 510. The audio samples may include speech, music, broadband stimulation, and / or ambient or any other sound. The hearing model may include a hearing computational model 310, or a previously trained hearing machine learning (ML) model 350, to generate a reference electroneurogram 520 representing normal hearing in substantially the same manner as described above. The audio signals may include microphone signals, the output of a beamformer combining multiple microphone signals, and / or audio signals from a telephone or other audio accessory. Furthermore, various preprocessing techniques (e.g., automatic gain control (AGC), noise reduction, etc.) can be applied. wait The hearing model 510 processes audio signals and generates a neural electrogram (e.g., as shown in the image). Figure 11AThe output of the reference electroneurogram 520 shown indicates the firing or activation patterns of neurons in the auditory nerve. The hearing model 510 can generate the reference electroneurogram 520 in the form of electroneurograms 410 and / or 420 as described above. The audio signal can be divided into frames of arbitrary desired duration or length, and an electroneurogram or neural activation pattern can be generated for each frame.

[0104] A sound processor machine learning (ML) model 1120 can be trained to generate channel amplitudes to be supplied to a focused multipole pulse generator 1123. The focused multipole pulse generator produces focused multipole pulses used to provide a neural electrogram similar to a reference neural electrogram 520 representing normal hearing. The sound processor ML model 1120 can employ any conventional or other machine learning model (e.g., mathematical / statistical models; classifiers; decision trees; random forests; feedforward, recursion, convolution, convolutional recursion, deep learning, gating, long short-term memory (LSTM), self-attention, encoder / decoder, or other neural networks). wait ) to generate an electroneurogram. For example, the sound processor ML model 1120 can employ the methods described above (e.g., Figure 6 ) neural network.

[0105] During the training phase, a reference electroneurogram 520 from the hearing model 510 is also provided to the sound processor ML model 1120. The sound processor ML model 1120 processes the reference electroneurogram and generates channel amplitudes to be provided to the multipole pulse generator 1123. The multipole pulse generator generates multipole pulse information to be provided to the electrical stimulation model 1125. For example, the multipole pulse information may indicate the current level and / or other characteristics (e.g., which electrodes are active, the current level used for the electrodes, the duration of activity) of a specific group of electrodes 144 of the implant 112 of the stimulation channel at corresponding times. wait Furthermore, it provides finer control to activate smaller populations or groups of neurons. The electrical stimulation model 1125 can be any conventional or other computational or machine learning model (e.g., finite element model, neural network) of the neural response to focused multipolar stimulation pulses delivered by the cochlear implant. wait And it is basically similar to the above-mentioned electrical stimulation model.

[0106] Electrical stimulation model 1125 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on multipole pulse information generated from the channel amplitude of the sound processor machine learning (ML) model 1120. The firing or activation patterns are represented as neuroelectrographs (e.g., as shown in the image). Figure 11AThe stimulation electroneurogram 1130 is shown in the figure. A cost function 530 compares the reference electroneurogram 520 with the stimulation electroneurogram 1130, the cost function being determined in essentially the same manner as described above, and provides the sound processor ML model 1120 with the differences between these electroneurograms (e.g., via backpropagation). wait The weights of the sound processor ML model 1120 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroencephalogram 520 and the stimulation electroencephalogram 1130.

[0107] Once the difference between the reference electroneurogram 520 and the stimulation electroneurogram 1130 converges (e.g., the difference remains constant or within a threshold range over a specific time period or number of training iterations), training is complete, and the hearing model 510, the sound processor ML model 1120, and the multipole pulse generator 1123 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to process audio signals and generate channel amplitude and multipole pulse information in substantially the same manner as described above. The multipole pulse information controls the stimulator unit 142 to generate and apply focused multipole stimulation pulses for stimulation.

[0108] Therefore, during the training phase, the reference output for the sound processor machine learning (ML) model 1120 corresponds to a reference electroneurogram 520 representing normal hearing. In this case, the weights of the sound processor ML model 1120 are adjusted to provide appropriate channel amplitudes to generate multipole pulse information for the electrical stimulation model 1125. The electrical stimulation model 1125 generates a stimulation electroneurogram 1130 of normal hearing that matches or approximates the reference electroneurogram 520. As described above, the difference between the reference electroneurogram and the stimulation electroneurogram is provided to adjust the weights of the sound processor ML model 1120. Thus, the sound processor ML model 1120 is trained such that the stimulation electroneurogram generated based on the multipole pulse information derived from the channel amplitudes of the sound processor ML model 1120 is a close approximation of the reference electroneurogram representing normal hearing. In other words, the sound processor ML model 1120 is trained to eliminate the effects of focused multipole stimulation on the recipient (e.g., modeled or introduced by the electrical stimulation model) to generate an electroneurogram that matches or approximates normal hearing. The stimuli ML learning model 1120 can be deployed to the device by providing the weights (and other parameters) of the trained model.

[0109] refer to Figure 11BThe diagram illustrates a functional block diagram of a method 1150 for training a sound processor machine learning (ML) model using an electroencephalogram of an input audio signal for focusing on multipolar stimulation, according to certain embodiments. Method 1150 is substantially similar to method 1100 described above, except that the sound processor ML model 1120 includes a multipolar pulse generator 1123 and directly generates multipolar pulse information.

[0110] During the training phase, as described above, a reference electroneurogram 520 from the hearing model 510 is provided to the sound processor ML model 1120. The sound processor ML model 1120 processes the reference electroneurogram and generates focused multipole pulse information to be provided to the electrical stimulation model 1125. The electrical stimulation model 1125 generates an output indicating the firing or activation patterns of neurons in the auditory nerve based on the multipole pulse information generated by the sound processor ML model 1120. The firing or activation patterns are represented as an electroneurogram (e.g., as shown in the image). Figure 11B The stimulation of the neurograph shown is 1130.

[0111] The reference electroneurogram 520 is compared with the stimulation electroneurogram 1130 by a cost function 530, which is determined in essentially the same manner as described above and provides the sound processor ML model 1120 with the differences between these electroneurograms. (e.g., via backpropagation) wait The weights of the sound processor ML model 1120 are adjusted to minimize the cost function of the difference (or error) between the quantization reference electroencephalogram 520 and the stimulation electroencephalogram 1130.

[0112] Once the difference between the reference electroencephalogram 520 and the stimulation electroencephalogram 1130 converges (e.g., the difference remains constant or within a threshold range over a specific time period or number of training iterations), training is complete, and the hearing model 510 and the sound processor ML model 1120 can be used in the sound processor of the exemplary embodiment (e.g., its stimulation generator model 192) to process audio signals and generate focused multipole pulse information in substantially the same manner as described above. As described above, the multipole pulse information controls the stimulator unit 142 to generate and apply focused multipole stimulation pulses for stimulation. As described above, the stimulation ML learning model 1120 can be deployed to the device by providing weights to the trained model.

[0113] The exemplary embodiments can be used for any stimulus pattern, wherein various machine learning models of the exemplary embodiments (e.g., sound processor machine learning (ML) models 540, 720, 820, 920, 1020, 1120) can be trained. wait This can be used to generate pulse information for one or more stimulation patterns. For example, training data including stimulation model data (e.g., audio signals, electroencephalograms, feature sets) can be utilized. waitThe machine learning model is trained using stimulus model data that indicates the type of stimulus model (e.g., unipolar, focused multipolar). wait This additional data effectively creates independent spaces for various stimulus patterns. Machine learning models can be trained in essentially the same way as described above to map inputs (with stimulus model data) to impulse information corresponding to the space of stimulus patterns indicated by the stimulus model data.

[0114] Various machine learning models (e.g., sound processor machine learning (ML) models 540, 720, 820, 920, 1020, 1120) can be pre-trained for exemplary embodiments targeting general or average cochlea. wait (For example, similar to the hearing machine learning model 350 mentioned above). In this case, these machine learning models are then retrained using receiver-specific data to tailor the machine learning model for a specific receiver. This allows for faster training of machine learning models with less training data.

[0115] The sound processor machine learning (ML) model can generate information on external devices (e.g., external component 104). wait On and / or in another computing system (e.g., computing device 110) that communicates with external devices. wait Executed on another computing system (e.g., to save processing power and / or battery life). The information is generated on another computing system (e.g., to save processing power and / or battery life). wait In the event of [the situation], other computing systems can send the information to the external device. The voice processor machine learning model can be trained on the external device, other computing systems, and / or independent systems using training data, and deployed for use on the external device and / or other computing systems. Additionally, the voice processor machine learning model can be dynamically or continuously updated or trained (and deployed) based on new information collected and obtained from the implant.

[0116] Sound processor machine learning (ML) models can be trained in various ways. These models can utilize training data (e.g., pre-defined training data, data from analog or actual hardware) on external devices, other computing systems, and / or standalone systems. wait The sound processor ML model can be trained and deployed for use on external devices and / or other computing systems. Additionally, the sound processor ML model can be dynamically or continuously updated or trained (and deployed) based on new information collected from the implant. Training can be performed by stimulus control logic 185.

[0117] In some embodiments, a sound processor machine learning (ML) model can be trained using various training data. For example, training data may include multiple audio files providing various different scenarios. The scenarios may include real-world examples. These various scenarios can be used as training data to train the sound processor ML model to conform to normal hearing.

[0118] Sound processor machine learning (ML) models can be trained using the entirety or any part of the training data in essentially the same way as described above.

[0119] In some embodiments, the technologies presented herein may also include vestibular devices (e.g., vestibular implants), visual devices (i.e., bionic eyes), sensors, pacemakers, drug delivery systems, defibrillators, functional electrical stimulation devices, catheters, seizure monitoring devices (e.g., devices for monitoring and / or treating epileptic events), sleep apnea devices, and electroporation devices. wait This can be implemented, or used in combination with them. The electrical stimulation model can model the physiological effects (or physiological responses) of determined information, and this modeling can be used to train a corresponding sound processor machine learning (ML) model to determine stimulation signals (e.g., impulses) that enable physiology to closely approximate or match normal physiological or sensory functions (e.g., vision, smell, hearing, cardiac function, etc.) in substantially the same manner as described above. Therefore, the techniques of the exemplary embodiments can be applied to any system providing electrical stimulation to improve or compensate for inappropriate or deteriorated physiological or sensory functions in substantially the same manner as described above.

[0120] Now for reference Figure 12 The flowchart of 1200 depicts a method 1200 for implementing the technology of this disclosure. Method 1200, beginning at 1205, may include receiving signals associated with physiological function at an implantable medical device system. At 1210, the method may include determining, based on the signals, information by a machine learning model to determine stimulation signals for stimulating physiological function. The machine learning model is trained based on modeling the physiological effects from the stimulation. At 1215, the method may include controlling stimulation of a recipient of the implantable medical device system based on the determined information. Therefore, the method of flowchart 1200 provides a process for determining and controlling stimulation based on machine learning.

[0121] Now for reference Figure 13The flowchart of 1300 depicts a method 1300 for implementing the techniques of this disclosure. Method 1300, beginning at 1305, may include operations that include a machine learning model, powered by at least one processor, determining information about a stimulus signal for stimulating a physiological function based on signals associated with that function. At 1310, the method may include modeling a physiological effect from the stimulus signal via at least one processor. At 1315, the method may include updating the machine learning model via at least one processor based on a difference between the modeled physiological effect and a reference physiological effect representing normal physiological function. Therefore, the method of flowchart 1300 provides a process for training a machine learning model to control a stimulus.

[0122] It should be understood that while the specific uses of this technology have been described and discussed above, the disclosed technology can be used with various devices based on many examples of this technology. The foregoing discussion is not intended to suggest that the disclosed technology is only suitable for implementation in systems similar to those shown in the accompanying drawings. In general, additional configurations can be used to practice the processes and systems described herein and / or some aspects can be excluded without departing from the processes and systems disclosed herein.

[0123] This disclosure describes some aspects of the invention with reference to the accompanying drawings, which illustrate only some possible aspects. However, other aspects may be embodied in many different forms and should not be construed as limited to those set forth herein. Rather, these aspects are provided to make this disclosure exhaustive and complete and to fully convey the scope of possible aspects to those skilled in the art.

[0124] It should be understood that various aspects described herein with respect to the accompanying drawings (e.g., parts, components) wait This document is not intended to limit the systems and processes to the specific aspects described herein. Therefore, additional configurations can be used to practice the methods and systems described herein, and / or some aspects can be excluded without departing from the methods and systems disclosed herein.

[0125] According to some aspects, a system and a non-transitory computer-readable storage medium are provided. The system is configured with hardware configured to perform operations similar to those of the present disclosure. One or more non-transitory computer-readable storage media include instructions that, when executed by one or more processors, cause one or more processors to perform operations similar to those of the present disclosure.

[0126] Similarly, where the steps of a process are disclosed, these steps are described for illustrative purposes of the method and system and are not intended to limit this disclosure to a particular sequence of steps. For example, these steps may be performed in a different order, two or more steps may be performed simultaneously, additional steps may be performed, and the disclosed steps may be excluded without departing from this disclosure. Furthermore, the disclosed process may be repeated.

[0127] Although specific aspects have been described herein, the scope of this technology is not limited to these specific aspects. Those skilled in the art will recognize other aspects or modifications within the scope of this invention. Therefore, specific structures, operations, or media are disclosed only as illustrative aspects. The scope of this technology is defined by the appended claims and any equivalents thereof.

[0128] It should also be understood that the embodiments presented herein are not mutually exclusive, and various embodiments can be combined with one embodiment in any of a variety of different ways.

Claims

1. A method, the method comprising: Receive signals associated with physiological functions at implantable medical device systems; The machine learning model determines information associated with the stimulus signal used to stimulate the physiological function based on the signal, wherein the machine learning model is trained based on modeling the physiological effects from the stimulus; as well as The stimulation of the recipient of the implantable medical device system is controlled based on the determined information.

2. The method of claim 1, wherein the physiological function includes hearing, and wherein the machine learning model is trained to minimize the difference between normal activation patterns of neurons in the auditory nerve and stimulus activation patterns generated based on determined information.

3. The method of claim 2, wherein the machine learning model is trained until the cost function indicating the error between the normal activation mode and the stimulus activation mode converges.

4. The method of claim 2, wherein the stimulus activation pattern is generated by the modeling of the physiological effects of the determined information.

5. The method of claim 2, wherein the physiological effect includes the effect of a stimulation signal generated based on the determined information on the cochlea.

6. The method of claim 2, wherein the normal activation mode and the stimulated activation mode each indicate an electroencephalographic representation of the inner hair cell voltage.

7. The method of claim 2, wherein the normal activation mode and the stimulus activation mode each indicate the electroneurographic representation of spikes in the fine structure of the auditory nerve.

8. The method of claim 2, wherein the normal activation mode and the stimulus activation mode each indicate the neuroelectrocardiographic representation of the central processes of the auditory system.

9. The method according to claim 1, further comprising: One or more features are extracted from the signal, wherein the machine learning model determines the information based on the extracted features.

10. The method of claim 1, wherein the machine learning model determines the information based on the activation patterns of neurons in the auditory nerve from an audio signal or an audio signal combined with noise.

11. The method of claim 1, wherein the machine learning model is pre-trained for average physiological function and retrained using recipient-specific data.

12. The method of claim 1, wherein the modeling of the physiological effect is recipient-specific.

13. One or more non-transitory computer-readable storage media, the one or more non-transitory computer-readable storage media comprising instructions that, when executed by one or more processors, cause the one or more processors to: Receive signals associated with physiological functions at implantable medical device systems; The machine learning model determines information about the stimulus signal used to stimulate the physiological function based on the signal, wherein the machine learning model is trained based on modeling the physiological effects from the stimulus; as well as The stimulation of the recipient of the implantable medical device system is controlled based on the determined information.

14. One or more non-transitory computer-readable storage media according to claim 13, wherein the physiological function includes hearing, and wherein the machine learning model is trained to minimize the difference between normal activation patterns of neurons in the auditory nerve and stimulus activation patterns generated based on determined information.

15. One or more non-transitory computer-readable storage media according to claim 14, wherein the stimulus activation pattern is generated by the modeling of the physiological effect of the determined information.

16. One or more non-transitory computer-readable storage media according to claim 14, wherein the physiological effect includes the effect of a stimulation signal generated based on the determined information on the cochlea.

17. An implantable medical device system, the implantable medical device system comprising: Memory used for storing data; as well as One or more processors, wherein the one or more processors are configured to: Receive signals associated with physiological functions; The machine learning model determines information about the stimulus signal used to stimulate the physiological function based on the signal, wherein the machine learning model is trained based on modeling the physiological effects from the stimulus; as well as The stimulation of the recipient of the implantable medical device system is controlled based on the determined information.

18. The implantable medical device system of claim 17, wherein the machine learning model is trained to minimize the difference between normal activation patterns of neurons in the auditory nerve and stimulus activation patterns generated based on the determined information.

19. The implantable medical device system of claim 17, wherein the machine learning model determines the information based on the activation patterns of neurons in the auditory nerve from audio signals.

20. The implantable medical device system of claim 17, wherein the machine learning model determines the information based on activation patterns of neurons in the auditory nerve from an audio signal combined with noise.

Images