Systems and methods for training machine learning models used by processing units in cochlear implant systems.

Abstract

An exemplary system is configured to maintain data representing a machine learning model used in a cochlear implant system and to train the machine learning model. Training may include: applying audio content as training input to the machine learning model, which is configured to apply machine learning heuristics to the audio content to output an electrical signal representing the audio content; applying the electrical signal to a brain processing model, which is configured to output synthesized audio content representing the electrical signal; generating an error metric representing the difference between the audio content and the synthesized audio content; and feeding the error metric back into the machine learning model, which is configured to use the error metric to adjust the machine learning heuristics applied to the audio content.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 873,114, filed July 11, 2019, the entire contents of which are hereby incorporated by reference.

[0003] Background Information

[0004] A cochlear implant system typically includes a processing unit (e.g., a postauricular sound processor) and a cochlear implant configured to be implanted in a hearing-impaired recipient. The processing unit is configured to process audio content (e.g., speech or other sounds) presented to the recipient according to a sound processing strategy to generate stimulation parameters. The processing unit is further configured to send the stimulation parameters to the cochlear implant, which uses the stimulation parameters to generate and apply electrical stimulation representing the audio content to the recipient. In this way, the recipient can perceive the audio content.

[0005] The performance of cochlear implant systems (e.g., the quality of the recipient's audio perception, power consumption, etc.) largely depends on the specific acoustic processing strategy used by the processing unit to process the audio content presented to the recipient. Unfortunately, acoustic processing strategies are often difficult to optimize because they depend on a combination of numerous variables and are challenging to evaluate objectively. In fact, conventional approaches to improving the acoustic processing strategy used by the acoustic processing unit often provide only minor performance gains and sometimes inadvertently degrade performance in certain use cases and / or for a subset of the cochlear implant recipient population. Attached Figure Description

[0006] The accompanying drawings illustrate different embodiments and are part of this specification. The illustrated embodiments are merely examples and do not limit the scope of this disclosure. Throughout the drawings, the same or similar reference numerals denote the same or similar elements.

[0007] Figure 1 The illustration shows an exemplary cochlear implantation system.

[0008] Figure 2 Show Figure 1 An exemplary configuration of a cochlear implantation system.

[0009] Figure 3 Show Figure 1 Another exemplary configuration of the cochlear implantation system.

[0010] Figure 4 The illustration shows an exemplary sound processing strategy employed by the processing unit.

[0011] Figure 5 An exemplary model management system is shown.

[0012] Figure 6 This illustrates an exemplary implementation of operations performed by a model management system to maintain and train machine learning models.

[0013] Figure 7 Show Figure 6 The alternative implementation of the machine learning model training process shown is illustrated.

[0014] Figure 8 An exemplary configuration is shown in which the network interconnects the processing units of the model management system and the cochlear implant system.

[0015] Figures 9A-9C Different implementations are shown, in which the machine learning model is used by the processing unit to implement one or more processing stages.

[0016] Figure 10 The diagram illustrates an exemplary configuration in which the processing unit is configured to implement a charge limiting element.

[0017] Figure 11 The diagram illustrates an exemplary configuration in which the processing unit is configured to further train the machine learning model based on recipient-specific feedback.

[0018] Figure 12-14 The illustration shows an exemplary method.

[0019] Figure 15 The illustration shows an exemplary computing device. Detailed Implementation

[0020] This paper describes systems and methods for training machine learning models used in cochlear implantation systems. For example, as described herein, a model management system can maintain data representing machine learning models used in cochlear implantation systems and train these models by: 1) applying audio content as training input to the machine learning model, which is configured to apply a machine learning heuristic to the audio content to output an electrical signal representing the audio content; 2) applying the electrical signal to a brain processing model, which is configured to output synthesized audio content representing the electrical signal; 3) generating an error metric representing the difference between the audio content and the synthesized audio content; and 4) feeding the error metric back into the machine learning model, which is configured to use the error metric to adjust the machine learning heuristic applied to the audio content.

[0021] In some examples, data representing a trained machine learning model may be sent to or otherwise provided for use by a processing unit within the cochlear implant system. The processing unit may be configured to use the machine learning model to implement one or more processing levels associated with processing audio content received by the processing unit, to generate one or more stimulus parameters representing the audio content.

[0022] Compared to conventional sound processing techniques, the machine learning-based systems and methods described herein offer numerous benefits and advantages. For example, the systems and methods described herein allow for global optimization of sound processing strategies used by the processing unit in a cochlear implant system for optimal performance, fully utilizing the vast amounts of training data available to the manufacturer or other entities associated with the cochlear implant system. Furthermore, the systems and methods described herein simplify or even eliminate the need for certain types of fitting procedures, saving time and resources for clinicians and recipients. Additionally, the systems and methods described herein significantly reduce the amount of power required by the processing unit to process audio content, thereby maintaining battery life and enabling the manufacture of smaller devices. These and other benefits and advantages of the systems and methods described herein will become apparent in this paper.

[0023] Figure 1 The figure illustrates an exemplary cochlear implant system 100 configured for use by a recipient. As shown, the cochlear implant system 100 includes a cochlear implant 102, electrode leads 104, and a processing unit 108. The electrode leads are physically coupled to the cochlear implant 102 and have an array of electrodes 106. The processing unit is configured to be communicatively coupled to the cochlear implant 102 via a communication link 110.

[0024] Figure 1 The cochlear implant system 100 shown is unilateral (i.e., associated with only one ear of the recipient). Alternatively, a bilateral cochlear implant system 100 may include separate cochlear implants and electrode leads for each ear of the recipient. In a bilateral configuration, the processing unit 108 may be implemented by a single processing unit configured to interface with both cochlear implants or by two separate processing units each configured to interface with different cochlear implants.

[0025] The cochlear implant 102 can be implemented by any suitable type of implantable stimulator. For example, the cochlear implant 102 can be implemented by an implantable cochlear stimulator. Alternatively or alternatively, the cochlear implant 102 can be implemented by a brainstem implant and / or any other type of device that is implantable in the recipient and configured to apply electrical stimulation to one or more stimulation sites located along the recipient's auditory pathway.

[0026] In some examples, the cochlear implant 102 may be configured to generate electrical stimulation representing an audio signal (also referred to herein as audio content) processed by the processing unit 108, based on one or more stimulation parameters sent to the cochlear implant 102 by the processing unit 108. The cochlear implant 102 may also be configured to apply electrical stimulation to one or more stimulation sites within the recipient body (e.g., one or more intracochlear locations) via one or more electrodes 106 on electrode leads 104. In some examples, the cochlear implant 102 may include multiple independent current sources, each associated with a channel defined by one or more of the electrodes 106. In this way, different stimulation current levels can be simultaneously applied to multiple stimulation sites via multiple electrodes 106.

[0027] The cochlear implant 102 may additionally or alternatively be configured to generate, store, and / or transmit data. For example, the cochlear implant may use one or more electrodes 106 to record one or more signals (e.g., one or more voltages, impedances, evoked responses in the recipient body, and / or other measurements) and transmit data representing said one or more signals to the processing unit 108 via a communication link 110. In some examples, this data is referred to as back telemetry data.

[0028] The electrode lead 104 can be implemented in any suitable manner. For example, the distal portion of the electrode lead 104 can be pre-bent so that the electrode lead 104 conforms to the spiral shape of the cochlea after implantation. Alternatively, the electrode lead 104 can be naturally straight or have any other suitable configuration.

[0029] In some examples, electrode leads 104 include multiple wires (e.g., within an outer sheath) that conductively connect electrodes 106 to one or more current sources within the cochlear implant 102. For example, if there are n electrodes 106 on electrode leads 104 and n current sources within the cochlear implant 102, then n independent wires may be provided within electrode leads 104, the n independent wires being configured to conductively connect each electrode 106 to a different one of the n current sources. Exemplary values ​​for n are 8, 12, 16, or any other suitable number.

[0030] Electrode 106 is located at least on the distal portion of electrode lead 104. In this configuration, after the distal portion of electrode lead 104 is inserted into the cochlea, electrical stimulation can be applied to one or more intracochlear locations via one or more electrodes 106. One or more other electrodes (e.g., including a ground electrode, not explicitly shown) may also be disposed on other portions of electrode lead 104 (e.g., the proximal portion of electrode lead 104) to provide a current return path for, for example, the stimulation current applied by electrode 106 and to remain outside the cochlea after the distal portion of electrode lead 104 is inserted into the cochlea. Alternatively or additionally, the housing of cochlear implant 102 may serve as a ground electrode for the stimulation current applied by electrode 106.

[0031] Processing unit 108 may be configured to interface with cochlear implant 102 (e.g., to control the cochlear implant and / or receive data from it). For example, processing unit 108 may send commands (e.g., stimulation parameters and / or other types of operating parameters in the form of data words included in a forward telemetry sequence) to cochlear implant 102 via communication link 110. Processing unit 108 may also additionally or alternatively provide operating power to cochlear implant 102 by sending one or more power signals to cochlear implant 102 via communication link 110. Additionally or alternatively, processing unit 108 may also receive data from cochlear implant 102 via communication link 110. Communication link 110 may be implemented by any suitable number of wired and / or wireless bidirectional and / or unidirectional links.

[0032] As shown in the figure, processing unit 108 includes a memory 112 and a processor 114 configured to be selectively and communicatively coupled to each other. In some examples, the memory 112 and processor 114 may be distributed among multiple devices and / or multiple locations, as may be to serve a particular implementation.

[0033] The memory 112 may be implemented by any suitable non-transitory computer-readable medium and / or non-transitory processor-readable medium, such as any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disk drives), ferromagnetic random access memory (“RAM”), and optical disks. Exemplary volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0034] Memory 112 may maintain (e.g., store) executable data used by processor 114 to implement one or more operations described herein as being implemented by processing unit 108. For example, memory 112 may store instructions 116 that can be executed by processor 114 to implement any of the audio content processing and cochlear implantation control operations described herein. Instructions 116 may be implemented by any suitable application, program (e.g., a sound processing program), software, code, and / or other instance of executable data. Memory 112 may also maintain any data received, generated, managed, used, and / or transmitted by processor 114.

[0035] Processor 114 may be configured to implement (e.g., execute instructions 116 stored in memory 112 to implement) various operations for cochlear implant 102.

[0036] For illustration, processor 114 may be configured to control the operation of cochlear implant 102. For example, processor 114 may receive audio signals (e.g., via a microphone, wireless interface (e.g., Bluetooth interface), and / or wired interface (e.g., auxiliary input port) communicatively coupled to processing unit 108). Processor 114 may process the audio signals according to an acoustic processing strategy (e.g., an acoustic processing program stored in memory 112) to generate appropriate stimulation parameters. Processor 114 may then send the stimulation parameters to cochlear implant 102 to guide cochlear implant 102 to apply electrical stimulation representing the audio signal to the recipient.

[0037] In some implementations, processor 114 may also be configured to deliver acoustic stimulation to the receptor. For example, a receiver (also referred to as a loudspeaker) may be optionally coupled to processing unit 108. In this configuration, processor 114 may deliver acoustic stimulation to the receptor via the receiver. The acoustic stimulation may represent an audio signal configured to elicit a response in the receptor and / or otherwise configured (e.g., an amplified version of an audio signal). In a configuration in which processor 114 is configured to both deliver acoustic stimulation to the receptor and guide cochlear implant 102 to deliver electrical stimulation to the receptor, cochlear implant system 100 may be referred to as a bimodal hearing system and / or any other suitable term.

[0038] Processor 114 may additionally or alternatively be configured to receive and process data generated by cochlear implant 102. For example, processor 114 may receive data representing signals recorded by cochlear implant 102 using one or more electrodes 106 and adjust one or more operating parameters of processing unit 108 based on the data. Additionally or alternatively, processor 114 may use the data to perform one or more diagnostic operations on cochlear implant 102 and / or the recipient.

[0039] Other operations may also be performed by processor 114, such as those servicing a particular implementation. In the description provided herein, any reference to operations performed by processing unit 108 and / or any implementation thereof should be understood as being performed by processor 114 based on instructions 116 stored in memory 112.

[0040] The processing unit 108 may be implemented by one or more devices configured to interface with the cochlear implant 102. For illustrative purposes, Figure 2 An exemplary configuration 200 of a cochlear implant system 100 is shown, in which a processing unit 108 is implemented by a sound processor 202 configured to be located outside the recipient body. In configuration 200, the sound processor 202 is communicatively coupled to a microphone 204 and communicatively coupled to a headpiece 206, both of which are configured to be located outside the recipient body.

[0041] The sound processor 202 can be implemented by any suitable device that can be worn or carried by the recipient. For example, the sound processor 202 can be implemented by an behind-the-ear (“BTE”) unit configured to be worn behind and / or on top of the recipient's ear. Alternatively, the sound processor 202 can be implemented by an off-the-ear unit (also known as a body-worn device) configured to be worn or carried by the recipient remotely from the ear. Alternatively, at least a portion of the sound processor 202 is implemented by circuitry within the headpiece 206.

[0042] Microphone 204 is configured to detect one or more audio signals (e.g., including speech and / or any other type of sound) in the recipient's environment. Microphone 204 can be implemented in any suitable manner. For example, microphone 204 can be implemented by a microphone configured to be placed in the outer ear near the entrance to the ear canal, such as the T-MIC from Advanced Bionics. TM Microphone. Such a microphone can be held in the outer ear near the entrance of the ear canal during normal operation by means of a stalk or stem attached to an ear hook, the ear hook being configured to selectively attach to the sound processor 202. Alternatively or additionally, the microphone 204 may also be implemented by one or more microphones in or on the headpiece 206, one or more microphones in or on the housing of the sound processor 202, one or more beamforming microphones, and / or any other suitable microphone that may serve a particular implementation.

[0043] Headpiece 206 may be selectively and communicatively coupled to sound processor 202 via communication link 208 (e.g., cable or any other suitable wired or wireless communication link), which may be implemented in any suitable manner. Headpiece 206 may include an external antenna (e.g., coil and / or one or more wireless communication components) configured to facilitate selective wireless coupling of sound processor 202 to cochlear implant 102. Alternatively or additionally, headpiece 206 may be used to selectively and wirelessly couple any other external device to cochlear implant 102. For this purpose, headpiece 206 may be configured to attach to the recipient's head and positioned such that an external antenna housed within headpiece 206 is communicatively coupled to a corresponding implantable antenna included within or otherwise connected to cochlear implant 102 (which may also be implemented by a coil and / or one or more wireless communication components). In this manner, stimulation parameters and / or power signals may be transmitted wirelessly and transdermally between sound processor 202 and cochlear implant 102 via wireless communication link 210.

[0044] In configuration 200, the sound processor 202 can receive the audio signal detected by the microphone 204 by receiving a signal (e.g., an electrical signal) representing an audio signal from the microphone 204. The sound processor 202 may also receive the audio signal additionally or alternatively via any other suitable interface described herein. The sound processor 202 can process the audio signal in any manner described herein and send stimulation parameters to the cochlear implant 102 via the headpiece 206 to guide the cochlear implant 102 to apply electrical stimulation representing the audio signal to the recipient.

[0045] In an alternative configuration, the sound processor 202 may be implanted within the recipient body rather than located outside the recipient body. In this alternative configuration (which may be referred to as the fully implantable configuration of the cochlear implant system 100), the sound processor 202 and the cochlear implant 102 may be combined into a single device or implemented as separate devices configured to communicate with each other via wired and / or wireless communication links. In the fully implantable implementation of the cochlear implant system 100, the headpiece 206 may be omitted, and the microphone 204 may be implemented by one or more microphones implanted within the recipient body, located within the recipient's ear canal, and / or located outside the recipient body.

[0046] Figure 3 The figure illustrates an exemplary configuration 300 of a cochlear implant system 100, in which the processing unit 108 is implemented by a combination of a sound processor 202 and a computing device 302 configured to be communicatively coupled to the sound processor 202 via a communication link 304, which may be implemented by any suitable wired or wireless communication link.

[0047] The computing device 302 may be implemented by any suitable combination of hardware and software. For illustration, the computing device 302 may be implemented by a mobile device (e.g., a mobile phone, laptop computer, tablet computer, etc.), a desktop computer, and / or any other suitable computing device that serves a particular implementation. As an example, the computing device 302 may be implemented by a mobile device configured to execute an application (e.g., a “mobile app”) that can be used by a user (e.g., a recipient, clinician, and / or any other user) to control one or more settings of the sound processor 202 and / or the cochlear implant 102 and / or to perform one or more operations (e.g., diagnostic operations) on data generated by the sound processor 202 and / or the cochlear implant 102.

[0048] In some examples, computing device 302 may be configured to send one or more commands to cochlear implant 102 via sound processor 202 to control the operation of cochlear implant 102. Similarly, computing device 302 may be configured to receive data generated by cochlear implant 102 via sound processor 202. Alternatively, computing device 302 may directly interface with cochlear implant 102 via a wireless communication link between computing device 302 and cochlear implant 102 (e.g., control the cochlear implant and / or receive data from it). In some implementations where computing device 302 directly interfaces with cochlear implant 102, sound processor 202 may or may not be included in cochlear implant system 100.

[0049] The computing device 302 is shown with an integrated display 306. The display 306 may be implemented by, for example, a screen and may be configured to display content generated by the computing device 302. Alternatively, the computing device 302 may be communicatively coupled to an external display device (not shown) configured to display content generated by the computing device 302.

[0050] In some examples, computing device 302 represents an adapter configured to be selectively used (e.g., by a surgeon) to adapt sound processor 202 and / or cochlear implant 102 to a recipient. In these examples, computing device 302 may be configured to execute an adaptation procedure configured to set one or more operating parameters of sound processor 202 and / or cochlear implant 102 to recipient-optimized values. Therefore, in these examples, computing device 302 may not be considered part of cochlear implant system 100. Alternatively, computing device 302 may also be considered separate from cochlear implant system 100 such that computing device 302 can be selectively coupled to cochlear implant system 100 when it is desired to adapt sound processor 202 and / or cochlear implant 102 to a recipient.

[0051] Figure 4 The figure illustrates an exemplary audio processing strategy employed by processing unit 108. As shown, processing unit 10 can process incoming audio content in multiple processing stages 402 (e.g., processing stages 402-1 to processing stages 402-N). Each processing stage 402 can be implemented in series, such as... Figure 4 As shown in the diagram. Alternatively, two or more processing stages 402 can also be implemented in parallel.

[0052] Each processing stage 402 may represent one or more processing operations performed on audio content received by processing unit 108. For example, an exemplary processing stage may include a preprocessing stage in which the audio content is filtered and / or gain corrected. Another exemplary processing stage may divide the audio signal into multiple analysis channels, each corresponding to a different frequency band, and perform channel-specific operations in each analysis channel. Another exemplary processing stage may include an electrode mapping stage in which patient-specific parameters (e.g., most comfortable stimulation level (M level), threshold level (T level), current guiding parameters, electrode enable and disable parameters, and electrode mapping operations are implemented. Another exemplary processing stage may include an M-choose-N channel selection stage in which N of the total M stimulation channels are selected for use by the cochlear implant to represent the audio content to the recipient. Another exemplary processing stage may include a noise reduction stage in which noise within the audio content is reduced or eliminated. Additional or alternative operations may also be performed in these and / or other processing stages 402, as may be applicable to a particular implementation.

[0053] As described herein, processing unit 108 may be configured to implement one or more of processing levels 402 using machine learning models. To this end, a model management system may maintain, train, and provide processing unit 108 with access to the machine learning models.

[0054] Figure 5 An exemplary model management system 500 (“System 500”) is illustrated. System 500 may be implemented by one or more computing devices (e.g., servers) not included in any cochlear implant system described herein (e.g., remote from any cochlear implant system). For example, System 500 may be implemented by one or more computing devices maintained and / or additionally associated with a manufacturer of a cochlear implant system, a provider of a cochlear implant system, and / or any other entity that can serve a particular implementation.

[0055] As shown in the figure, system 500 includes a memory 502 and a processor 504 configured to be selectively and communicatively coupled to each other. In some examples, the memory 502 and processor 504 may be distributed among multiple devices and / or multiple locations, as may be required for a particular implementation.

[0056] The memory 502 may be implemented by any suitable non-transitory computer-readable medium and / or non-transitory processor-readable medium, such as any combination of non-volatile storage media and / or volatile storage media described herein.

[0057] Memory 502 may maintain (e.g., store) executable data used by processor 504 to implement one or more operations described herein as being implemented by system 500. For example, memory 502 may store instructions 506 that may be executed by processor 504 to implement any of the machine learning model maintenance and training operations described herein. Instructions 506 may be implemented by any suitable application, program, software, code, and / or other instance of executable data. Memory 502 may also maintain any data received, generated, managed, used, and / or sent by processor 504.

[0058] Processor 504 may be configured to implement (e.g., execute instructions 506 stored in memory 502 to implement) various operations for maintaining and training machine learning models for use in one or more cochlear implant systems. In the description provided herein, any reference to system 500 and / or any implementation thereof shall be understood as being implemented by processor 504 based on instructions 506 stored in memory 502.

[0059] Figure 6 An exemplary implementation 600 of operations for maintaining and training a machine learning model, implemented by system 500, is shown. Combined with... Figure 6 The described operations are merely an illustration of the various ways that can be used to maintain and train machine learning models in System 500.

[0060] As shown in the figure, system 500 can maintain data representing machine learning models used in cochlear implant systems. Machine learning model 602 can be configured to apply any suitable machine learning heuristic (also known as artificial intelligence heuristic) to input data, which can be in the time domain or the frequency domain. Machine learning model 602 can be supervised or unsupervised (e.g., serving a specific implementation) and can be configured to implement one or more decision tree algorithms, association rule learning algorithms, artificial neural network learning algorithms, deep learning algorithms, bitmap algorithms, and / or any other suitable data analysis techniques serving a specific implementation.

[0061] In some examples, the machine learning model 602 is implemented by one or more neural networks, such as one or more deep convolutional neural networks (CNNs), recurrent neural networks (RNNs), and / or long / short-term memory neural networks (LSTMs) utilizing the internal memory of their respective kernels (filters). The machine learning model 602 can be multi-layered. For example, the machine learning model 602 can be implemented by a neural network comprising an input layer, one or more hidden layers, and an output layer.

[0062] System 500 can train machine learning model 602 in any suitable manner. For example, system 500 can use an autoencoder scheme that incorporates a brain processing model (e.g., a vocoder) to train machine learning model 602.

[0063] For the purpose of explanation, Figure 6 The system 600 is shown to apply audio content (also referred to herein as "training audio content") as training input to a machine learning model 602. The audio content can include any suitable type of audio content that serves a particular implementation. For example, the audio content can include pre-recorded or live speech, music, noisy audio, clean audio without noise, etc. The audio content can be applied to the machine learning model 602 as a time-domain signal or as a frequency-domain signal.

[0064] The machine learning model 602 can be trained using any number of audio content instances (e.g., different audio samples). Each audio content instance can have any suitable duration. Furthermore, since training the machine learning model 602 does not require human intervention and / or interpretation, the system 600 can apply as much audio content as might be needed (e.g., audio content that can last for hours or even days) to efficiently train the machine learning model 602. Because labeled data may not be required, the machine learning model 602 can be trained using a large number of audio samples available from various sources.

[0065] In some examples, system 500 is configured to preprocess the audio content before it is applied as training input to machine learning model 602. Such preprocessing may include filtering, gain correction, and / or any other preprocessing operations that processing unit 108 can typically perform on the audio content before it is processed by a digital signal processor (DSP). Alternatively, such preprocessing may not be performed, allowing the raw audio content to be applied as training input to machine learning model 602.

[0066] Machine learning model 602 can be configured to apply machine learning heuristics to audio content to output an electrical signal representing the audio content. The machine learning heuristic can be any machine learning heuristic described herein, such as one or more neural network processing heuristics.

[0067] As mentioned, the machine learning model 602 can be configured to output an electrical signal representing audio content. The electrical signal may include one or more electrical stimulation pulses, which the cochlear implant applies to a receptor within the cochlear implant system to represent the audio content to the receptor. In some examples, the machine learning model 602 can generate the electrical signal by generating one or more stimulation parameters that define the electrical signal. For example, the machine learning model 602 can generate data representing amplitude, pulse width, frequency, duration, electrode mapping scheme, etc., which together define various characteristics of the electrical signal.

[0068] As shown in the figure, the electrical signal output by the machine learning model 602 can be applied to the brain processing model 604. The brain processing model 604 can be configured to output synthesized audio content representing the electrical signal. This can be implemented in any suitable manner.

[0069] The brain processing model 604 can be implemented by any suitable model configured to model how the brain of the cochlear implant recipient will process and / or perceive electrical signals representing audio content. For example, the brain processing model 604 can be implemented by a vocoder configured to process the electrical signals output by the machine learning model 602 according to one or more audio synthesis heuristics to output synthesized audio content representing the electrical signals.

[0070] System 500 can be configured to generate an error metric representing the difference between the audio content and the synthesized audio content. For example, such as Figure 6 As shown, the synthesized audio content and the original audio content can be input into the error generation function unit 606. The error generation function unit 606 can be configured to compare the audio content and the synthesized audio content in any suitable manner to output an error metric representing the difference between the audio content and the synthesized audio content.

[0071] For example, the error generation function 606 can implement a cost / error function. For example, the cost function may include, for instance, the total energy of the difference signal or spectrum between the audio content and the synthesized audio content, the maximum peak value of the difference signal or the spectrum of the difference signal, and / or any other norm of the difference signal or spectrum. In some examples, the difference signal may be created such that the synthesized audio content is transformed before the difference is acquired, for example, by frequency shifting based on the expected frequency mismatch caused by the location of the electrodes within the cochlea, as modeled by the brain processing model 604.

[0072] In some examples, the error generation function 606 can generate an error metric based on a model of the perceptible difference between the audio content and the synthesized audio content, for example, by using a psychoacoustic model. The psychoacoustic model can be of any suitable type.

[0073] In some examples, the audio content applied as training input to the machine learning model 602 includes noise. In these examples, a noise-free version of the audio content (rather than the audio content itself) may, in some cases, be input into the error generation function 606 for comparison with the synthesized audio content. In this way, the machine learning model 602 can be trained to reduce noise (e.g., distracting sounds).

[0074] As shown in the figure, system 500 can feed error metrics back into machine learning model 602. Machine learning model 602 can be configured to use the error metrics to adjust the machine learning heuristics applied to the audio content. For example, machine learning model 602 can adjust the machine learning heuristics in a way that causes the machine learning model 602 to output an adjusted electrical signal.

[0075] Combination Figure 6 The described machine learning model training process can continue until the error metric falls below a predetermined threshold, thereby indicating that the machine learning model 602 has learned how to generate electrical signals that accurately and effectively represent the audio content input into the machine learning model 602. The threshold can be set in any suitable manner.

[0076] Figure 7 It shows Figure 6 An alternative implementation 700 of the machine learning model training process is shown. Implementation 700 is similar to implementation 600, except that it includes a charge limiting function 702. As shown, in implementation 700, the electrical signal output by the machine learning model 602 passes through the charge limiting function 702 before being input to the brain processing model 604. The charge limiting function 702 is configured to limit the amount of charge in the electrical signal before it is applied to the brain processing model 604. In this way, system 500 can ensure that the charge of the electrical signal remains within medically safe limits.

[0077] In some examples, system 500 may incorporate a classifier to train machine learning model 602, the classifier being configured to categorize training audio content into specific environments (e.g., quiet speech). In this way, machine learning model 602 can learn how to adapt to the different environments in which the cochlear implant recipient will be located.

[0078] The training process for the machine learning model described in this paper can be recipient-agnostic. In other words, the machine learning model 602 can be trained in a manner that does not take into account the specific characteristics of the cochlear implant recipient and / or hearing profile. More precisely, the machine learning model 602 can be trained to output electrical signals optimized for general cochlear implant recipients.

[0079] In some alternative implementations, system 500 may train machine learning model 602 to be receptor-specific. For example, system 500 may receive data representing specific receptor characteristics and / or hearing profiles and use such data to further train machine learning model 602 to be specifically optimized for the receptor. In some examples, receptor-specific data may include data representing images (e.g., computed tomography (CT) scans of the receptor cochlea) and / or other model data representing the receptor cochlea and / or other anatomical structures.

[0080] In some examples, system 500 may provide machine learning model 602 with an operating power constraint for the cochlear implant system, such that machine learning model 602 can be trained to output a stimulus signal that causes the cochlear implant system to remain within the operating power constraint. This can be implemented in any suitable manner.

[0081] After the machine learning model 602 is trained, the system 500 can provide the processing unit 108 (and / or any other processing unit included in any other cochlear implant system) with access to the machine learning model 602. For example, the system 500 can send data representing the machine learning model 602 to the processing unit 108. This can be implemented in any suitable manner.

[0082] For example, Figure 8 An exemplary configuration configuration 800 is shown, in which network 802 interconnects system 500 and processing unit 108. Network 802 may include a local area network, a wireless network (e.g., Wi-Fi), a wide area network, the Internet, a cellular data network, and / or any other suitable network. Data may flow between components connected to network 802 using any communication technology, device, media, and protocol that can serve a particular implementation.

[0083] In configuration configuration 800, system 500 may be configured to send model data 804 representing machine learning model 602 to processing unit via network 802. This may be implemented in any suitable manner. Processing unit 108 may store model data 804 in local memory (e.g., memory 112) and thereby access machine learning model 602 to implement one or more processing stages 402 as described herein.

[0084] Alternatively, system 500 may provide processing unit 108 with access to machine learning model 602 by sending model data 804 to a computing device (e.g., an adapter used by a clinician), the computing device being configured to load model data 804 onto processing unit 108.

[0085] Alternatively, system 500 may provide processing unit 108 with access to machine learning model 602 by providing one or more application programming interfaces (APIs) to one or more providing processing units 108. These APIs allow processing unit 108 to use machine learning model 602 to process audio content, while machine learning model 602 is remotely maintained by system 500. For example, processing unit 108 may use one or more APIs to send data representing audio content to system 500 via network 802. System 500 may apply data to machine learning model 602 and then send data representing the electrical signals output by machine learning model 602 back to processing unit 108.

[0086] Processing unit 108 may be configured to use machine learning model 602 to implement at least one processing level 402 associated with processing audio content received by processing unit 108. For example, Figures 9A-9C Different implementations are shown, in which the machine learning model 602 is used by the processing unit 108 to implement one or more processing stages 402.

[0087] Specifically, Figure 9A One implementation is shown in which a machine learning model 602 is used to implement processing stage 402-1 (e.g., instead of the processing components conventionally used to implement processing stage 402-1). Figure 9A In this implementation, conventional processing components (e.g., one or more DSPs) can still be used by the processing unit 108 to implement one or more other processing levels (e.g., processing levels 402-2 to processing levels 402-N).

[0088] As Figure 9A As an example of the implementation shown, processing unit 108 may be configured to use machine learning model 602 to perform one or more audio content preprocessing operations (e.g., filtering and / or gain correction) on the audio content received by processing unit 108. The preprocessed audio content can then be processed normally in processing stages 402-2 to 402-N.

[0089] Figure 9B Another implementation is shown, in which a machine learning model 602 is used to implement processing stage 402-2 (e.g., instead of the processing components conventionally used to implement processing stage 402-2). Figure 9B In this implementation, conventional processing components (e.g., one or more DSPs) can still be used by the processing unit 108 to implement one or more other processing levels (e.g., processing level 402-1 and processing level 402-N).

[0090] As Figure 9B In the example of the embodiment shown, processing unit 108 may use processing stage 402-1 to perform one or more preprocessing operations on the audio content received by processing unit 108. Processing unit 108 may then apply the preprocessed audio content to machine learning model 602. In an alternative implementation, the audio content is not preprocessed before being applied to machine learning model 602. In either case, processing unit 108 may use the machine learning model to generate receptor-independent parameter data. Receptor-independent parameter data includes any parameter data that is not specifically configured for use with a receptor. For example, receptor-independent parameter data may include data representing: a noise-reduced version of the audio content, a selection of N from M stimulation channels for use by cochlear implant 102 to represent the audio content to the receptor, and / or any other operations not specific to the receptor.

[0091] Continuing this example, processing unit 108 can apply receptor-independent parameter data to an electrode mapping level configured to determine receptor-specific stimulation parameters based on the receptor-independent parameter data and receptor-specific settings. Such receptor-specific stimulation parameters may include M-level, T-level, current-directing parameters, electrode enable and disable parameters, and / or any other suitable receptor-specific parameters that may serve a particular implementation. These receptor-specific stimulation parameters may constitute one or more stimulation parameters sent to cochlear implant 102.

[0092] Figure 9C Another implementation is shown in which machine learning model 602 is used to implement all of processing stages 402-1 through 402-N. In this implementation, conventional processing components are not used by processing unit 108 to implement any of the processing stages 402. More precisely, machine learning model 602 is used end-to-end to generate one or more stimulation parameters to be sent to cochlear implant 102.

[0093] As Figure 9C As an example of the implementation shown, processing unit 108 may be configured to apply audio content to machine learning model 602 and use machine learning model 602 to generate receptor-specific stimulation parameters constituting one or more stimulation parameters sent to cochlear implant 102. To this end, machine learning model 602 may be configured to implement any audio content processing operations described herein.

[0094] As Figures 9A-9C In another example of any of the implementations shown, processing unit 108 may use machine learning model 602 to determine N of the M stimulation channels to be used by cochlear implant 102 to represent audio content to the receptor. In this example, machine learning model 602 may consider various factors, including energy content within each channel, noise, reverberation, receptor-specific characteristics, and / or other contextual cues that may serve a particular implementation.

[0095] Figure 10 The figure illustrates an exemplary configuration in which processing unit 108 is configured to implement charge limiting element 1002. Charge limiting element 1002 is configured to prevent stimulation parameters generated using machine learning model 602 from causing electrical stimulation applied by cochlear implant 102 to have a charge amount exceeding a threshold (e.g., a medical safety limit). Charge limiting element 1002 can be implemented using any suitable combination of circuitry and processing elements, and can limit charge in any suitable manner.

[0096] Figure 11 The figure illustrates an exemplary configuration in which processing unit 108 is configured to further train machine learning model 602 based on receptor-specific feedback. Receptor-specific feedback may be provided by the receptor and / or another user and may indicate the effectiveness of electrical stimulation applied by cochlear implant 102. Receptor-specific feedback may be subjective (e.g., in the form of a verbal answer to a question about the effectiveness of the electrical stimulation). Alternatively, receptor-specific feedback may also be objective. For example, receptor-specific feedback may take the form of evoked responses that occur in response to electrical and / or acoustic stimulation. Evoked responses may include, for example, electrocochlear electrograph (ECoG) potentials (e.g., cochlear microphonic potentials, action potentials, summation potentials, etc.), auditory nerve responses, brainstem responses, compound action potentials, stapedius reflexes, and / or any other type of neural or physiological response that may occur within the receptor in response to the application of electrical and / or acoustic stimulation. Evoked responses may originate from neural tissue, hair cells to synapses, inner or outer hair cells, or other sources.

[0097] Alternatively, system 500 may train machine learning model 602 based on receptor-specific feedback. For example, processing unit 108 and / or any other computing device may send data representing receptor-specific feedback to system 500. System 500 may then apply the receptor-specific feedback to machine learning model 602 as training input.

[0098] Figure 12The figure illustrates an exemplary method 1200 that can be implemented by a model management system (e.g., system 500 or any implementation thereof, such as at least one computing device). Although Figure 12 Exemplary operation according to one embodiment is shown; however, other embodiments may omit, add, reorder, and / or modify this. Figure 12 Any of the operations shown. Figure 12 Each operation shown can be implemented in any of the ways described herein.

[0099] At operation 1202, the model management system maintains data representing machine learning models used in cochlear implant systems.

[0100] At operation 1204, the model management system trains the machine learning model.

[0101] Figure 13 The figure illustrates an exemplary method 1300, which shows an operation that can be implemented at operation 1204 to train a machine learning model. Figure 13 Each operation shown can be implemented in any of the ways described herein.

[0102] At operation 1302, the model management system applies the audio content as training input to the machine learning model, which is configured to apply machine learning inspiration to the audio content to output an electrical signal representing the audio content.

[0103] At operation 1304, the model management system applies electrical signals to the brain processing model, which is configured to output synthesized audio content representing the electrical signals.

[0104] At operation 1306, the model management system generates an error metric representing the difference between the audio content and the synthesized audio content.

[0105] At operation 1308, the model management system feeds the error metric back into the machine learning model, which is configured to use the error metric to adjust the machine learning heuristics applied to the audio content.

[0106] Figure 14 An exemplary method 1400 is illustrated, which can be implemented by a processing unit communicatively coupled to a cochlear implant. Although Figure 14 The figure illustrates exemplary operation according to one embodiment; however, other embodiments may omit, add, reorder, and / or modify these features. Figure 14 Any of the operations shown. Figure 14 Each operation shown can be implemented in any of the ways described herein.

[0107] At operation 1402, the processing unit receives audio content.

[0108] At operation 1404, the processing unit accesses data representing the machine learning model.

[0109] At operation 1406, the processing unit uses a machine learning model to implement at least one processing level associated with the processed audio content to generate one or more stimulus parameters representing the audio content.

[0110] At operation 1408, the processing unit sends one or more stimulation parameters to the cochlear implant.

[0111] In some examples, a non-transitory computer-readable medium for storing computer-readable instructions can be provided based on the principles described herein. These instructions, when executed by a processor of a computing device, can instruct the processor and / or the computing device to perform one or more operations, including one or more operations described herein. Such instructions can be stored and / or transmitted using any of a variety of known computer-readable media.

[0112] As mentioned herein, non-transitory computer-readable media may include any non-transitory storage medium that contributes to providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., by the processor of the computing device). For example, non-transitory computer-readable media may include, but is not limited to, any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferromagnetic random access memory (“RAM”), and optical discs (e.g., optical discs, digital video discs, Blu-ray discs, etc.). Exemplary volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0113] Figure 15 The figure illustrates an exemplary computing device 1500, which may be specifically configured to implement one or more processes described herein. For this purpose, any of the systems, processing units, and / or devices described herein may be implemented by computing device 1500.

[0114] like Figure 15 As shown, computing device 1500 may include a communication interface 1502, a processor 1504, a storage device 1506, and an input / output (“I / O”) module 1508 that are communicatively connected to each other via communication infrastructure 1510. Although the exemplary computing device 1500... Figure 15 It is shown in the middle, however Figure 15 The components illustrated are not intended to be limiting. Additional or alternative components may be used in other embodiments. These will now be described in further detail. Figure 15 The components of the computing device 1500 shown.

[0115] Communication interface 1502 can be configured to communicate with one or more computing devices. Examples of communication interface 1502 include, but are not limited to, wired network interfaces (e.g., network interface cards), wireless network interfaces (e.g., wireless network interface cards), modems, audio / video connections, and any other suitable interfaces.

[0116] Processor 1504 generally represents any type or form of processing unit capable of processing data and / or interpreting, executing, and / or directing the execution of one or more of the instructions, procedures, and / or operations described herein. Processor 1504 may operate by executing computer-executable instructions 1512 (e.g., application programs, software, code, and / or other executable data instances) stored in storage device 1506.

[0117] Storage device 1506 may include one or more data storage media, devices, or configurations and may take the form of data storage media and / or devices of any type, form, and combination. For example, storage device 1506 may include, but is not limited to, any combination of non-volatile media and / or volatile media described herein. Electronic data, including the data described herein, may be temporarily and / or permanently stored in storage device 1506. For example, data representing computer-executable instructions 1512 configured to boot processor 1504 to perform any of the operations described herein may be stored within storage device 1506. In some examples, data may be arranged in one or more databases residing within storage device 1506.

[0118] I / O module 1508 may include one or more I / O modules configured to receive user input and provide user output. I / O module 1508 may include any hardware, firmware, software, or combinations thereof that support input and output functions. For example, I / O module 1508 may include hardware and / or software for capturing user input, including but not limited to a keyboard or keypad, a touch screen component (e.g., a touch screen display), a receiver (e.g., an RF or infrared receiver), a motion sensor, and / or one or more input buttons.

[0119] I / O module 1508 may include one or more means for presenting output to a user, including but not limited to a graphics engine, a display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O module 1508 is configured to provide graphics data to the display for presentation to a user. The graphics data may represent one or more graphical user interfaces and / or any other graphical content that may serve a particular implementation.

[0120] Various exemplary embodiments have been described in the preceding description with reference to the accompanying drawings. However, it will be apparent that various modifications and variations can be made to the described embodiments without departing from the scope of the invention as set forth in the appended claims, and other embodiments may be implemented. For example, certain features of one embodiment described herein may be combined with or substitute for features of another embodiment described herein. Accordingly, the description and drawings should be viewed in an illustrative rather than restrictive sense.

Claims

1. A model management system for use with a cochlear implant system, comprising: a memory storing instructions; a processor communicatively coupled to the memory and configured to execute the instructions to: maintain data representative of a machine learning model for use in the cochlear implant system; and train the machine learning model by applying audio content as a training input to the machine learning model, the machine learning model configured to apply machine learning heuristics to the audio content to output an electrical signal representative of the audio content, the electrical signal comprising one or more electrical stimulation pulses to represent the audio content; applying the electrical signal to a brain processing model, the brain processing model configured to output synthesized audio content representative of the electrical signal; generating an error metric representative of a difference between the audio content and the synthesized audio content; and feeding the error metric back into the machine learning model, the machine learning model configured to use the error metric to adjust the machine learning heuristics applied to the audio content; an acoustic processor; and a cochlear implant configured to be implanted in a recipient and communicatively coupled to the acoustic processor by way of a wireless communication link; wherein the acoustic processor is configured to: receive an audio signal, use the machine learning model to implement at least one processing stage with respect to the audio signal to generate one or more stimulation parameters, and send the one or more stimulation parameters to the cochlear implant by way of the wireless communication link; and wherein the cochlear implant is configured to: receive the one or more stimulation parameters by way of the wireless communication link, and apply electrical stimulation representative of the audio signal to the recipient based on the one or more stimulation parameters. The training continues until the error metric is below a predetermined threshold. The processor is further configured to pre-process the audio content before the audio content is applied as a training input to the machine learning model.

2. The system of claim 1, wherein, The machine learning model is implemented by a multi-layer neural network.

3. The system of claim 1, wherein, The brain processing model is implemented by a vocoder.

4. The system of claim 1, wherein, Training the machine learning model further comprises limiting an amount of charge of the electrical signal before the electrical signal is applied to the brain processing model.

5. The system of claim 1, wherein, The processor is further configured to execute the instructions to send the data representative of the machine learning model to the acoustic processor.

6. The system of claim 1, wherein, The sending comprises sending the data representative of the machine learning model to the acoustic processor via a network interconnecting the system and the acoustic processor.

7. The system of claim 1, wherein, The sending comprises sending the data representative of the machine learning model to a computing device configured to load the data representative of the machine learning model onto the acoustic processor.

8. The system of claim 7, wherein, ​ 9. The system of claim 7, wherein, ​

Images