Remote testing validation system for a medical device

Abstract

Systems and methods for tracking task focus of a recipient of a medical device during unsupervised remote self-testing of the medical device are disclosed herein. One or more signals perceptible to the recipient are transmitted to an external device for presenting signals to the recipient. The recipient then inputs a response at the external device, wherein the input is responsive to the one or more signals perceived by the recipient. The external device then measures, at a plurality of time intervals, at least one eye tracking data metric of the recipient in association with the one or more signals and the input received from the recipient at the external device. At least one focus tracking metric of the recipient is determined based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient at the external device.

Description

CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))REMOTE TESTING VALIDATION SYSTEM FOR A MEDICAL DEVICEInventors: Geert De Ceulaer and Weiyi WangTECHNICAL FIELD

[0001] The technical field relates generally to systems and methods for measuring task focus of recipients during remote testing of medical devices.BACKGROUND

[0002] Medical devices are devices that are intended to be used for medical purposes and can provide a wide range of therapeutic benefits to recipients. They can vary in both their intended use and indications for use. Examples of medical devices range from simple, low-risk medical supplies to complex, potentially high-risk devices that are implanted and / or sustain life, such as deep brain stimulators and brain-computer interfaces. Other categories of medical devices include diagnostic equipment. Medical devices can include internal / implantable components / devices, external or wearable components / devices, or combinations thereof (e.g., a device having an external component communicating with an internal component). Implantable medical devices now often include one or more instruments, apparatus, sensors, processors, controllers, or other functional mechanical or electrical components that are permanently or temporarily implanted in a recipient. These functional devices are typically used to diagnose, prevent, monitor, treat or manage a disease / injury or symptom thereof, or to investigate, replace, or modify the anatomy or a physiological process. Many of these functional devices utilize power and / or data received from external devices that are part of or operate in conjunction with implantable components.

[0003] Hearing devices are devices that act on an actual or potential auditory perception of an individual, including to improve perception of sound signals, to reduce perception of sound signals, etc. In particular, a hearing device can deliver sound signals to a user in any form, including in the form of acoustical stimulation, mechanical stimulation, electrical stimulation, etc., and / or can operate to suppress all or some sound signals. As such, a hearing device can be a medical device for use by a hearing-impaired person (e.g., hearing aids, partially or fullyCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))implantable hearing prostheses, middle ear auditory prostheses, bone conduction devices, direct acoustic stimulators, electro-acoustic hearing prostheses, auditory brainstem stimulators, bimodal hearing prostheses, bilateral hearing prostheses, dedicated tinnitus therapy devices, tinnitus therapy devices, combinations or variations thereof, etc.) or a device for use by a person with normal hearing (e.g., a consumer device that provides audio streaming, a consumer headphone, an earphone, etc.), a hearing protection device (e.g., a noise cancellation headset, a loudness reduction apparatus, etc.), etc.SUMMARY

[0004] In some embodiments, various systems, and methods for tracking task focus of a recipient of a medical device during unsupervised remote self-testing of the medical device are disclosed herein. One or more signals perceptible to the recipient are transmitted to an external device for presenting signals to the recipient. The recipient then inputs a response at the external device, wherein the input is responsive to the one or more signals perceived by the recipient. The external device then measures, at a plurality of time intervals, at least one eye tracking data metric of the recipient in association with the one or more signals and the input received from the recipient at the external device. At least one focus tracking metric of the recipient is determined based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient at the external device.

[0005] In another aspect, embodiments of the disclosed invention comprise a system for tracking task focus of a recipient of a device during remote self-testing of the device by the recipient. The system comprises at least one camera and a user interface comprising at least a display. The device further comprises a processor communicatively coupled to the camera and the user interface and configured to transmit one or more signals perceptible to the recipient via a first device. The processor is further configured to receive input from the recipient at the user interface, wherein the input is responsive to one or more signals perceived by the recipient. The processor then proceeds to measure, substantially in real time at a plurality of time intervals, at least one eye tracking data metric captured by the camera in association with the one or more signals and the input received from the recipient. At least one focus tracking metric is determinedCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient.

[0006] In another aspect, embodiments of the disclosed invention comprise a non-transitory computer readable medium comprising one or more computer readable instructions, which upon execution by the one or more processors, perform several operations, including transmitting one or more signals perceptible to a recipient via a first device. An external device of the recipient receives input from the recipient responsive to the one or more signals perceived by the recipient. The external device then measures, at a plurality of time intervals and substantially in real time, at least one eye tracking data metric of the recipient in association with the one or more signals and the input received from the recipient at the external device. At least one focus tracking metric of the recipient is determined based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient at the external device.

[0007] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Various objects, features, aspects, and advantages of the inventive subject matter will become more apparent from the following Detailed Description, along with the accompanying drawing figures in which like numerals represent like components.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] For the purposes of illustration only, several aspects of embodiments of the invention are described by reference to the following figures. In the following figures, the same number represents the same type of element in all drawings.

[0009] FIG. 1 A is a schematic diagram illustrating a cochlear implant system with which aspects of technologies presented herein can be implemented;CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0010] FIG. IB is a side view of a recipient wearing a sound processing unit of the cochlear implant system of FIG. 1A;

[0011] FIG. 2 is a schematic view of components of the cochlear implant system of FIG.1A;

[0012] FIG. 3 A illustrates a schematic diagram of an exemplary implantable medical device system configured to implement aspects of technologies presented herein in conjunction with an implantable auditory prosthesis system;

[0013] FIG. 3B illustrates a block diagram of the exemplary implantable medical device system of FIG. 3 A;

[0014] FIG. 4 illustrates a block diagram of an example implantable auditory prosthesis system configured to implement aspects of technologies described herein;

[0015] FIG. 4A illustrates a schematic diagram of a device configured to implement aspects of technologies presented and described in conjunction with the example implantable auditory prosthesis system of FIG. 4;

[0016] FIG. 5 illustrates a block diagram of an exemplary system and method configured to implement aspects of technologies described herein;

[0017] FIG. 6 illustrates a continuation of the block diagram of the exemplary system and method depicted in FIG. 5; and

[0018] FIG. 7 illustrates an exemplary flow diagram of an example method of validating remote testing of a device as described in aspects of technologies presented herein.

[0019] While the invention is described with reference to the above drawings, the drawings are intended to be illustrative, and other embodiments are consistent with and within the scope of the invention.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))DETAILED DESCRIPTION

[0020] Hearing loss is one of the main contributors to the global burden of disease. Hearing devices, such as cochlear implants (Cis), can help to significantly improve hearing function for speech understanding and quality of life for those with significant hearing loss, sensorineural or otherwise. To ensure that recipients of CI or other hearing devices continue to maintain stable outcomes, follow-up clinical appointments are typically conducted frequently in the first year after surgical implantation of the CI device or other receipt of a hearing device by the recipient, and then every six months or annually thereafter, for as long as the recipient uses the device, which is generally for many years after receipt or implantation, and is often lifelong. However, patient compliance to recommended health care visits to such specialists is hampered by difficulties in traveling to the specialized medical facilities where such follow-up clinical appointments are conducted. Such difficulties in traveling could be due to the long travel distances required to reach the medical facilities, lack of available transportation options or infrastructure to access the medical facilities, and / or lack of qualified health care (e.g., audiological) professionals in the vicinity of the recipient’s home.

[0021] Telehealth options greatly improve access for patients. Many common telehealth services provided are synchronous services, where a clinician conducts a care session remotely with a recipient by means of an audio / video call in real time. Such synchronous services, however, can take the same overall amount of time as an in-clinic appointment. Synchronous services can also require specialized programming hardware to be available at the remote site, along with uninterrupted internet connectivity for video conferencing. The clinician may require control of the remote computer in order to conduct the telehealth session, which may require specialized software or equipment. For these reasons, synchronous services are often conducted at a medical facility situated closer to the recipient, but seldom at the recipient’s home. Thus, more flexible telehealth options are needed, such as remote care in the form of asynchronous services, where the recipient completes certain tasks or tests by themselves at their convenience and then sends the results to their clinician for off-line analysis. However, such unsupervised remote self-testing generates performance results that are difficult to evaluate if they do not contain information on the focus of the recipient on the task during the test. Methods of tracking the focus of the recipient during remote self-testing activities are therefore needed.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0022] Given the above, it is important to note that there are a number of different types of devices in / with which the techniques and examples presented herein can be implemented.Merely for ease of description, the techniques and examples presented herein are primarily described with reference to a specific device, such as a CI. However, it is to be appreciated that the techniques and examples presented herein can also be partially or fully implemented by any of a number of different types of devices or systems, including hearing devices, implantable medical devices, consumer electronic devices (e.g., consumer hearing devices, consumer computing devices such as mobile phones and tablets, consumer wearable devices such as smart watches, audio equipment such as home theatre and car audio systems, etc.), computing systems (e.g., servers in data centers, Internet-of-Things (loT) devices), various types of software systems, such as databases, machine learning and artificial intelligence systems, other medical devices, such as diagnostic equipment or life sustaining equipment, etc. For example, the techniques and examples presented herein can be used in or with sensory prostheses, including hearing aids and cochlear implants, and various medical devices, such as pacemakers, drug delivery systems, implantable defibrillators, functional electrical stimulation devices, seizure devices (e.g., devices for monitoring and / or treating epileptic events), sleep disorder devices (e.g., sleep apnea devices), balance and / or movement disorder devices (e.g., vestibular stimulation devices), tinnitus management devices, visual implants (e.g., bionic eyes), and other neuromodulation devices (e.g., brain-computer interfaces).

[0023] FIGs. 1A-1B and FIG. 2 illustrate an example cochlear implant system 102 with which aspects of the embodiments presented herein can be implemented. The cochlear implant system 102 comprises an external component 104 that is configured to be directly or indirectly attached to the body of the user, and an internal / implantable component 112 that is configured to be implanted in or worn on the head of the user. In the examples of FIGs. 1 A-1B and FIG. 2, the implantable component 112 is sometimes referred to as a “cochlear implant.” FIG. 1 A illustrates the cochlear implant 112 implanted in the head 154 of a user, while FIG. IB is a schematic drawing of the external component 104 worn on the head 154 of the user. FIG. 2 is another schematic view of the cochlear implant system 102. For ease of description, FIGs. 1A-1B and FIG. 2 will generally be described together. The subcranial components of an implantableCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))electrode array as discussed with respect to examples of the disclosed invention are not visible in FIGs. 1A-1B and 2.

[0024] In the examples of FIGs. 1A-1B and FIG. 2, the external component 104 comprises a sound processing unit 106, and generally, a magnet 150 fixed relative to an external coil. The cochlear implant 112 includes an implantable coil 114, and an elongate stimulating assembly 116 configured to be implanted in the user’s cochlea. In one example, the sound processing unit 106 is an off-the-ear (OTE) sound processing unit, sometimes referred to herein as an OTE component, which is configured to send data and power to the implantable component 112. In general, an OTE sound processing unit is a component having a generally cylindrically shaped housing 111 and which is configured to be magnetically coupled to the user’s head 154 (e.g., includes an integrated external magnet 150 configured to be magnetically coupled to an internal / implantable magnet 152 in the implantable component 112). The OTE sound processing unit 106 also includes an integrated external (headpiece) coil 108 (the external coil 108) that is configured to be inductively coupled to the implantable coil 114.

[0025] It is to be appreciated that the OTE sound processing unit 106 is merely illustrative of the external devices that can operate with implantable component 112. For example, in alternative examples, the external component 104 comprises a behind-the-ear (BTE) sound processing unit configured to be attached to, and worn adjacent to, the recipient’s ear. In general, a BTE sound processing unit comprises a housing that is shaped to be worn on the outer ear of the user and is connected to the separate external coil assembly via a cable, where the external coil assembly is configured to be magnetically and inductively coupled to the implantable coil 114. It is also to be appreciated that alternative external components can be located in the user’s ear canal, worn on the body, etc.

[0026] Although the cochlear implant system 102 includes the sound processing unit 106 and the cochlear implant 112, as described below, the cochlear implant 112 can operate independently from the sound processing unit 106, for at least a period, to stimulate the user. For example, the cochlear implant 112 can operate in a first general mode, sometimes referred to as an “external hearing mode,” in which the sound processing unit 106 captures sound signals which are then used as the basis for delivering stimulation signals to the user. The cochlear implant 112 can alsoCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))operate in a second general mode, sometimes referred as an “invisible hearing” mode, in which the sound processing unit 106 is unable to provide sound signals to the cochlear implant 112 (e.g., the sound processing unit 106 is not present, the sound processing unit 106 is powered-off, the sound processing unit 106 is malfunctioning, etc.). As such, in the invisible hearing mode, the cochlear implant 112 captures sound signals itself via implantable sound sensors and then uses those sound signals as the basis for delivering stimulation signals to the user. It is to be appreciated that reference to the external hearing mode and the invisible hearing mode is merely illustrative and that the cochlear implant 112 can also operate in alternative modes.

[0027] In FIGs. 1 A-1B and FIG. 2, the cochlear implant system 102 is shown with an external device 110, configured to implement aspects of the embodiments presented. The external device 110, is a computing device, such as a personal computer (e.g., laptop, desktop, tablet), a mobile phone (e.g., smartphone), remote control unit, etc. The external device 110 and the cochlear implant system 102 (e.g., sound processing unit 106 or the cochlear implant 112) wirelessly communicate via a bi-directional communication link 126. The bi-directional communication link 126 comprises, for example, a short-range communication, such as Bluetooth link, Bluetooth Low Energy (BLE) link, a proprietary link, etc.

[0028] FIG. 3A is a schematic diagram of an exemplary neural processing and stimulation system presented herein in conjunction with exemplary cochlear implant system 300 that can be configured to implement aspects of the technologies presented herein, while FIG. 3B is a block diagram of the exemplary neural processing and stimulation system shown in FIG. 3A. For ease of illustration, FIGs. 3A and 3B will be described together. Although aspects of the technologies presented herein are discussed in the context of the exemplary cochlear implant 300 shown in FIGs. 3A and 3B, it is understood that the technologies described herein can be implemented in any of a wide range of implantable medical devices, including devices that include a transcutaneous closely coupled induction link used in many implantable medical devices. Other examples of implantable medical devices that can utilize the technologies described herein include retinal implants, implantable devices for tinnitus treatment such as middle ear implantable devices, implantable medical devices for vestibular stimulation, or implantable medical devices for deep brain stimulation. For retinal implants, electrodes can be positioned on or near the retinal ganglion cell layer or the optic nerve to stimulate visual perception in patientsCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))with degenerative retinal diseases. In the case of tinnitus treatment, electrodes can be strategically placed in the cochlear nucleus or on auditory nerve fibers to modulate aberrant neural activity associated with tinnitus. Vestibular implants can utilize electrodes placed on or near the vestibular nerve branches or semicircular canals to restore balance and spatial orientation in patients with vestibular dysfunction. For deep brain stimulation, electrodes are typically implanted in target regions such as the subthalamic nucleus, globus pallidus, or ventral intermediate nucleus of the thalamus to alleviate symptoms in neurological conditions like Parkinson's disease, essential tremor, or dystonia.

[0029] The exemplary cochlear implant 300 depicted in FIG. 3A comprises an external component 302 and an internal / implantable component 304. The external component 302 is directly or indirectly attached to the body of the recipient as described above and typically comprises an external coil 306 and, generally, a magnet (not shown in FIGs. 3A or 3B) fixed relative to the external coil 306. The external component 302 also comprises one or more input elements / devices 313 for receiving input signals at a sound processing unit 312. In this example, the one or more one or more input devices 313 include sound input devices 308 (e.g., microphones positioned by auricle 310 of the recipient, telecoils, etc.) configured to capture / receive input signals, one or more auxiliary input devices 309 as shown in FIG. 3B (e.g., audio ports, such as a Direct Audio Input (DAI), data ports, such as a Universal Serial Bus (USB) port, cable port, etc.), and a wireless transmitter / receiver (transceiver) 311 as shown in FIG. 3B, each located in, on, or near the sound processing unit 312.

[0030] With reference to FIG. 3B, the sound processing unit 312 also includes, for example, at least one power source 307 (e.g., a battery), a radio-frequency (RF) transceiver 321, and a processing module 325. The processing module 325 comprises a number of elements, including an environmental classifier 331, a sound processor 333, and an individualized own voice detector 334. Each of the environmental classifier 331, the sound processor 333, and the individualized own voice detector 334 can be formed by one or more processors (e.g., one or more Digital Signal Processors (DSPs), one or more uC cores, etc.), firmware, software, etc. arranged to perform operations described herein. That is, the environmental classifier 331, the sound processor 333, and the individualized own voice detector 334 can each be implemented asCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))firmware elements, partially or fully implemented with digital logic gates in one or more application-specific integrated circuits (ASICs), partially or fully in software, etc.

[0031] The individualized own voice detector 334 includes a decision tree, sometimes referred to herein as an own voice detection decision tree, which can be trained / updated. Similarly, the environmental classifier 331 includes a decision tree, sometimes referred to as an environmental classifier decision tree that, in certain embodiments, can also be trained / updated. To provide the ability to train / update the own voice detection decision tree and / or the environmental classifier decision tree, the decision trees are stored in volatile memory and exposed to, for example, other processes for updating thereof. As such, the environmental classifier 331 and the individualized own voice detector 334 are at least partially implemented in volatile memory.

[0032] In the examples of FIGs. 3 A and 3B, the sound processing unit 312 is a behind-the-ear (BTE) sound processing unit configured to be attached to, and worn adjacent to, the recipient’s ear. However, it is to be appreciated that embodiments of the present invention can be implemented by sound processing units having other arrangements, such as by an OTE processing unit as described above with respect to FIGs. 1 A-1B and FIG. 2 (e.g., a component having a generally cylindrical shape and which is configured to be magnetically coupled to the recipient’s head), etc., a mini or micro-BTE unit, an in-the-canal unit that is configured to be located in the recipient’s ear canal, a body- worn sound processing unit, etc.

[0033] Returning to the example embodiment of FIGs. 3A and 3B, the implantable component 304 comprises an implant body (main module) 314, one or more lead regions 316, and an intra-cochlear stimulating assembly 318, all configured to be implanted under the skin / tissue (tissue) 305 of the recipient. The implant body 314 generally comprises a hermetically sealed housing 315 in which RF interface circuitry 324, and a stimulator unit 320 are disposed. The implant body 314 also includes an internal / implantable coil 322 that is generally external to the housing 315, but which is connected to the RF interface circuitry 324 via a hermetic feedthrough (not shown in FIG. 3B).

[0034] As noted, stimulating assembly 318 is configured to be at least partially implanted in the recipient’s cochlea 337. Stimulating assembly 318 includes a plurality of longitudinally spaced intra-cochlear electrical stimulating contacts (electrodes) 326 that collectively form a contact orCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))electrode array 328 for delivery of electrical stimulation (current) to the recipient’s cochlea. Stimulating assembly 318 extends through the temporal bone 319 to an opening in the recipient’s cochlea 337 (e.g., cochleostomy 332, the round window 334, etc.) and has a proximal end connected to stimulator unit 320 via lead region 316 and a hermetic feedthrough (not shown in FIG. 3B). Lead region 316 includes a plurality of conductors (wires) that electrically couple the electrodes 326 to the stimulator unit 320.

[0035] As noted, the cochlear implant 300 includes the external coil 306 and the implantable coil 322. The coils 306 and 322 are typically wire antenna coils each comprised of multiple turns of electrically insulated single-strand or multi-strand platinum or gold wire. Generally, a magnet is fixed relative to each of the external coil 306 and the implantable coil 322. The magnets fixed relative to the external coil 306 and the implantable coil 322 facilitate the operational alignment of the external coil with the implantable coil through the skin and / or tissue 305 of the recipient. This operational alignment of the coils 306 and 322 enables the external component 302 to transmit data, as well as possibly power, to the implantable component 304 via a closely coupled wireless link formed between the external coil 306 with the implantable coil 322. In certain examples, the closely coupled wireless link is a radio frequency (RF) link. However, various other types of energy transfer, such as infrared (IR), electromagnetic, capacitive, and inductive transfer, can be used to transfer the power and / or data from an external component to an implantable component and, as such, FIG. 3B illustrates only one example arrangement.

[0036] As noted above, sound processing unit 312 includes the processing module 325. The processing module 325 is configured to convert input audio signals into stimulation control signals 336 for use in stimulating a first ear of a recipient (i.e., the processing module 325 is configured to perform sound processing on input audio signals received at the sound processing unit 312). Stated differently, the sound processor 333 (e.g., one or more processing elements implementing firmware, software, etc.) is configured to convert the captured input audio signals into stimulation control signals 336 that represent electrical stimulation for delivery to the recipient. The input audio signals that are processed and converted into stimulation control signals can be audio signals received via the sound input devices 308, signals received via the auxiliary input devices 309, and / or signals received via the wireless transceiver 311.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0037] In the embodiment of FIG. 3B, the stimulation control signals 336 are provided to the RF transceiver 321, which transcutaneously transfers the stimulation control signals 336 (e.g., in an encoded manner) to the implantable component 304 via external coil 306 and implantable coil 322. That is, the stimulation control signals 336 are received at the RF interface circuitry 324 via implantable coil 322 and provided to the stimulator unit 320. The stimulator unit 320 is configured to utilize the stimulation control signals 336 to generate electrical stimulation signals (e.g., current signals) for delivery to the recipient’s cochlea via one or more stimulating contacts (electrodes) 326. In this way, cochlear implant 300 electrically stimulates the recipient’s auditory nerve cells, bypassing absent or defective hair cells that normally transduce acoustic vibrations into neural activity, in a manner that causes the recipient to perceive one or more components of the input audio signals.

[0038] As noted, in addition to the sound processor 333, the processing module 325 also includes the environmental classifier 331. As described further below, the environmental classifier 331 (e.g., one or more processing elements implementing firmware, software, etc.) is configured to determine an environmental classification of the sound environment (i.e., determines the “class” or “category” of the sound environment) associated with the input audio signals received at the cochlear implant 300. In addition, also as described further below, the processing module 325 comprises the individualized own voice detector 335 (e.g., one or more processing elements implementing firmware, software, etc.) that is configured to perform individualized own voice detection (OVD). As used herein, own voice detection (OVD) generally refers to a process in which speech signals received at a hearing prosthesis are classified as either including the speech of the recipient of the hearing prosthesis (referred to herein as the recipient’s own voice or simply own voice) or speech generated by one or more persons other than the recipient (referred to herein as external voice). Also as used herein, individualized own voice detection (or individualized OVD) refers to own voice detection that is recipient-specific, meaning the own voice detection is at least partly trained to perform the own voice detection using (based on) the specific voice (speech) of the recipient of the hearing prosthesis, as captured by the hearing prosthesis itself. As a result, the individualized own voice detection is specific / customized to the recipient of the hearing prosthesis and to the hearing prosthesis itself.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))Example auditory prosthesis system

[0039] FIG. 4 illustrates an example auditory prosthesis system 400 that includes an auditory prosthesis 410 can benefit from the use of technologies described herein. The system 400 further includes a recipient computing device 420, a clinician computing device 430, and a server 440, which are connected over a network 402. The network 402 is a computer network, such as the Internet, which facilitates the communication of data among computing devices connected to the computer network.

[0040] As illustrated, the auditory prosthesis 410 and the recipient computing device 420 are operated by the recipient in an environment 401. The environment 401 defines the conditions in which the auditory prosthesis 410 and the recipient computing device 420 operate. In many examples herein, the environment 401 includes the sonic conditions in which the auditory prosthesis 410 functions. Such sonic conditions can include, for example, a loudness of noise (e.g., whether the environment 401 is loud or quiet), a number of sources of noise (e.g., a crowded restaurant with many sources of noise or a one-on-one conversation with fewer sources of noise) and a kind of noise (e.g., music or speech). The environment 401 can also define an activity in which the recipient is engaged, such as a conversation or exercise. The environment 401 can affect the operation of the auditory prosthesis 410, and the auditory prosthesis 410 can be customized to operate differently in different environments 401. In addition, the environment 401 can affect the focus of the recipient when the recipient performs one or more specific tasks.

[0041] The auditory prosthesis 410 is a medical apparatus relating to a recipient’s auditory system, such as a cochlear implant or bone conduction devices (e.g., percutaneous bone conduction devices, transcutaneous bone conduction devices, active bone conduction devices, and passive bone conduction devices), and middle ear stimulators, among others. The auditory prosthesis 410 can take any of a variety of forms. In the illustrated example, the auditory prosthesis includes an auditory prosthesis sensor set 412 and operates according to auditory prosthesis settings 414.

[0042] The auditory prosthesis sensor set 412 is a collection of one or more hardware or software components of the auditory prosthesis 410 that obtain data, such as data regarding the environment 401, the auditory prosthesis 410, or the recipient. In many examples, the auditoryCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))prosthesis sensor set 412 include a microphone (e.g., an implanted or external microphone). The auditory prosthesis sensor set 412 can include one or more other sensors, such as one or more accelerometers, gyroscopic sensors, location sensors, telecoils, biosensors (e.g., heart rate or blood pressure sensors), and light sensors, among others. The auditory prosthesis sensor set 412 can include components disposed within a housing of the auditory prosthesis 410 as well as devices electrically coupled to the auditory prosthesis 410 (e.g., via wired or wireless connections). In examples, the auditory prosthesis sensor set 412 includes a remote device connected to the auditory prosthesis 410 via an FM (Frequency Modulation) connection, such as a remote microphone (e g., a COCHLEAR TRUE WIRELESS MINI MICROPHONE2+), a television audio streaming device, or a phone clip device, among other devices having FM transmission capabilities. The auditory prosthesis sensor set 412 can further include sensors that obtain data regarding usage of the auditory prosthesis 410, such as software sensors operating on the auditory prosthesis 410 that track: when the auditory prosthesis 410 is worn by the recipient, when the auditory prosthesis 410 (e.g., an external portion thereof) is removed from the recipient, when one or more of the auditory prosthesis settings 414 are modified, and how long the auditory prosthesis 410 is operated using particular settings of the auditory prosthesis settings 414, among other data.

[0043] In examples, the auditory prosthesis sensor set 412 can further include a scene classifier. A scene classifier is a hardware- or software-implemented classifier that obtains data regarding the environment 401 (e.g., from one or more other sensors of the auditory prosthesis sensor set 412) and determines a classification of the environment 401. Classifications can include, for example, speech, noise, and music, among other classifications. The auditory prosthesis 410 can then use the classification to automatically switch the auditory prosthesis settings 414 to suit the environment 401. An example scene classifier is described in US 2017 / 0359659, filed June 9, 2016, and entitled “Advanced Scene Classification for Prosthesis”. The classification can serve as useful data on which changes to auditory prosthesis settings 414 are based.

[0044] The auditory prosthesis settings 414 are one or more parameters having values that affect how the auditory prosthesis 410 operates. For instance, the auditory prosthesis settings 414 can include a map having minimum and maximum stimulation levels for frequency bands of stimulation channels. The map is then used by the auditory prosthesis 410 to control an amountCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))of stimulation to be provided. For instance, where the auditory prosthesis 410 is a cochlear implant, the map affects which electrodes of the cochlear implant to stimulate and in what amount based on a received sound input. In some examples, the auditory prosthesis settings 414 include two or more predefined groupings of settings selectable by the recipient. One of the two or more predefined groupings of settings can be a default setting.

[0045] The auditory prosthesis settings 414 can also include sound processing settings that modify sound input before it is converted into a stimulation signal. Such settings can include, for example, particular audio equalizer settings can boost or cut the intensity of sound at various frequencies. In examples, the auditory prosthesis settings 414 can include a minimum threshold for which received sound input causes stimulation, a maximum threshold for preventing stimulation above a level which would cause discomfort, gain parameters, loudness parameters, and compression parameters. The auditory prosthesis settings 414 can include settings that affect a dynamic range of stimulation produced by the auditory prosthesis 410. As described above, many of the auditory prosthesis settings 414 affect the physical operation of the auditory prosthesis 410, such as how the auditory prosthesis 410 provides stimulation to the recipient in response to sound input received from the environment 401.

[0046] The recipient computing device 420 is a computing device associated with the recipient of the auditory prosthesis 410. In many examples, the recipient computing device 420 is a cell phone (e.g., smart phone), smart watch, or heart rate monitor, but can take other forms. Although described primarily in the context of the recipient, the recipient computing device 420 can be a computing device owned or primarily used by a parent or caregiver for the recipient. As illustrated, the recipient computing device 420 includes a recipient computing device sensor set 422.

[0047] The recipient computing device sensor set 422 is group of one or more components of the recipient computing device 420 that obtains data. The recipient computing device sensor set 422 can include one or more sensors, such as cameras, touch screens, input devices (e.g., keyboards, mouse, trackballs, trackpads), microphones, accelerometers, gyroscopic sensors, location sensors, biosensors (e.g., heart rate or blood pressure sensors), magnetic sensors (e.g., Hall sensors), and light sensors, among others. The recipient computing device sensor set 422 canCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))include components disposed within a housing of the recipient computing device 420 as well as devices electrically coupled to the recipient computing device 420 (e.g., via wired or wireless connections). In some examples, the recipient computing device sensor set 422 includes software sensors, such as software that obtains data from one or more data streams (e.g., audio streamed from the recipient computing device 420 to the auditory prosthesis 410), as well as hardware sensors as described above. The recipient computing device sensor set 422 can further include sensors that obtain data regarding how the recipient computing device 420 itself is being used.

[0048] In examples, the recipient computing device 420 includes an auditory prosthesis application 424 that operates on the recipient computing device 420 and cooperates with the auditory prosthesis 410. The auditory prosthesis application 424 is a computer program stored as computer-executable instructions in memory on the recipient computing device 420 that, when executed, performs one or more tasks relating to the auditory prosthesis 410. For instance, the auditory prosthesis application 424 can control the auditory prosthesis 410 (e.g., based on input received from the recipient), monitor usage of the auditory prosthesis 410, and obtain data from the auditory prosthesis 410. The recipient computing device 420 can connect to the auditory prosthesis 410 using, for example, a wireless radio frequency communication protocol (e.g., BLUETOOTH, or BLUETOOTH Low Energy (BLE)). The auditory prosthesis application 424 transmits or receives data from the auditory prosthesis 410 over such a connection. The auditory prosthesis application 424 can also stream audio to the auditory prosthesis 410, such as from a microphone of the recipient computing device sensor set 422 or an application running on the recipient computing device 420 (e.g., a video or audio application).

[0049] In examples, the auditory prosthesis application 424 functions as part of the recipient computing device sensor set 422 by obtaining data regarding the auditory prosthesis 410, as well as obtaining data from the recipient’s interaction with auditory prosthesis application 424. The recipient computing device 420 can be in communication with one or both of the clinician computing device 430 and the server 440, such as via the auditory prosthesis application 424 communicating over the network 402. In some examples, auditory prosthesis application 424 can comprise a remote self-testing application including one or more tests designed to be completed by the recipient with or without the assistance of a parent, carer, and / or caregiver, on their own at home. In one example, the one or more tests can include audiological, objective, and subjectiveCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))tests. Example tests can include activities such as a questionnaire for the recipient and / or caregiver to complete, implant site photographs provided by the recipients, self-administered speech recognition tests such as a Digit Triplets Test (DTT), and / or an Aided Threshold Test (ATT), automated impedance test, and collecting usage data and sound processor diagnostics.

[0050] The clinician computing device 430 is a computing device used by a clinician. A clinician is a medical professional, such as an audiologist, an otolaryngologist, an internist or other primary care physician, or a nurse practitioner. In an example, the clinician is a medical professional that provides care or supervision for the recipient. The clinician computing device 430 includes one or more software programs usable to communicate with auditory prosthesis application 424 to schedule and / or provide access to the self-testing applications, as well as to monitor or control the auditory prosthesis 410, such as customization and / or calibration of the auditory prosthesis settings 414.

[0051] The server 440 is a server remote from the auditory prosthesis 410, recipient computing device 420, and the clinician computing device 430. The server 440 is communicatively coupled to the recipient computing device 420 and the clinician computing device 430 via the network 402. In many examples, the server 440 is indirectly communicatively coupled to the auditory prosthesis 410 through the recipient computing device 420 (e.g., via the auditory prosthesis application 424). In some examples, the server 440 is directly communicatively coupled to the auditory prosthesis 410. The server 440 includes one or more server applications 442, which can, as an alternative to clinician computing device 430, also include one or more software programs usable to communicate with auditory prosthesis application 424 to schedule and / or provide access to the self-testing applications, as well as to monitor or control the auditory prosthesis 410, such as customization and / or calibration of the auditory prosthesis settings 414.

[0052] The one or more server applications 442 are computer programs stored as computerexecutable instructions in memory on the server 440 that, when executed, perform one or more tasks relating to the system 400. The one or more server applications 442 are operable to perform one or more operations described herein, such as operations that customize the auditory prosthesis 410. As illustrated, the one or more server applications 442 operate on the server 440.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0053] In some examples, a clinician schedules an unsupervised remote self-test of a recipient of auditory prosthesis 410. In such examples, performance results obtained through such remote self-testing often does not contain information on the focus of the recipient to the task during the test. Thus, providing feedback to the recipient or the clinician about the recipient’s focus on the task during the test can help validate the performance results of remote self-testing. For example, a camera mounted on recipient computing device 420 (e.g., a smart phone) can collect data on the recipient’s eye gaze position (on the screen) to determine the recipient’s task focus during the remote-self test. Such data is collected from recipient computing device 420 (e.g., a smart phone) via mobile application 424. The recipient’s task focus or attention can in some examples be calculated as a metric from the recipient’s eye gaze position or from tracking patterns of eye gaze movement on (and / or off) the display or screen of recipient computing device 420.

[0054] The recipient can in some examples receive visual, auditory, tactile and / or other feedback about the recipient’s own focus point during the remote self-test via application 424 in order to adjust their focus as needed during the remote self-test, such that self-proctoring or remote surveillance is applied during the recipient’s speech or performance test / task. This can be useful in a remote self-testing scenario, as the incidence of an unforeseen event causing loss of focus on the task at hand is more likely to occur in a home environment, where the test subject (e.g., the recipient) is no longer in a professional counseling setting (e.g., no counselor or clinician present, no private counseling or clinic room where the test can be conducted in a quiet environment). Performance data regarding task focus obtained via remote care platforms can therefore be valuable to validate remote self-administered tests to show that the recipient was sufficiently focused on the task of the remote self-test, so that the recipient and clinician can save the time and effort needed to conduct an in-clinic test. In the remote self-test, the recipient’s eye movement and positions are tracked and measured during the test to monitor or proctor the recipient’s test performance and task focus and can be compared with the recipient eye movements and positions during in-clinic test sessions.

[0055] In some examples, the recipient can also share their task focus data with their family, friends, or health professionals via auditory prothesis application 424, recipient computer devices 420, clinician computing device 430, and server applications 440, to communicate the recipient’s task focus and performance results. As described above, server applications 440, softwareCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))programs on clinician computing device 430, or auditory prosthesis application 424 can in some examples transmit auditory or visual feedback to the recipient, e.g., via application 424 or auditory prosthesis 410, to guide the recipient’s attention.

[0056] FIG. 4A illustrates a schematic diagram of a device 420A configured to implement aspects of technologies presented and described in conjunction with the example implantable auditory prosthesis system of FIG. 4. As discussed below with respect to FIGs. 5, 6 and 7, device 420A can be a smart phone or tablet computer that functions as recipient computing device 424. In example implementations of device 420A, streaming audio via speaker is played for the user during a remote auditory prosthesis testing, evaluation, and / or calibration session. On the example display screen of device 420A, the current test stimuli are indicated to be a CVC (consonant vowel consonant) test. In this example, a recipient can be prompted on the display of device 420A to enter a response to the test stimuli (e.g., type a word or number heard and / or comprehended, or type or speak a response to indicate whether a tone or sound was heard) via a user interface (e.g., a keyboard, button, slider, or other user interface of the touch screen, or a microphone for detecting a spoken response of the recipient). In some example implementations, an indicator can be displayed (e.g., a speaker symbol / icon that pulsates when sound is playing) to show when there is sound playing on a speaker of device 420A. In another example implementation, indicators or prompts can be displayed on device 420A (e.g., a green light or star) to show whether the recipient’s measured attention metric meets a predetermined threshold showing sufficient focus on the test such that the test results will be determined to be valid. In example implementations, prompts to the recipient can additionally be provided to the user via the display on device 420A or alert sounds via the speaker on device 420A to indicate that the recipient is not focusing sufficiently on the hearing test and needs to return and / or focus their gaze and / or attention on the screen.

[0057] FIG. 5 illustrates a block diagram of an exemplary system and method 500 configured to implement aspects of the attention focus tracking technologies described herein, particularly directed to a recipient testing phase. At step 502, a hearing test is initiated at the start of test. In some examples, the hearing test is conducted as a remote test where the test can be done as a part of a regular, e.g., annual checkup in place of coming into the clinic. If the remote test results (after validation of the test indicating that the recipient was focused during the test) indicates thatCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))further evaluation and / or calibration of the auditory prosthesis is needed, the recipient can then be invited or requested to come to the clinic via the auditory prosthesis application 424. In other examples, a hearing test can be conducted as a general hearing screening test at home, in a retail environment like a store, or in a clinical environment like a clinic, prior to a recipient receiving an over-the-counter hearing aid. Such diagnostic hearing tests can be used to measure the focus of the recipient during the hearing test to ensure the test results are valid, so that the auditory prosthesis (e.g., over-the-counter hearing aid and / or consumer hearables, or therapeutic hearing devices such as medical grade hearing aids and Cis) are correctly calibrated for the recipient’s hearing capabilities.

[0058] At step 504 the acoustic environment is assessed. In some examples, the hearing test should be conducted in a quiet or relatively quiet area (e.g., indoors in a quiet room) in order to ensure that the hearing test is valid. In other examples, the hearing test may be conducted in other environments (e.g., outdoors, or in a public area like a restaurant or lobby) to assess the performance of the auditory prosthesis in these different environments. At step 506, test stimuli are transmitted to the recipient, where the test stimuli comprise one or more signals perceptible to the recipient, as described below with respect to FIG. 7. In examples, the test stimuli include streamed audio files that are available on auditory prosthesis application 424 or streamed from server application 442 or clinician computing device 430 to auditory prosthesis application 424. Once the test stimuli begin to transmit, data gathering takes place.

[0059] The general environment is monitored at 512, including environmental conditions such as ambient sounds, GPS location, temperature, health data of the recipient (e.g., body temperature, heart rate, pulse ox, etc.). Environmental conditions 512 are monitored and gathered as raw data in step 540. At the end of the test, if the environmental conditions as monitored fall outside predetermined thresholds (e.g., background noise level is too high), the hearing test can be invalidated in some examples.

[0060] In some example implementations, eye camera data 514 and electro-oculo data 508 are recorded and measured during the test. Eye camera data 514 is recorded via a front facing camera (e.g., the camera facing the user) on recipient computing device 424 (e.g., the recipient’s smart phone or tablet) to determine where the recipient’s eye gaze and / or focus is directed and / orCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))measure the coordinates of the recipient’s gaze on the display of recipient computing device 424. In general, when the recipient’s eye gaze or focus is on a display of recipient computing device 424, the recipient’s focus on the task can be more highly indicated, particularly when the user is inputting a response. In some examples, electro-oculo data 508 recording eye movements during the hearing test can be measured in some examples by placing electrodes on the skin near the eyes, or by using electrodes placed in or near the ear canal. Eye movements indicating focus (or lack of focus) during the hearing test can be measured using electro-oculo data 508. Taken together, eye camera data 514 and electro-oculo data 508 comprise eye focus data 524, which is used in part along with electroencephalogram (EEG) data 510, if available, to calculate attention metric 530 in some examples. In some examples, EEG data can be used to record levels of concentration of the recipient during the hearing test as known to those skilled in the art. EEG data can be recorded using a conventional cap EEG (e.g., EEG recorded over the entire head), or mobile ear-EEG in some examples, although such EEG data acquisition methods can be more challenging to execute in a remote test environment (e.g., in the recipient’s home).

[0061] At step 526, the recipient inputs their response to the test stimuli using a user interface available on recipient computing device 424 (e.g., keyboard, button, slider, microphone, etc.). In some example implementations, the response 526 and the attention metric 530 are recorded, timestamped and analyzed to derive an interim test result 522, where the attention metric 530 and response 526 are analyzed to determine whether the recipient response 526 is within a predetermined response time frame and / or predetermined range or set of correct responses.Feedback can be provided to the recipient at 520. In some example implementations, user feedback 520 can be displayed on recipient computer device 424 to indicate to the user during the course of the test session, for example, whether a response is correct, and / or whether the eye focus data 524 and EEG data 510 indicate that the recipient’s attention metric 530 shows that the recipient task focus meets a predetermined threshold and / or acceptable level for validity of the test results, or that the recipient needs to improve their focus on the task to help ensure the hearing test results are validated.

[0062] In example implementations, if the end of the testing session is not reached at step 534, then a second (or third, and so on) test stimuli is presented at step 506. If the end of the testing session is reached at 534, then the recipient is notified of the conclusion of the test session at stepCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))540 and all raw test data from the hearing test, including eye focus data 524, EEG data 510, environmental monitoring data 512, and recipient’s response 526, are gathered at step 540. In example implementations, the process then concludes a recipient testing phase at point A at 542. The test result can be reported to the recipient and / or the clinician in some examples, including the recipient’s responses and the attention / focus evaluation results. In other example implementations, the process can, optionally, continue on to a data analysis phase at point A at 605 of FIG. 6 as described below, prior to the reporting of the test results.

[0063] FIG. 6 thus illustrates a continuation of the block diagram of the exemplary system and method depicted in FIG. 5, particularly directed to a data analysis phase 600 of the recipient testing session that picks up at point A 605 which is the same point as the conclusion of the recipient testing phase at 542 of FIG. 5. At operation 630, raw data may be processed together with external knowledge using machine learning techniques. External knowledge may comprise, for example, raw test data (e.g., recipient response data and recipient task focus data) from results of previous hearing tests taken in the past by the recipient and / or other recipients of auditory prostheses. Such data may be stored in a database 610 that may be accessible to server 440, clinician computing device 430, and / or recipient computing device 420 via network 402. In some example implementations, training data related to task focus for hearing and / or other tests may be accessed from external sources 620 other than database 610 which are accessible via the internet.

[0064] Training data can include, for example, EEG data and eye camera data that can be collected during hearing tests conducted for many recipients of auditory prostheses over time. In other examples, training data can include data collected from recipients that use a brain- computer interface (BCI) such as a neural electrode array, which detects and records signals in the form of electrocorticography (ECoG) signals that can be transmitted to, e.g., an implantable or nonimplantable auditory prosthesis. In these example implementations, ECoG signals can be recorded that monitor various aspects of hearing quality during a hearing test, such as loudness, pitch, clarity, speech recognition, mood, level of attention, level of focus, and sound localization. In other examples, the neurological signals can comprise local field potential (LFP) or extracellular potential (EP) signals. ECoG signals as described herein can be interchangeable with EEG and other neurological signals known to those skilled in the art.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0065] The raw test data (eye camera data, electro-oculo data, EEG data, and / or ECoG data) can be processed using various algorithms to extract features and information relevant to an assessment of focus of the recipient. For example, the device can use spectral analysis to estimate the power and frequency of the ECoG signals, which can reflect the level of interest, attention, and cognitive load of the recipient. Neural processing unit 502 can also use event-related analysis to measure the latency, amplitude, and morphology of the auditory evoked potentials (AEPs), which can indicate the auditory processing and perception of the recipient. Neural processing unit 522 can also utilize machine learning techniques to classify the EEG, eye camera data, electro-oculo data, and / or ECoG data into various cognitive states that may help assess the level of task focus, including: understanding, confusion, memory, concentration, alertness, etc.

[0066] To process data for analyzing comprehension of hearing, mood, and attention, a combination of advanced signal processing techniques and machine learning algorithms can be used. Initially, in example implementations, raw data will undergo signal preprocessing steps, such as filtering, to reduce noise and segmentation to extract relevant time periods. This can be followed in example implementations by feature extraction, where both time-domain and frequency-domain features are computed. Time-frequency representations like the short-time Fourier transform (STFT) and connectivity measures such as coherence can be critical for capturing the complexity of brain signals. Dimensionality reduction techniques like Principal Component Analysis (PC A) or Independent Component Analysis (ICA) can help manage the high-dimensional nature of the data, making subsequent analysis more efficient.

[0067] Machine learning algorithms can play a pivotal role in interpreting the raw data signals in some examples, so that the validity of the test result can be accurately determined at step 640 based on the recipient’s raw attention / focus data recorded during the hearing test. Classification algorithms, such as Support Vector Machines (SVM), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM) networks, are used to identify and predict levels of attention and task focus of recipients. Regression algorithms can be employed if continuous variables related to mood or attention need to be predicted. Unsupervised learning techniques, including clustering, can uncover hidden patterns in the data. Additionally, machine learning facilitates the adaptation and improvement of models over time, enhancing their accuracy. Post-CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))processing involves statistical analysis to validate findings and visualization techniques to present the data meaningfully. The integration of these methods allows for the real-time decoding of cognitive states such as level of attention and / or focus.

[0068] FIG. 7 illustrates an exemplary flow diagram of an example method 700 of validating remote testing of a medical device as described in aspects of technologies presented herein. Example method 700 is described below in one exemplary context as a remote test of a medical device for hearing (e.g., an auditory prosthesis as described herein). However, it is clear that example method 700 can be used with presenting test stimuli (e.g., whether auditory stimuli, visual stimuli, or verbal or text instructions) for recipients of other medical devices such as medical devices for adjacent therapies to address tinnitus, vestibular implants to address balance issues, or medical devices to address visual impairments, such as retinal implants.

[0069] At step 710, the method begins. At operation 720, one or more signals perceptible to a recipient (e.g., auditory signals) are transmitted. The signals can be transmitted from an external device of the recipient (e.g., recipient computing device 420), or other device capable of transmitting signals perceptible by the recipient. In some examples, such signals perceptible to the recipient can comprise test stimuli streamed to a recipient device (e.g., recipient computing device 420). In general, the test stimuli streamed to the device can comprise speech-like stimuli, such as Consonant- Vowel-Consonant (CVC), or more generally Consonant-Nucleus-Consonant (CNC) monosyllabic words (e.g., words such as “bus” or “cab” that may be taken out of context). Other speech-like stimuli can comprise a wide range of simple to complex linguistic units, from individual speech sounds or phonemes (isolated vowels or consonants) to nonsense syllables, pseudo-words, and other non-CNC or non-CVC monosyllabic words. Speech-like stimuli can also comprise complex linguistic units such as multi-syllabic words or complete sentences. In one example, a remote hearing test comprising speech-like stimuli can comprise a triplet test, where the user is asked to hear three spoken digits or short words. Background noise can be added to successive iterations of the triplet test until the user makes a mistake in hearing comprehension.

[0070] In other examples, non-speech like sounds can be transmitted, such a single tone, a series of tones, music (e.g., instrumental music or vocal music), or other sounds (e.g., ambient natureCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))sounds, water, waves, bird song, animals, wind, traffic, machinery, background noise in a restaurant, etc.). In one example, a remote hearing test comprising non-speech-like stimuli can comprise an aided threshold test (ATT) where the sounds streamed to and played for the user comprise a series of tones of progressively higher frequencies to determine the threshold frequency at which the recipient’s perception of the sound changes and / or the recipient can no longer hear the sound.

[0071] At operation 730, the recipient inputs a response at the external device, wherein the input is responsive to the one or more signals. For example, as shown in FIG. 4A, the recipient is instructed by the display on the screen to “Type what you hear” and is presented with a keyboard on the touch screen to type a response. This instruction can be used, for example, when the test stimuli is speech-like stimuli used to test the recipient’s hearing perception while using the recipient’s prescribed auditory prosthesis, such that the recipient can type words, sentences, letters (e.g., vowels or consonants), or speech-like or other sounds heard by the recipient which are then recorded during a hearing test. In other examples, a user interface displayed at the recipient computing device 424 can provide a button, slider, or other input mechanism for the recipient to indicate the type of sound or level of sound perceived by the recipient.

[0072] At operation 740, at least one eye tracking data metric of the recipient is measured substantially in real time at the external device. In examples, the eye tracking data metric is measured and recorded in association and substantially simultaneously with the transmission of the one or more signals and the corresponding input received by the recipient. In some examples, the camera of recipient computing device 424 can record one or more coordinates of the gaze of the recipient’s eyes with respect to a display of recipient computing device 424. In some other examples, the camera of recipient computing device can record the direction, speed, and location of the recipient’s eye, head, and / or hand movements during the hearing test. Pauses or movements of the eyes and / or head and closing and opening of the eyes can be recorded in some examples. Timestamps can be applied to the recipient’s input, as well as of the recipient’s various, eye, head, and / or hand movements, pauses, and / or closing or opening of the eyes during the hearing test recording.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0073] After each instance of test stimuli is provided to the recipient at operation 720, and the recipient’s corresponding response (e.g., a word or sentence that the recipient is asked to type into the interface, button press, or other feedback received from the recipient) is received at operation 730, and the simultaneous measurement and recording of the eye tracking and / or other data metrics at operation 740, steps 720, 730, and 740 can be repeated for another instance of test stimuli until the hearing test is complete. In some examples, the recording of the eye tracking and / or other data metrics, and the recording of the recipient’s responses can be transmitted via network 402 to clinician computing device 430 or to server application 442 on server 440 for further analysis, or analysis can be conducted directly on recipient computing device 420.

[0074] At operation 750, data from the at least one eye tracking data metric, the one or more signals, and the input received from the recipient is analyzed and used to determine at least one focus tracking metric of the recipient. In some examples, a confidence score in a range between 0 - 100% can be calculated for the recipient using the above data to measure whether the hearing test was done with full focus. In some examples, a percentage of time in which the recipient’s eye gaze is directed at the screen can be calculated to determine a confidence score. In other examples, other recipient behavior or actions indicative of recipient focus on a task can be recorded and analyzed, such as closing the eyes while the sound is playing, then opening the eyes to enter a response. Other examples of recipient behavior or actions indicative of recipient focus on a task can include looking away from the screen at a distant point while the test stimuli are playing, then returning the gaze to the screen of recipient computing device 424 to enter a response.

[0075] In operation 760, the hearing test performance data can be validated based on the at least one focus tracking metric of the recipient. If the hearing test performance data indicates that the confidence score meets a predetermined threshold, and that the recipient’s hearing test results are validated, then the hearing test concludes in step 770. However, in one example, if the confidence score falls below a predetermined threshold, the results of the hearing test can be invalidated. Once the hearing test is invalidated, in some examples, the recipient can be requested to retake the hearing test (e.g., where the flow of the method 700 returns from operation 760 to operation 720. In another example, the hearing test may conclude at operation 770 when the hearing test results are invalidated, and the recipient will be requested to have aCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))follow up in-office appointment or a live telehealth visit with a clinician, perhaps to retake the hearing test.

[0076] The teachings detailed herein are implemented in sensory prostheses, such as hearing implants specifically, and brain-computer interfaces (BCIs) and neural stimulation devices in general. Other types of sensory prostheses can include retinal implants. Accordingly, any teaching herein with respect to a BCI and sensory prosthesis corresponds to a disclosure of utilizing those teachings in / with a hearing implant and in / with a retinal implant, unless otherwise specified, providing the art enables such. Moreover, with respect to any teachings herein, such corresponds to a disclosure of utilizing those teachings with a cochlear implant, a bone conduction device (active and passive transcutaneous bone conduction devices, and percutaneous bone conduction devices) and a middle ear implant, providing that the art enables such, unless otherwise noted. To be clear, any teaching herein with respect to a specific sensory prosthesis corresponds to a disclosure of utilizing those teachings in / with any of the aforementioned hearing prostheses, and visa-versa. Corollary to this is at least some teachings detailed herein can be implemented in somatosensory implants and / or chemosensory implants. Accordingly, any teaching herein with respect to a sensory prosthesis corresponds to a disclosure of utilizing those teachings with / in a somatosensory implant and / or a chemosensory implant.

[0077] While the teachings detailed herein will be described for the most part with respect to hearing prostheses, in keeping with the above, it is noted that any disclosure herein with respect to a hearing prosthesis corresponds to a disclosure of another embodiment of utilizing the associated teachings with respect to any of the other prostheses noted herein, whether a species of a hearing prosthesis, or a species of a sensory prosthesis, such as a retinal prosthesis. In this regard, any disclosure herein with respect to evoking a hearing percept corresponds to a disclosure of evoking other types of neural percepts in other embodiments, such as a visual / sight percept, a tactile percept, a smell precept or a taste percept, unless otherwise indicated and / or unless the art does not enable such. Any disclosure herein of a device, system and / or method that is used to or results in ultimate stimulation of the auditory nerve corresponds to a disclosure of an analogous stimulation of the optic nerve utilizing analogous components / methods / systems.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0078] All publications identified herein are incorporated by reference to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies and the definition of that term in the reference does not apply.

[0079] It should be apparent to those skilled in the art that many more modifications besides those already described are possible without departing from the inventive concepts herein. The inventive subject matter, therefore, is not to be restricted except as set forth in the appended claims. Moreover, in interpreting both the specification and the claims, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprise” and “comprising” should be interpreted as referring to elements, compounds, or steps in a non-exclusive manner, indicating that the referenced elements, components, or steps are present, or utilized, or combined with other elements, components, or steps that are not expressly referenced. Where the specification or claims refer to at least one of something selected from the group consisting of A, B, C, ... and N, the text should be interpreted as requiring only one element from the group, not A plus N, or B plus N, etc.

[0080] In some embodiments, the numbers expressing properties or parameters such as supply voltages, stimulating currents, resistances, and so forth, used to describe and claim certain embodiments of the invention are to be understood as being modified in some instances by the term “about.” Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the invention are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable. The numerical values presented in some embodiments of the invention can contain certain errors necessarily resulting from the standard deviation found in their respective testing measurements.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))

[0081] Unless the context dictates the contrary, all ranges set forth herein should be interpreted as being inclusive of their endpoints and open-ended ranges should be interpreted to include only commercially practical values. Similarly, all lists of values should be considered as inclusive of intermediate values unless the context indicates the contrary.

[0082] As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.

[0083] The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.

[0084] Groupings of alternative elements or embodiments of the invention disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and / or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of any Markush groups used in the appended claims.

[0085] It should be noted that any language directed to a computer should be read to include any suitable combination of computing devices, including servers, interfaces, systems, databases, agents, peers, engines, controllers, or other types of computing devices operating individually or collectively. One should appreciate the computing devices comprise one or more processors,CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))such as a general purpose processor, or an application specific integrated circuit (ASIC) configured to execute software instructions stored on a tangible, non-transitory computer readable storage medium (e.g., hard drive, solid state drive, RAM, flash, ROM, PLA, PLD, FPGA, etc.). The software instructions preferably configure or program the computing device to provide the roles, responsibilities, or other functionality as discussed below with respect to the disclosed apparatus. Further, the disclosed technologies can be embodied as a computer program product that includes a non-transitory computer readable medium storing the software instructions that causes a processor to execute the disclosed steps. In especially preferred embodiments, the various servers, systems, databases, or interfaces exchange data using standardized protocols or algorithms, possibly based on HTTP, HTTPS, AES, public-private key exchanges, web service APIs, or other electronic information exchanging methods. Data exchanges preferably are conducted over a packet-switched network, the Internet, LAN, WAN, VPN, or other type of packet switched network; a circuit switched network; cell switched network, or other type of network.

[0086] The above discussion provides many example embodiments of the inventive subject matter. Although each embodiment represents a single combination of inventive elements, the inventive subject matter is considered to include all possible combinations of the disclosed elements. Thus, if one embodiment comprises elements A, B, and C, and a second embodiment comprises elements B and D, then the inventive subject matter is also considered to include other remaining combinations of A, B, C, or D, even if not explicitly disclosed.

[0087] As used in the description herein and throughout the claims that follow, when a system, engine, module, device, server, processor or other computing element is described as configured to perform or execute functions on data in a memory, the meaning of “configured to” or “programmed to” is defined as one or more processors or cores of the computing element being programmed by a set of software instructions stored in the memory of the computing element to execute the set of functions on target data or data objects stored in the memory thereby forming a structure having a specific purpose.

[0088] As used herein, and unless the context dictates otherwise, the term "coupled to" is intended to include both direct coupling (in which two elements that are coupled to each otherCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))contact each other) and indirect coupling (in which at least one additional element is located between the two elements). Therefore, the terms "coupled to" and "coupled with" are used synonymously.

[0089] While the invention has been particularly described with respect to the illustrated embodiments and examples discussed herein, it will be appreciated that various alterations, modification, and adaptations can be made based on the present disclosure and are intended to be within the scope of the invention. While the invention has been described in connection with what are presently considered to be some practical examples, it is to be understood that the invention is not limited to any of the disclosed embodiments or examples but only by the following claims.

Claims

CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))CLAIMSWhat is claimed is:

1. A method comprising:transmitting one or more signals perceptible to a recipient via a first device; receiving input from the recipient at the external device, wherein the input is responsive to the one or more signals perceived by the recipient;at a plurality of time intervals, measuring substantially in real time at the external device at least one eye tracking data metric of the recipient in association with the one or more signals and the input received from the recipient at the external device; and determining at least one focus tracking metric of the recipient based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient at the external device.

2. The method of claim 1 , wherein the one or more transmitted signals comprise audio signals perceptible to a human.

3. The method of any of claims 1 or 2, wherein the audio signals comprise one or more simple or complex tones of varying frequencies or frequency content.

4. The method of any of claims 1 or 2, wherein the audio signals comprise verbal data including one or more spoken speech sounds, syllables, pseudo-words, monosyllabic words having a consonant-vowel-consonant (CNC) structure or other, or more complex lexical configuration.

5. The method of any of claims 1 through 4, wherein the input received from the recipient comprises one or more keystrokes entered by the recipient or one or more oral responses from the recipient.

6. The method of any of claims 1 through 5, wherein measuring the at least one eye tracking data metric comprises capturing through a camera of the external device, at least one of a coordinate location of an eye gaze of the recipient on a screen of the external device, a directionCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))of movement of the eye gaze of the recipient on the screen of the external device, or a pause in the eye gaze of the recipient on the screen of the external device.

7. The method of any of claims 1 through 6, wherein the first device comprises at least a portion of an auditory prosthesis system.

8. The method of any of claims 1 through 6, wherein the first device comprises a consumer hearing device.

9. The method of any of claims 1 through 8, wherein the external device comprises a consumer computing device.

10. The method of any of claims 1 through 9, wherein the at least one focus tracking metric is determined at least in part based on a neural network analysis of the at least one eye tracking data metric of the recipient measured at the external device.

11. A device comprising:a camera;a user interface comprising at least a display; anda processor communicatively coupled to the camera and the user interface and configured to:transmit one or more signals perceptible to a recipient via a first device; receive input from the recipient at the user interface responsive to the one or more signals perceived by the recipient;at a plurality of time intervals, measuring substantially in real time at least one eye tracking data metric captured by the camera in association with the one or more signals and the input received from the recipient; anddetermining at least one focus tracking metric based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))12. The device of claim 11, wherein the one or more transmitted signals comprise audio signals including verbal data or pseudo-verbal data.

13. The device of any of claims 11 or 12, wherein the first device comprises at least a portion of an auditory prosthesis.

14. The device of any of claims 11 through 13, wherein the first device comprises a consumer hearing device.

15. The device of any of claims 11 through 14, wherein the at least one eye tracking data metric comprises at least one of a coordinate location of an eye gaze of the recipient on the display of the device, a direction of movement of the eye gaze of the recipient, or a pause in the eye gaze of the recipient on the display of the device.

16. The device of any of claims 11 through 15, wherein the input received from the recipient comprises one or more keystrokes entered by the recipient or one or more oral responses from the recipient.

17. The device of claim 11, wherein the first device comprises a retinal implant.

18. The device of any of claims 11 through 17, wherein the at least one focus tracking metric is determined at least in part based on a neural network analysis of the at least one eye tracking data metric of the recipient.

19. The device of any of claims 11 through 18, wherein the one or more processors is further configured to display one or more instructions to the recipient based on an assessment of the at least one focus tracking metric.CID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))20. A non-transitory computer readable medium comprising one or more computer readable instructions, which upon execution by one or more processors, performs the following operations:transmitting one or more signals perceptible to a recipient via a first device; receiving input from the recipient at an external device responsive to the one or more signals perceived by the recipient;measuring substantially in real time at the external device at a plurality of time intervals, at least one eye tracking data metric of the recipient in association with the one or more signals and the input received from the recipient at the external device; and determining at least one focus tracking metric of the recipient based on data associated with the at least one eye tracking data metric, the one or more signals, and the input received from the recipient at the external device.

21. The non-transitory computer readable medium of claim 20, wherein the one or more computer readable instructions are stored in a memory of the external device.

22. The non-transitory computer readable medium of any of claims 20 or 21, wherein the one or more transmitted signals comprise audio signals perceptible to a human.

23. The non-transitory computer readable medium of any of claims 20 through 22, wherein the audio signals comprise verbal data including one or more spoken speech sounds, syllables, pseudo-words, monosyllabic words having a consonant-nucleus-consonant (CNC) configuration or other, or more complex lexical configuration.

24. The non-transitory computer readable medium of any of claims 20 through 23, wherein the input received from the recipient comprises one or more keystrokes entered by the recipient or one or more oral responses from the recipient.

25. The non-transitory computer readable medium of any of claims 20 through 24, wherein measuring the at least one eye tracking data metric comprises capturing through a camera of the external device, at least one of a coordinate location of an eye gaze of the recipient on a screen ofCID03979WOPC1 (Attorney Docket No. C6413.10005W001 (PCT))the external device, a direction of movement of the eye gaze of the recipient on the screen of the external device, or a pause in the eye gaze of the recipient on the screen of the external device.

26. The non-transitory computer readable medium of any of claims 20 through 25, wherein the first device comprises at least a portion of an auditory prosthesis system.

27. The non-transitory computer readable medium of any of claims 20 through 26, wherein the first device comprises a consumer hearing device.

28. The non-transitory computer readable medium of any of claims 20 through 27, wherein the at least one focus tracking metric is determined at least in part based on a neural network analysis of the at least one eye tracking data metric of the recipient measured at the external device.

29. The non-transitory computer readable medium of any of claims 20 through 28, wherein the at least one focus tracking metric is transmitted to a clinician for further assessment of one or more of the first device, the external device, and the recipient.

30. The non-transitory computer readable medium of any of claims 20 through 28, wherein the at least one eye tracking data metric is displayed on the external device to the recipient.

Images