Interactive aural language competency development

Abstract

Presented herein are systems and methods for generating one or more conversational dialogue parts for interaction with a user device user via an intelligent agent. One or more conversational dialogue parts for interaction with a user device user are generated via at least one acoustic transducer. One or more responses from the user device user to the one or more conversational dialogue parts are obtained via at least one sound input unit. One or more subsequent conversational dialogue parts for interaction with the user device user are generated based on the responses from the user device user.

Description

Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1INTERACTIVE AURAL LANGUAGE COMPETENCY DEVELOPMENT BACKGROUNDField of the Invention[oooi] The present invention relates generally to interactive aural language competency development.Related Art

[0002] Medical devices are devices that are intended to be used for medical purposes. They can vary in both their intended use and indications for use. Examples range from simple, low-risk medical supplies to complex, potentially high-risk devices that are implanted and / or sustain life, such as deep brain stimulators and brain-computer interfaces. Other categories of medical device include diagnostic equipment.

[0003] Hearing devices act on an actual or potential auditory perception of an individual, including to improve perception of sound signals, to reduce perception of sound signals, etc. In particular, a hearing device can deliver sound signals to a user in any form, including in the form of acoustical stimulation, mechanical stimulation, electrical stimulation, etc., and / or can operate to suppress all or some sound signals. As such, a hearing device can be a device for use by a hearing-impaired person (e.g., hearing aids, middle ear auditory prostheses, bone conduction devices, direct acoustic stimulators, electro-acoustic hearing prostheses, auditory brainstem stimulators, bimodal hearing prostheses, bilateral hearing prostheses, dedicated tinnitus therapy devices, tinnitus therapy devices, etc.) or a device for use by a person with normal hearing (e.g., a consumer device that provides audio streaming, a consumer headphone, an earphone, etc.), a hearing protection device (e.g., a noise cancellation headset, a loudness reduction apparatus, etc.), etc.SUMMARY

[0004] In one aspect, a system is provided. The system comprises at least one acoustic transducer; at least one sound input unit; an aural language competency development device, comprising: a memory; and at least one processor operably coupled to the memory, the at least one acoustic transducer, and the at least one sound input unit, wherein the at least one processor is configured to execute an intelligent agent to: generate one or more conversational dialogue parts for interaction with a user device user via the at least one acoustic transducer; obtain one or more responses from the user device user to the one or more conversational dialogue partsAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1via the at least one sound input unit; and generate one or more subsequent conversational dialogue parts for interaction with the user device user based on the one or more responses from the user device user.

[0005] In another aspect, another system is provided. The system comprises: at least one acoustic transducer; at least one sound input unit; an aural language competency development device, comprising: a memory storing an intelligent agent; and at least one processor operably coupled to the memory, the at least one acoustic transducer, and the at least one sound input unit, wherein the at least one processor is configured to execute the intelligent agent to: deliver one or more interactive exercises to a user device user via the at least one acoustic transducer; obtain one or more responses of the user device user to the one or more interactive exercises; use the one or more responses of the user device user to assess at least one of a preference of the user device user or an aural language competency development of the user device user; and adapt the one or more interactive exercises delivered to the user device user over time based on the assessment of at least one of the preference of the user device user or the aural language competency development of the user device user.

[0006] In another aspect, one or more non-transitory computer readable storage media are provided. The one or more non-transitory computer readable storage media comprises instructions that, when executed by a processor, cause the processor to: obtain a personality map associated with a character object, wherein the personality map identifies one or more characteristics forming a personality of the character object; train, based on the personality map, an intelligent agent to generate one or more interactive exercises to elicit a behavioral response from a user; and update the personality map based on the behavioral response.

[0007] In one aspect, a method is provided. The method comprises: obtaining at least one response from a user to at least one stimulus provided via an acoustic transducer; determining whether the at least one response matches an expected response; upon determining that the at least one response matches the expected response, updating a personality map of a character object to generate an updated personality map; and training, based on the updated personality map, an intelligent agent to generate a subsequent stimulus.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments of the present invention are described herein in conjunction with the accompanying drawings, in which:

[0009] FIG. 1A is a schematic diagram illustrating a cochlear implant system with which aspects of the techniques presented herein can be implemented;[ooio] FIG. IB is a side view of a user wearing a sound processing unit of the cochlear implant system of FIG. 1A;[ooii] FIG. 1C is a schematic view of components of the cochlear implant system of FIG. 1 A;

[0012] FIG. ID is a block diagram of the cochlear implant system of FIG. 1 A;

[0013] FIG. IE is a schematic diagram illustrating a computing device with which aspects of the techniques presented herein can be implemented;

[0014] FIG. 2A is a schematic diagram illustrating a system for engaging a user in one or more interactions to develop the user’s aural language competency, in accordance with certain embodiments presented herein;

[0015] FIG. 2B is a block diagram of the aural language competency development device of FIG. 2A, in accordance with certain embodiments presented herein;

[0016] FIG. 2C is a schematic diagram illustrating another system for engaging a user in one or more interactions to develop the user’s aural language competency, in accordance with certain embodiments presented herein;

[0017] FIG. 2D is a schematic diagram illustrating another system for engaging a user in one or more interactions to develop the user’s aural language competency, in accordance with certain embodiments presented herein;

[0018] FIG. 3A is a schematic diagram illustrating a system for engaging a user in a conversation to develop the user’s aural language competency, in accordance with certain embodiments presented herein;

[0019] FIG. 3B is a flowchart illustrating an exemplary interaction between a sleeve structure and a user, in accordance with certain embodiments presented herein;

[0020] FIG. 4A is a schematic diagram illustrating a system for engaging a user in one or more interactions in an interactive environment, in accordance with certain embodiments presented herein;Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0021] FIG. 4B is a schematic diagram illustrating another system for engaging a user in one or more interactions in an interactive environment, in accordance with certain embodiments presented herein;

[0022] FIG. 5 is a flowchart illustrating a method for determining a communication style based on an engagement level of a user, in accordance with certain embodiments presented herein;

[0023] FIG. 6A is a flowchart illustrating a method for adapting an educational routine based on a user’s engagement level, in accordance with certain embodiments presented herein;

[0024] FIG. 6B is a flowchart illustrating further details of the embodiment described in FIG.6A;

[0025] FIG. 7A is a flowchart illustrating a method for determining a user’s favorite interaction, in accordance with certain embodiments presented herein;

[0026] FIG. 7B is a flowchart illustrating a method for generating one or more lesson plans adapted to incorporate one or input words or sounds, in accordance with certain embodiments presented herein;

[0027] FIG. 7C is a flowchart illustrating a method for generating one or more conversations adapted to incorporate one or more words used by a user, in accordance with certain embodiments presented herein;

[0028] FIG. 8 A is a schematic diagram illustrating a system for monitoring a user’s aural language competency development, in accordance with certain embodiments presented herein;

[0029] FIG. 8B is a schematic diagram illustrating another view of the system for monitoring a user’s aural language competency development of FIG. 8 A;

[0030] FIG. 9 is a schematic diagram illustrating a system for managing one or more hearing devices, in accordance with certain embodiments presented herein;

[0031] FIG. 10 is a flowchart illustrating a method for generating one or more conversational dialogue parts for interaction with a hearing device user, in accordance with certain embodiments presented herein;

[0032] FIG. 11 is a flowchart illustrating a method for adapting one or more interactive exercises based on one or more responses from a user, in accordance with certain embodiments presented herein;Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0033] FIG. 12 is a flowchart illustrating a method for training an intelligent agent to generate one or more interactive exercises, in accordance with certain embodiments presented herein;

[0034] FIG. 13 is a flowchart illustrating a method for training an intelligent agent to generate a subsequent stimulus, in accordance with certain embodiments presented herein;

[0035] FIG. 14 is a schematic diagram illustrating a vestibular stimulator system with which aspects of the techniques presented herein can be implemented;

[0036] FIG. 15 is a schematic diagram illustrating a retinal prosthesis system with which aspects of the techniques presented herein can be implemented;

[0037] FIG. 16 is a schematic diagram illustrating a tinnitus therapy device with which aspects of the techniques presented herein can be implemented; and

[0038] FIG. 17 is a perspective view of an upper airway stimulation device with which aspects of the techniques presented herein can be implemented.DETAILED DESCRIPTION

[0039] Presented herein are techniques for interactive development (e.g., evaluation, training, habitualization, rehabilitation, etc.) of the aural language competency of a recipient / user (e.g., hearing-impaired individuals, user device (e.g., medical or hearing device) users / recipients, etc.). As used herein, “aural language competency” refers to the user’ s hearing ability, language understanding and / or speech production.

[0040] More specifically, presented herein are techniques in which at least one sleeve structure, such as a character object (e.g., toy, pet, doll, etc.), includes an aural language competency development device. The aural language competency development device is configured to execute an intelligent agent (e.g., generative artificial intelligent (Al) system / chatbot) to facilitate aural interactivity with, for example, a hearing-impaired individual, a user device (e.g., hearing or device) user / recipient, etc., collectively and generally referred to herein as “users.” For example, during a testing / leaming session, the intelligent agent can be configured to deliver one or more interactions (e.g., conversations, educational routines, challenges and / or activities) that recruit and evaluate the user’s aural language faculties. Moreover, the intelligent agent can be configured to compare the user’s response with one or more normative expected responses to determine an engagement level. Based on the engagement level, the intelligent agent can dynamically adapt one or more subsequent interactions to increase the user’sAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1engagement level until a predetermined learning goal (e.g., language development, sound differentiation, balance, movement, sleep, etc.) is met.

[0041] Moreover, the aural language competency development device can be configured to leverage spatially-separated transducers for delivering stimuli to the user and / or spatially-separated sensors for capturing responses from the user in an interactive environment. Such arrangements enable the user to freely incorporate their own physical movement / activity into their interaction (e.g., conversational engagement) with the sleeve structure to replicate familiar everyday interaction scenarios, such as a child physically playing with toys or adults physically engaging with a companion animal. As a result, user engagement can be increased through playful and natural physical activity, thus reducing inhibition and compensation strategies associated with “test” like conditions while simultaneously improving clinical compliance, validity and efficacy.

[0042] Moreover, the user’s responses can be provided as feedback to the intelligent agent to generate “smart” aural rehabilitation that adapts to the user’s progress. Further, the aural language competency development device can be configured to leverage a trained machine learning model to generate / select one or more educational exercises adapted to the user’s personal learning journey. The determined educational exercises can be delivered to the user via a character object having a personality (as exemplified by tone of voice, choice of vocabulary, etc.) that matches the personality / needs of the user, thus dynamically engaging the user in an interactive learning experience.

[0043] There are a number of different types of user devices (e.g., devices worn by, or implanted in a recipient / user) in / with which the techniques presented herein can be implemented. Merely for ease of description, the techniques presented herein are primarily described with reference to a specific user device. However, it is to be appreciated that the techniques presented herein can also be partially or fully implemented by any of a number of different types of devices or systems, including consumer electronic devices (e.g., consumer hearing devices, consumer computing devices such as mobile phones and tablets, audio equipment such as home theatre and car audio systems, etc.), computing systems (e.g., servers in data centers, Intemet-of-Things (loT) devices), various types of software systems, such as databases, machine learning and artificial intelligence systems, other medical devices, such as diagnostic equipment or life sustaining equipment, etc. For example, the techniques presented herein could be used in or with sensory protheses, including hearing aids and cochlear implants, and various medical devices, such as pacemakers, drug delivery systems, implantableAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1defibrillators, functional electrical stimulation devices, sleep disorder devices (e.g., sleep apnea devices), seizure devices (e.g., devices for monitoring and / or treating epileptic events), balance or movement disorder devices (e.g., vestibular stimulation devices), tinnitus management devices, visual implants (e.g., bionic eyes), and other neuromodulation devices (e.g., braincomputer interfaces).

[0044] FIGs. 1A-1D illustrate an example user device in the form of cochlear implant system 102 with which aspects of the techniques presented herein can be implemented. The cochlear implant system 102 comprises an external component 104 that is configured to be directly or indirectly attached to the body of the user, and an intemal / implantable component 112 that is configured to be implanted in or worn on the head of the user. In the examples of FIGs. 1A-1D, the implantable component 112 is sometimes referred to as a “cochlear implant.” FIG. 1 A illustrates the cochlear implant 112 implanted in the head 154 of a user, while FIG. IB is a schematic drawing of the external component 104 worn on the head 154 of the user. FIG. 1C is another schematic view of the cochlear implant system 102, while FIG. ID illustrates further details of the cochlear implant system 102. For ease of description, FIGs. 1 A-1D will generally be described together.

[0045] In the examples of FIGs. 1A-1D, the external component 104 comprises a sound processing unit 106, an external coil 108, and generally, a magnet system (e.g., one or more magnets) 150 fixed relative to the external coil 108. The cochlear implant 112 includes an implantable coil 114, an implant body 134, and an elongate stimulating assembly 116 configured to be implanted in the user’s cochlea. In one example, the sound processing unit 106 is an off-the-ear (OTE) sound processing unit, sometimes referred to herein as an OTE component, that is configured to send data and power to the implantable component 112. In general, an OTE sound processing unit is a component having a generally cylindrically shaped housing 111 and which is configured to be magnetically coupled to the user’s head 154 (e.g., includes an integrated external magnet 150 configured to be magnetically coupled to an internal / implantable magnet system 152 in the implantable component 112). The OTE sound processing unit 106 also includes an integrated external (headpiece) coil 108 (the external coil 108) that is configured to be inductively coupled to the implantable coil 114.

[0046] It is to be appreciated that the OTE sound processing unit 106 is merely illustrative of the external devices that could operate with implantable component 112. For example, in alternative examples, the external component 104 can comprise a behind-the-ear (BTE) sound processing unit configured to be attached to, and worn adjacent to, the user’s ear. A BTE soundAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1processing unit comprises a housing that is shaped to be worn on the outer ear of the user. In certain examples, the BTE is connected to a separate external coil assembly via a cable, where the external coil assembly is configured to be magnetically and inductively coupled to the implantable coil 114, while in other embodiments the BTE includes a coil disposed in or on the housing worn on the outer ear of the user. It is also to be appreciated that alternative external components could be located in the user’s ear canal, worn on the body, etc. For example, the external component could be a micro-BTE unit, an in-the-ear (ITE) unit, etc. In addition, as described elsewhere herein, the techniques could be implemented without an external component (e.g., in a totally-implantable arrangement, a mostly-implantable arrangement, etc.).

[0047] Although the cochlear implant system 102 includes the sound processing unit 106 and the cochlear implant 112, as described below, the cochlear implant 112 can operate independently from the sound processing unit 106, for at least a period, to stimulate the user. For example, the cochlear implant 112 can operate in a first general mode, sometimes referred to as an “external hearing mode,” in which the sound processing unit 106 captures sound signals which are then used as the basis for delivering stimulation signals to the user. The cochlear implant 112 can also operate in a second general mode, sometimes referred as an “invisible hearing” mode, in which the sound processing unit 106 is unable to provide sound signals to the cochlear implant 112 (e.g., the sound processing unit 106 is not present, the sound processing unit 106 is powered-off, the sound processing unit 106 is malfunctioning, etc.). As such, in the invisible hearing mode, the cochlear implant 112 captures sound signals itself via implantable sound sensors and then uses those sound signals as the basis for delivering stimulation signals to the user. Further details regarding operation of the cochlear implant 112 in the external hearing mode are provided below, followed by details regarding operation of the cochlear implant 112 in the invisible hearing mode. It is to be appreciated that reference to the external hearing mode and the invisible hearing mode is merely illustrative and that the cochlear implant 112 could also operate in alternative modes.

[0048] In FIGs. 1 A and 1C, the cochlear implant system 102 is shown with an external device 110, configured to implement aspects of the techniques presented. The external device 110, which is shown in greater detail in FIG. IE, is a computing device, such as a personal computer (e.g., laptop, desktop, tablet), a mobile phone (e.g., smartphone), a remote control unit, etc. The external device 110 and the cochlear implant system 102 (e.g., sound processing unit 106 or the cochlear implant 112) wirelessly communicate via a bi-directional communication link 126.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1The bi-directional communication link 126 can comprise, for example, a short-range communication, such as Bluetooth link, Bluetooth Low Energy (BLE) link, a proprietary link, etc.

[0049] Returning to the example of FIGs. 1 A-1D, the sound processing unit 106 of the external component 104 also comprises one or more input devices configured to capture and / or receive input signals (e.g., sound or data signals) at the sound processing unit 106. The one or more input devices include, for example, one or more sound input units / devices 118 (e.g., one or more external microphones, audio input ports, telecoils, etc.), one or more auxiliary input devices 128 (e.g., audio ports, such as a Direct Audio Input (DAI), data ports, such as a Universal Serial Bus (USB) port, cable port, etc.), and a short-range wireless transmitter / receiver (wireless transceiver) 120 (e.g., for communication with the external device 110), each located in, on or near the sound processing unit 106. However, it is to be appreciated that one or more input devices can include additional types of input devices and / or less input devices (e.g., the short-range wireless transceiver 120 and / or one or more auxiliary input devices 128 could be omitted).

[0050] The sound processing unit 106 also comprises the external coil 108, a charging coil 130, a closely-coupled radio frequency transmitter / receiver (RF transceiver) 122, at least one rechargeable battery 132, and an external sound processing module 124. The external sound processing module 124 can be configured to perform a number of operations that are represented in FIG. ID by a sound processor 133. The sound processor 133 can be formed by one or more processors (e.g., one or more Digital Signal Processors (DSPs), one or more uC cores, etc.), firmware, software, etc. arranged to perform operations described herein. That is, the sound processor 133 can each be implemented as firmware elements, partially or fully implemented with digital logic gates in one or more application-specific integrated circuits (ASICs), partially or fully in software, etc. Although FIG. ID illustrates the sound processor 133 as being implemented / performed at the external sound processing module 124, it is to be appreciated that these elements (e.g., functional operations) could also or alternatively be implemented / performed as part of the implantable sound processing module 158, as part of the external device 110, etc.

[0051] Returning to the example of FIGs. 1A-1D, the implantable component 112 comprises an implant body (main module) 134, a lead region 136, and the stimulating assembly 116, all configured to be implanted under the skin (tissue) 115 of the user. The implant body 134 generally comprises a hermetically-sealed housing 138 that includes, in certain examples, atAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1least one power source 125 (e.g., one or more batteries, one or more capacitors, etc.), in which the RF interface circuitry 140 and a stimulator unit 142 are disposed. The implant body 134 also includes the internal / implantable coil 114 that is generally external to the housing 138, but which is connected to the RF interface circuitry 140 via a hermetic feedthrough (not shown in FIG. ID).

[0052] As described further below, at least the implantable coil 114 is disposed in a biocompatible encapsulation layer (encapsulation) 101. The encapsulation layer 101 includes a magnet pocket (not shown in FIG. 1C), which can be any suitable feature in the encapsulation layer 101 configured to receive and retain an implantable magnet system 152. The implantable magnet system 152 comprises one or more magnets / magnetic components, potentially disposed in a hermetic housing. The magnets / magnetic materials disposed of an implantable magnet system can take any of a number of different forms / arrangements and can include, for example, fixed magnets / magnetic materials, uniaxially or biaxially rotatable magnets (e.g., diametric magnets), flexible omnidirectional rotating magnetic arrays (FORMAs) (e.g., miniaturized magnetic balls made sealed insides interconnected cavities where the magnetic balls can rotate freely in the cavities), etc.

[0053] As noted, the stimulating assembly 116 is configured to be at least partially implanted in the user’s cochlea. The stimulating assembly 116 includes a plurality of longitudinally spaced intra-cochlear electrical stimulating contacts (electrodes) 144 that collectively form a contact array (electrode array) 146 for delivery of electrical stimulation (current) to the user’s cochlea. The stimulating assembly 116 extends through an opening in the user’s cochlea (e.g., cochleostomy, the round window, etc.) and has a proximal end connected to stimulator unit 142 via lead region 136 and a hermetic feedthrough (not shown in FIG. ID). Lead region 136 includes a plurality of conductors (wires) that electrically couple the electrodes 144 to the stimulator unit 142. The implantable component 112 also includes an electrode outside of the cochlea, sometimes referred to as the extra-cochlear electrode (ECE) 139.

[0054] As noted, the cochlear implant system 102 includes the external coil 108 and the implantable coil 114. The external magnet 150 is positioned adjacent to the external coil 108, and the internal / implantable magnet system 152 is positioned adjacent to the implantable coil 114. The external magnet 150 and the internal / implantable magnet system 152 adjacent to the external coil 108 and the internal / implantable coil 114, respectively, facilitate the operational alignment of the external coil 108 with the implantable coil 114. This operational alignment of the coils enables the external component 104 to transmit data and power to the implantableAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1component 112 via a closely-coupled wireless link 148 formed between the external coil 108 with the implantable coil 114. In certain examples, the closely-coupled wireless link 148 is an RF link. However, various other types of energy transfer, such as infrared (IR), electromagnetic, capacitive, and inductive transfer, can be used to transfer the power and / or data from an external component to an implantable component and, as such, FIG. ID illustrates only one example arrangement.

[0055] As noted above, the sound processing unit 106 includes the external sound processing module 124. The external sound processing module 124 is configured to process the received input audio signals (received at one or more of the input devices, such as sound input devices 118 and / or auxiliary input devices 128) and convert the received input audio signals into output control signals for use in stimulating a first ear of a user or user (i.e., the external sound processing module 124 is configured to perform sound processing on input signals received at the sound processing unit 106). Stated differently, the one or more processors (e.g., processing element(s) implementing firmware, software, etc.) in the external sound processing module 124 are configured to execute sound processing logic in memory to convert the received input audio signals into output control signals (stimulation signals) that represent electrical stimulation for delivery to the user.

[0056] As noted, FIG. ID illustrates an embodiment in which the external sound processing module 124 in the sound processing unit 106 generates the output control signals. In an alternative embodiment, the sound processing unit 106 can send less processed information (e.g., audio data) to the implantable component 112, and the sound processing operations (e.g., conversion of input sounds to output control signals 156) can be performed by a processor within the implantable component 112.

[0057] In FIG. ID, according to an example embodiment, output control signals (stimulation signals) are provided to the RF transceiver 122, which transcutaneously transfers the output control signals (e.g., in an encoded manner) to the implantable component 112 via the external coil 108 and the implantable coil 114. That is, the output control signals (stimulation signals) are received at the RF interface circuitry 140 via the implantable coil 114 and provided to the stimulator unit 142. The stimulator unit 142 is configured to utilize the output control signals to generate electrical stimulation signals (e.g., current signals) for delivery to the user’s cochlea via one or more of the stimulating contacts 144. In this way, the cochlear implant system 102 electrically stimulates the user’s auditory nerve cells, bypassing absent or defective hair cellsAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1that normally transduce acoustic vibrations into neural activity, in a manner that causes the user to perceive one or more components of the input audio signals (the received sound signals).

[0058] As detailed above, in the external hearing mode, the cochlear implant 112 receives processed sound signals from the sound processing unit 106. However, in the invisible hearing mode, the cochlear implant 112 is configured to capture and process sound signals for use in electrically stimulating the user’s auditory nerve cells. In particular, as shown in FIG. ID, an example embodiment of the cochlear implant 112 can include a plurality of implantable sound sensors 165(1), 165(2) that collectively form a sensor array 160, and an implantable sound processing module 158. Similar to the external sound processing module 124, the implantable sound processing module 158 can comprise, for example, one or more processors and a memory device (memory) that includes sound processing logic. The memory device can comprise any one or more of Non-Volatile Memory (NVM), Ferroelectric Random Access Memory (FRAM), read only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. The one or more processors are, for example, microprocessors or microcontrollers that execute instructions for the sound processing logic stored in memory device.

[0059] In the invisible hearing mode, the implantable sound sensors 165(1), 165(2) of the sensor array 160 are configured to detect / capture input sound signals 166 (e.g., acoustic sound signals, vibrations, etc.), which are provided to the implantable sound processing module 158. The implantable sound processing module 158 is configured to convert received input sound signals 166 (received at one or more of the implantable sound sensors 165(1), 165(2)) into output control signals 156 for use in stimulating the first ear of a user or user (i.e., the implantable sound processing module 158 is configured to perform sound processing operations). Stated differently, the one or more processors (e.g., processing element(s) implementing firmware, software, etc.) in the implantable sound processing module 158 are configured to execute sound processing logic in memory to convert the received input sound signals 166 into output control signals 156 that are provided to the stimulator unit 142. The stimulator unit 142 is configured to utilize the output control signals 156 to generate electrical stimulation signals (e.g., current signals) for delivery to the user’s cochlea, thereby bypassing the absent or defective hair cells that normally transduce acoustic vibrations into neural activity.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0060] It is to be appreciated that the above description of the so-called external hearing mode and the so-called invisible hearing mode are merely illustrative and that the cochlear implant system 102 could operate differently in different embodiments. For example, in one alternative implementation of the external hearing mode, the cochlear implant 112 could use signals captured by the sound input devices 118 and the implantable sound sensors 165(1), 165(2) of sensor array 160 in generating stimulation signals for delivery to the user.

[0061] FIG. IE is a block diagram illustrating one example arrangement for an external computing device 110 configured to perform one or more operations in accordance with certain embodiments presented herein. As shown in FIG. IE, in its most basic configuration, the external computing device 110 includes at least one processing unit 183 and a memory 184. The processing unit 183 includes one or more hardware or software processors (e.g., Central Processing Units) that can obtain and execute instructions. The processing unit 183 can communicate with and control the performance of other components of the external computing device 110. The memory 184 is one or more software or hardware-based computer-readable storage media operable to store information accessible by the processing unit 183. The memory 184 can store, among other things, instructions executable by the processing unit 183 to implement applications or cause performance of operations described herein, as well as other data. The memory 184 can be volatile memory (e.g., RAM), non-volatile memory (e.g., ROM), or combinations thereof. The memory 184 can include transitory memory or non-transitory memory. The memory 184 can also include one or more removable or non-removable storage devices. In examples, the memory 184 can include RAM, ROM) EEPROM (Electronically-Erasable Programmable Read-Only Memory), flash memory, optical disc storage, magnetic storage, solid state storage, or any other memory media usable to store information for later access. By way of example, and not limitation, the memory 184 can include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, other wireless media, or combinations thereof. In certain embodiments, the memory 184 comprises logic 195 that, when executed, enables the processing unit 183 to perform aspects of the techniques presented.

[0062] In the illustrated example of FIG. IE, the external computing device 110 further includes a network adapter 186, one or more input devices 187, and one or more output devices 188. The external computing device 110 can include other components, such as a system bus, component interfaces, a graphics system, a power source (e.g., a battery), among other components. The network adapter 186 is a component of the external computing device 110Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1that provides network access (e.g., access to at least one network 189). The network adapter 186 can provide wired or wireless network access and can support one or more of a variety of communication technologies and protocols, such as Ethernet, cellular, Bluetooth, near-field communication, and RF, among others. The network adapter 186 can include one or more antennas and associated components configured for wireless communication according to one or more wireless communication technologies and protocols. The one or more input devices 187 are devices over which the external computing device 110 receives input from a user. The one or more input devices 187 can include physically-actuatable user-interface elements (e.g., buttons, switches, or dials), a keypad, keyboard, mouse, touchscreen, and voice input devices, among other input devices that can accept user input. The one or more output devices 188 are devices by which the external computing device 110 is able to provide output to a user. The output devices 188 can include a display 190 (e.g., a liquid crystal display (LCD)) and one or more speakers 191, among other output devices for presentation of visual or audible information to the user, a clinician, an audiologist, or other user.

[0063] It is to be appreciated that the arrangement for the external computing device 110 shown in FIG. IE is merely illustrative and that aspects of the techniques presented herein can be implemented at a number of different types of systems / devices including any combination of hardware, software, and / or firmware configured to perform the functions described herein. For example, the external computing device 110 can be a personal computer (e.g., a desktop or laptop computer), a hand-held device (e.g., a tablet computer), a mobile device (e.g., a smartphone), a surgical system, and / or any other electronic device having the capabilities to perform the associated operations described elsewhere herein.

[0064] FIG. 2A illustrates a system 200A that is configured to engage a user in one or more interactions to develop the user’s aural language competency. In particular, the system 200A includes a sleeve structure 211 A, an aural language competency development device 213A, and a sound input unit, such as a microphone 215 A. In certain embodiments, the aural language competency development device 213 A and / or the microphone 215 A are disposed within the sleeve structure 211 A. In certain embodiments, the aural language competency development device 213 A and / or the microphone 215 A are attached to an exterior of the sleeve structure 211 A via any suitable attachment mechanism. In certain embodiments, the aural language competency development device 213A can include one or more sensors (e.g., a sound input unit, such as microphone, a motion sensor, an image sensor, etc.) in addition to the microphone 215A.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0065] In certain embodiments, the microphone 215 A can be connected to the aural language competency development device 213 A via a wired connection. In certain embodiments, the microphone 215 A can be connected to the aural language competency development device 213 A via a wireless connection. In one example, the wireless connection can be established via a bi-directional communication link. The bi-directional communication link can comprise, for example, a short-range communication, such as Bluetooth link, Bluetooth Low Energy (BLE) link, a proprietary link, etc. In certain embodiments, the aural language competency development device 213A can be external to the sleeve structure 211 A and connected to the microphone 215A and / or other sensors in the sleeve structure 211 A through a network via suitable wireless communication technology.

[0066] The sleeve structure 211 A can be a character object exhibiting certain visual characteristics and / or personalities. In certain embodiments, the character object can be a toy that matches the personality / needs of the user and engages with the user in one or more interactions. That is, for example, the sleeve structure 211 A, via the microphone 215A, can obtain an initial verbal input from the user indicating that the user wishes to begin an interaction (e.g., a conversation) with the sleeve structure 211 A. In response to the verbal input, the aural language competency development device 213 A is configured to detect an intent of the user and generate one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) that can be delivered to the user via one or more acoustic transducers. The one or more stimuli can recruit the hearing faculties of the user to elicit an expected response, thus engaging the user in a sustained interaction.

[0067] In certain embodiments, based on the responses from the user, the aural language competency development device 213 A is configured to adapt the one or more stimuli (e.g., the one or more conversational dialogue parts) based on the user’s experience level and / or engagement level. In one example, the user can be a hearing device user engaging with the sleeve structure 211 A in an aural rehabilitation session. In another example, the user can engage with the sleeve structure 211 A in their daily activities to develop their aural language competency or communication skills. Further, in certain embodiments, the aural language competency development device 213 A can be configured to perform sound analysis on the user response to identify vocal repetition, vocal patterns, or other suitable characteristics.

[0068] FIG 2B is a functional block diagram of the aural language competency development device 213 A that is configured to generate one or more stimuli to interact with a user in one or more aural competency development sessions. The aural language competency developmentAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1device 213A comprises a sound input unit connection 221, a charging port 223, a Bluetooth connection 225, a battery 227, an artificial intelligence (Al) data storage 229, an audio processing unit 231, a memory 233 storing an intelligent agent 235, a processor 237, a database 239, an acoustic transducer 241, and a controller 243 for managing communication between the aural language competency development device 213 A and one or more external devices.

[0069] The language competency development device 213 A is connected to a sound input unit (e.g., microphone 215A) configured to receive one or more input audio signals (e.g., verbal inputs from a user) in a wired connection via the sound input unit connection 221 or in a wireless connection via the Bluetooth connection 225. In certain embodiments, the sound input unit connection 221 or the Bluetooth connection 225 can be configured to transmit the one or more received input audio signals to the audio processing unit 231. The audio processing unit 231 is configured to perform processing techniques to convert the one or input audio signals into one or more datasets suitable for audio processing models. For example, the audio processing unit 231 is configured to normalize the one or more audio signals, remove noise, and / or perform feature extraction to generate one or more audio datasets. The audio datasets can be stored in the Al data storage 229 for training the intelligent agent 235 to generate one or more conversational dialogue parts in an interaction with the user. The Bluetooth connection 225, battery 227, and Al data storage 229 can be connected to the charging port 223.

[0070] The processor 237, operably coupled to the memory 233, the acoustic transducer 241, and at least one sensor (e.g., microphone 215A), can be configured to execute the intelligent agent 235. In certain embodiments, the intelligent agent 235 can be implemented as a software module and / or application arranged to perform operations described herein. That is, for example, the intelligent agent 235 can be configured to obtain the one or more audio datasets from the audio processing unit 231, assess the context and / or intent of the user’s verbal input, and generate one or more stimuli (e.g., conversational dialogue parts, sounds, sound sequences, etc.) to engage the user in an interaction (e.g., natural language conversation). For example, the one or more conversational dialogue parts can be generated based on pre-programmed lessons pre-curated by an expert (e.g., speech therapist, clinician, subject matter expert, etc.) that are stored in the database 239. The one or more conversational dialogue parts can be delivered to the user via the acoustic transducer 241 to recruit one or more behavioral responses such as a verbal response and / or bodily movement (e.g., eye movement, hand gesture, etc.).

[0071] In certain embodiments, the intelligent agent 235 can train a machine learning model to generate the one or more stimuli that are delivered to the user. For example, the machineAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1learning model can be trained to generate one or more conversational dialogue parts exhibiting certain linguistic characteristics based on a personality map associated with a character object, one or more historical conversations associated with the user and / or a population of users (e.g., hearing device users), and / or personal data (e.g., engagement level, experience level, medical records, etc.) associated with the user and / or a population of users (e.g., hearing device users). That is, for example, the trained machine learning model can be configured to select one or more conversational dialogue parts provided in one or more pre-curated lesson plans or generate one or more conversational dialogue parts that adhere to requirements / guidelines established in the one or more pre-curated lesson plans. The machine learning model can include a transformer-based large language model, a neural network (e.g., multi-layer perceptron, convolutional neural network, recurrent neural network, etc.), a support vector machine, generative Al models, etc. Transformer-based large language models can include one or more transformers with self-attention mechanisms configured to learn and understand contextual relationships in the input data. The machine learning model can be an unsupervised, semi-supervised, or supervised model.

[0072] The one or more conversational dialogue parts provided by the trained machine learning model can adhere to a speech style (e.g., rate of speech, tone, volume, accent, background noise, etc.), vocabulary, sentence structure, etc. that best engage a user of a certain experience level (e.g., age, education level, aural language competency level, etc.) to meet a predetermined engagement goal (e.g., to develop and / or rehabilitate aural language competency, to develop language comprehension, to expand the user’s vocabulary, etc.). For example, the one or more conversational dialogue parts generated by the trained machine learning model can include unique word combination and phrases, unique topic areas guided by user interactions, and / or unique focus areas based on problem sounds. In certain embodiments, the one or more conversational dialogue parts can request the user perform sound differentiation between similar words.

[0073] Further, in certain embodiments, the intelligent agent 235 is associated with a personality map that establishes the personality of the intelligent agent 235. That is, for example, the personality map of the intelligent agent 235 can include a tone of voice (e.g., conversational, educational, etc.), a dialect or accent, a type of vocabulary, or any suitable linguistic or speech characteristic. After the trained machine learning model generates the one or more conversational dialogue parts, the intelligent agent 235 can be configured to convert the one or more conversational dialogue parts (e.g., in text format) to an acoustic stimulus in aAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1voice reflecting one or more characteristics identified in the personality map of the intelligent agent 235 (e.g., a toy bear, a toy dog, etc.) via suitable text-to-speech techniques. Then, the acoustic stimulus can be delivered to the user via the acoustic transducer 241.

[0074] The aural language competency development device 213 A can be configured to capture the user’s response (e.g., verbal response, bodily movement, etc.) via one or more sensors (e.g., microphone 215 A, motion sensor, etc.) to the acoustic stimulus. Based on the user’s response, the intelligent agent 235 is configured to assess the user’s engagement level and / or experience level and update its personality map. The updated personality map can be provided as feedback to further train the machine learning model to generate one or more conversational dialogue parts that would meet a predetermined engagement goal. In certain embodiments, the machine learning model can be trained iteratively until the predetermined engagement goal is met. In certain embodiments, the engagement goal can be an expected behavioural response (e.g., an expected verbal response, an expected movement, etc.) that is to be elicited based on an acoustic stimulus.

[0075] FIG. 2C illustrates a system 200C that is configured to engage a user in one or more interactions to develop the user’s aural language competency. In particular, the system 200C includes a sleeve structure 211C and an aural language competency development device 213C (e.g., functionally / operationally similar to aural language competency development device 213 A). The sleeve structure 211C can be a character object exhibiting certain visual characteristics and / or personalities. In certain embodiments, the character object can be a toy that engages with a user in one or more verbal interactions to develop the aural competency of the user. That is, for example, the sleeve structure 211C, via one or more sensors (e.g., sound input units, such as a microphone, a motion sensor, etc.), can obtain an input (e.g., verbal input, gesture, etc.) from the user indicating that the user wishes to begin a conversation with the sleeve structure 211C. In response to the input from the user, the aural language competency development device 213C is configured to detect an intent of the user and generate one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) that can be delivered to the user via one or more acoustic transducers. The one or more stimuli can recruit the hearing faculties of the user to elicit an expected response, thus engaging the user in a sustained interaction. In certain embodiments, based on the responses from the user, the aural language competency development device 213C is configured to adapt the one or more stimuli (e.g., the one or more conversational dialogue parts) based on the user’s experience level and / or engagement level.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0076] In certain embodiments, the aural language competency development device 213C can be attached to an exterior of the sleeve structure 211C via any suitable attachment mechanism (e.g., adhesion mechanism, magnetic closure, etc.) or placed into an opening (e.g., pocket) on or attached to the sleeve structure 211C. Thus, the aural language competency development device 213C can be easily retrieved and transferred to another sleeve structure, thereby increasing the flexibility and adaptability of the aural language competency development device 213C in providing engaging interactions to one or more users. For example, the aural language competency development device 213C can be removed from the sleeve structure 211C and placed into another sleeve structure (e.g., another character object) with different appearances to accommodate user interests and / or requirements.

[0077] FIG. 2D illustrates a system 200D that is configured to engage a user in one or more interactions to develop the user’s aural language competency. In particular, the system 200D includes a sleeve structure 21 ID and an aural language competency development device 213D (e.g., functionally / operationally similar to aural language competency development device 213 A). The sleeve structure 21 ID can be a character object exhibiting certain visual characteristics and / or personalities. In certain embodiments, the character object can be a toy that engages with a user in one or more verbal interactions to develop the aural competency of the user. That is, for example, the sleeve structure 21 ID, via one or more sensors (e.g., a sound input unit, such as a microphone, a motion sensor, etc.), can obtain an input (e.g., verbal input, gesture, etc.) from the user indicating that the user wishes to begin a conversation with the sleeve structure 21 ID. In response to the input from the user, the aural language competency development device 213D is configured to detect an intent of the user and generate one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) that can be delivered to the user via one or more acoustic transducers. The one or more stimuli can recruit the hearing faculties of the user to elicit an expected response, thus engaging the user in a sustained interaction. In certain embodiments, based on the responses from the user, the aural language competency development device 213D is configured to adapt the one or more stimuli (e.g., the one or more conversational dialogue parts) based on the user’s experience level and / or engagement level.

[0078] In certain embodiments, the aural language competency development device 213D is disposed within the sleeve structure 21 ID such that the aural language competency development device 213D is not visible to an observer. For example, the sleeve structure 21 ID can be a character object (e.g., toy bear) having an opening through which the aural languageAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1competency development device 213D can be placed or removed. In certain embodiments, the opening can be secured via a zipper or any suitable mechanism. By keeping the aural language competency development device 213D disposed within the sleeve structure 21 ID, the sleeve structure 21 ID retains the look and feel of a character object, thus providing a more realistic and playful interactive experience.

[0079] FIG. 3 A illustrates a system 300A that is configured to engage a user in a conversation to develop the user’s aural language competency. In particular, the system 300A includes a sleeve structure 311 and a user 351. The sleeve structure 311 comprises a sound input unit 315, such as a microphone 315, and an aural language competency development device 313 that, in certain examples, can be functionally / operationally similar to aural language competency development device 213 A. In certain embodiments, the aural language competency development device 313 and / or the sound input unit 315 are disposed within the sleeve structure 311. The sleeve structure 311 can be configured to interact with a user 351 in one or more conversations. That is, for example, the user 351 can initiate a conversation with sleeve structure 311 using one or more trigger words such as a greeting (e.g., “Hey” or “Hi”), a name associated with the sleeve structure 311 (e.g., “Soro”), and / or any suitable trigger words. In certain embodiments, the name associated with the sleeve structure 311 can be customized by the user, thus enabling the user to establish a personal relationship with the sleeve structure 311.

[0080] In one example, the user 351 initiates a conversation with the sleeve structure 311 with a verbal input including a greeting and the name of the sleeve structure 311 (e.g., “Hey Soro!”). The sleeve structure 311, via the sound input unit 315, can be configured to obtain one or more input audio signals representing the user’s verbal input. The aural language competency development device 313 is configured to process the one or more input audio signals and generate one or more conversational dialogue parts that can elicit one or more responses from the user as described above. That is, for example, the aural language competency development device 313 can generate one or more conversational dialogue parts representing a response to the user, such as “Hi Peter! How are you?” The one or more conversational dialogue parts can be converted from text to an acoustic stimulus using any suitable text-to-speech techniques. Then, the acoustic stimulus can be delivered to the user via a transducer (e.g., acoustic transducer). In certain embodiments, the transducer can be located in the sleeve structure 311 (e.g., within the aural language competency development device 313) or near / around the sleeveAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1structure 311 such that the transducer is spatially-separated from the sound input unit 315 and / or other sensor(s) in the sleeve structure 311.

[0081] FIG. 3B is a flow chart 300B illustrating an exemplary interaction between a sleeve structure (e.g., sleeve structure 311) and a user (e.g., user 351). For example, at 353, the user can initiate an interaction (e.g., a conversation) with the sleeve structure via a verbal input that includes a greeting and the name of the sleeve structure (e.g., “Hey Soro!”). At 355, the sleeve structure (e.g., a toy bear named “Soro”) having an aural language competency development device disposed within can respond with a greeting, such as “Hi Peter! How are you?” Then, at 357, the user provides a response of “I’m bored...” Based on the user’s response, the aural language competency development device is configured to execute an intelligent agent to parse the user’s response and assess the user’s intent.

[0082] That is, for example, the intelligent agent is configured to apply a trained machine learning model to select one or more conversational dialogue parts provided in one or more pre-curated lesson plans or generate one or more conversational dialogue parts that adhere to requirements / guidelines established in the one or more pre-curated lesson plans. For example, based on the keyword “bored” in the user’s response, the trained machine learning model is configured to determine that the user’s engagement level can be increased with an educational game (e.g., a guessing game) to meet a predetermined engagement goal (e.g., developing the user’ s aural language competency). Thus, the intelligent agent, via the trained machine learning model, can generate the one or more conversational dialogue parts “Hmm, let me see. How about we play the guessing game?”, which is provided as an audio output at 359.

[0083] Upon obtaining a user response of “Yes” at 361, the intelligent agent is configured to provide the one or more conversational dialogue parts “Which one of these tunes is in ascending order?”, which can be delivered via the transducer at 363. Then, the aural language competency development device can be configured to select two tunes from pre-curated tunes or generate two tunes that adhere to requirements / guidelines established in the pre-curated lesson plans. The two tunes are delivered to the user via the transducer at 365. One or more sensors (e.g., a sound input unit, such as microphone, a motion sensor, an image sensor, etc.) can be configured to detect a response (e.g., verbal response or gesture) from the user. For example, a microphone located within or near / around the sleeve structure can detect the user saying “The first one” at 367 or “The second one” at 369. The aural language competency development device can be configured to provide a response of “That is correct!” at 371 inAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1response to the user saying “The first one” or a response of “Not quite, let’s try again!” at 373 in response to the user saying “The second one.”

[0084] In certain embodiments, the sleeve structure can deliver the stimulus (e.g., one or more conversational dialogue parts) in accordance with a personality as established by a personality map associated with the intelligent agent of the sleeve structure. The personality map can include a tone of voice (e.g., conversational, educational, etc.), a dialect or accent, type of vocabulary, or any suitable linguistic or speech characteristic. The intelligent agent can update the personality map based on the response from the user to adapt the next conversational dialogue parts to meet a predetermined engagement goal. For example, when the intelligent agent determines a user response indicates a lack of engagement, the intelligent agent can adjust its tone from educational to playful to elicit more engagement from the user.

[0085] FIGs. 4A and 4B collectively illustrate operation of a system 400 that is configured to engage a user in one or more interactions in an interactive environment 480. The system 400 includes a sleeve structure 411, an aural language competency development device 413, one or more transducers (e.g., acoustic transducers), and one or more sensors (e.g., a microphone 415, motion sensor, etc.). Disposed in the interactive environment 480 is a user 451, at least one environmental object 481, and / or a virtual assistant device 487. The interactive environment 480 can be a room, an outdoor space, or any suitable space that can be occupied by the user 451. In certain embodiments, the aural language competency development device 413, the at least one environmental object 481, and the virtual assistant device 487 can be connected via a wired or wireless network. In certain embodiments, the interactive environment 480 can include one or more sensors that are spatially separated from one or more transducers. For example, the one or more sensors can be located in or immediately around the sleeve structure 411, within the interactive environment 480, in a hearing device associated with the user 451, or any suitable location.

[0086] The sleeve structure 411 can be a character object exhibiting certain visual characteristics and / or personalities. In certain embodiments, the character object can be a toy that, using the aural language competency development device 413, engages with the user 451 in one or more interactions to develop the aural language competency of the user 451. The one or more sensors (e.g., a microphone 415, motion sensor, etc.) of the sleeve structure 411 can capture the response of the user 451 (e.g., verbal input, gesture, etc.). Based on the user response, the aural language competency development device 413 can be configured to generate one or more stimuli that can be delivered to the user 451 via the one or moreAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1transducers in the interactive environment 480. In certain embodiments, the one or more stimuli can be determined via an intelligent agent of the aural language competency development device 413. For example, the intelligent agent can be configured to apply a trained machine learning model to generate one or more stimuli that would recruit the hearing faculties of a user to elicit an expected response.

[0087] Further, the user 451 can interact with other objects and / or devices in the interactive environment 480, such as the at least one environmental object 481 and the virtual assistant device 487. The at least one environmental object 481 can include a transducer 483 (e.g., acoustic transducer) configured to deliver audio output and one or more sensors 485 configured to capture a response from the user 451. For example, the one or more sensors 485 can include a sound input unit (e.g., microphone) for detecting sound, a motion sensor for detecting movement of the user, or any suitable sensor. In certain embodiments, the at least one environmental object 481 can be a character object exhibiting certain visual characteristics and / or personalities. Moreover, the virtual assistant device 487 can include one or more sensors 489 (e.g., a sound input unit, such as microphone, a motion sensor, an image sensor, etc.), a transducer 493, a memory 491, and a processor 495. The processor 495 can be operably coupled to the memory 491 and can be configured to perform one or more operations of the virtual assistant device 487. In certain embodiments, the virtual assistant device 487 can be a mobile device configured to recognize voice and / or motion commands from a user and perform one or more tasks based on commands from the user.

[0088] In certain embodiments, the user 451 can converse with the system 400 (e.g., the aural language competency development device 413 in the sleeve structure 411) in an interactive manner, starting with a first interaction 497 A. For example, the system 400 can direct the user 451 to perform one or more tasks to evaluate the aural language competency of the user 451. The interaction 497A can include the system 400 delivering a prompt 499A (e.g., an acoustic prompt) having one or more conversational dialogue parts, such as “Hi Peter! Please say hello to my friend Doro the Dog over there!”, via a transducer of the sleeve structure 411. In certain embodiments, the instruction provided in the prompt 499A can be selected from a set of precurated lesson plans. For example, the aural language competency development device 413, via an intelligent agent, can determine that the prompt 499A represents an appropriate command to evaluate the aural language competency of the user 451 based on a profile of the user 451.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0089] A response from the user 451 can be captured via one or more sensors in the interactive environment 480, such as one or more sensors (e.g., microphone 415) in or near the sleeve structure 411, one or more sensors 485 in the at least one environmental object 481, or one or more sensors 489 in the virtual assistant device 487. For example, a response 499B (e.g., verbal response) can be captured and converted to text via any suitable speech-to-text techniques. The response 499B can indicate an affirmation from the user 451, such as “Sure! Let me walk over there.” Based on the verbal affirmation from the user, the aural language competency development device 413 can be configured to monitor any additional responses, such as bodily movement (e.g., walking, gesturing, eye movement, head movement, etc.) of the user 451, that would indicate the user 451 understands the instructions in the prompt 499A.

[0090] That is, for example, one or more sensors (e.g., one or more sensors 485, one or more sensors 489, or one or more sensors in or near the sleeve structure 411) in the interactive environment 480 can be configured to capture the user 451 walking towards the at least one environmental object 481 that is the character object of “Doro the Dog.” The aural language competency development device 413 can be configured to assess one or more responses from the user 451 to determine whether the one or more responses differ from one or more expected responses. For example, the aural language competency development device 413 can determine that one or more responses (e.g., response 499B and movement of the user) adequately reflect the user 451 understands instructions in the prompt 499 A. In certain embodiments, the user’s ability to recognize voice, speech, or sound can be measured based the one or more responses.

[0091] In certain embodiments, the transducer 483 and / or the one or more sensors 485 in the at least one environmental object 481 can be spatially separated from the one or more transducers and / or one or more sensors in or near the sleeve structure 411. In certain embodiments, the transducer 493 and / or the one or more sensors 489 in the virtual assistant device 487 can be spatially separated from the one or more transducers and / or one or more sensors in or near the sleeve structure 411. With these spatially-separated transducers and / or sensors in the interactive environment 480, one or more responses of the user 451 to the one or more stimuli can be captured even if the user 451 moves away from the sleeve structure 411 and interacts with another object / device in the interactive environment 480 (e.g., the at least one environmental object 481, the virtual assistant device 487, etc.). Thus, the user 451 can freely incorporate their own physical movement / activity into their interaction (e.g., conversational engagement) with the sleeve structure 411 (e.g., character object) to replicate familiar everyday interaction scenarios, such as a child physically playing with toys or adultsAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1physically engaging with a companion animal. As a result, user engagement can be increased through playful and natural physical activity, thus reducing inhibition and compensation strategies associated with “test” like conditions while simultaneously improving clinical compliance, validity and efficacy.

[0092] In certain embodiments, spatially-separated sensors and / or transducers can be leveraged for spatial play and interaction in a complex monitoring and / or auditory environment (e.g., spatially-separated transducers providing different kinds of audio sources). For example, the system 400 can ask the user 451 (e.g., a child) to interact with a specific object (e.g., at least one environmental object 481) at a distance from the sleeve structure 411 to confirm whether the user 451 understands the instructions. Moreover, in certain embodiments, spatially-separated sensors and / or transducers can provide continued play after the user 451 loses interest with the sleeve structure 411 / system 400. For example, the user 451 can be detected, via one or more sensors, as interacting with an environmental object (e.g., another toy in the room). The system 400 can reach the user 451 with one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) via a transducer that is located closer to the current position of the user 451. Based on the response of the user 451, the system 400 (e.g., aural language competency development device 413) can adapt the next stimulus based on the distance or perceived engagement level of the user 451.

[0093] In certain embodiments, at least one sensor in the interactive environment 480 is mutually spaced from at least two transducers. That is, for example, at least one sensor in or near the sleeve structure 411 is mutually spaced from the transducer 483 and the transducer 493. In one or more exemplary interactions, the one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) determined and / or selected by the aural language competency development device 413 can be provided via the mutually spaced transducers (e.g., transducer 483 and transducer 493), thus creating a highly directional sound environment in which naturalized interactivity is provided. In certain embodiments, spatially-separated sensors and / or transducers can be leveraged to test the user 451’s multidirectional auditory ability in a loud environment. For example, a transducer can be placed in a louder area of a house (e.g., kitchen or near a television) to test the user 451’s ability to differentiate sounds. This can assist with a user’s ability to cope in louder environments (e.g., classroom, supermarket, etc.)

[0094] As shown in FIG. 4B, subsequent to the interaction 497A (FIG. 4A), the user 451 can engage with the at least one environmental object 481 in an interaction 497B by following theAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1instructions in the prompt 499A (“Hi Peter! Please say hello to my friend Doro the Dog over there!”) from the sleeve structure 411. For example, the interaction 497B includes a prompt 499C from the user 451 (e.g., “Hi Doro!”) and a response 499D provided via the transducer 483 of the at least one environmental object 481 (e.g., “Nice to meet you, Peter”). In certain embodiments, the aural language competency development device 413 can evaluate the prompt 499C from the user 451 and determine that the user 451 has adequately followed the instructions provided by the sleeve structure 411 in the prompt 499A. That is, for example, the aural language competency development device 413 can determine that an engagement goal has been met (e.g., an engagement goal of ensuring the user 451 is engaged and following the instructions from the sleeve structure 411).

[0095] FIG. 5 is a flowchart illustrating a method 500 for determining a communication style based on an engagement level of a user. For example, the communication can be between a sleeve structure (e.g., character object) and a user. Method 500 begins at 501 where an engagement level of a user is determined. In certain embodiments, the engagement level can be associated with a predetermined threshold. If the engagement level is above the predetermined threshold, then the user is determined as having engagement at 503 and communication is provided to the user using the same communication style at 505. However, if the engagement level is not above the predetermined threshold, then the user is determined to have no engagement at 507 and the communication style is changed at 509. The operations of method 500 can proceed iteratively, thus providing continuous adjustments to the communication style to increase the engagement level of the user. In certain embodiments, the engagement level can be determined based on one or more of: response time, response length, accuracy, repeatability, willingness to re-engage with the sleeve structure, or any suitable measure of engagement.

[0096] FIG. 6A is a flow chart illustrating a method 600 for adapting an educational routine based on a user’s engagement level. Method 600 begins at 601 where an artificial intelligence (Al) system begins an interaction (e.g., conversation, educational routine, etc.) with a user. In certain embodiments, the Al system can include a sleeve structure (e.g., character object) having an intelligent agent as described in various embodiments herein. Then, at 603, a personality map, a character voice (e.g., voice of a toy bear, toy dog, etc.), and / or any suitable data for establishing the personality of the Al system are loaded. For example, the personality map can include a communication style (e.g., friendly, assertive, etc.), a tone of voice (e.g., conversational, educational, etc.), a dialect or accent, a type of vocabulary (e.g., formal, casual,Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1etc.), or any suitable linguistic characteristic. At 605, the Al system can be configured to engage the user in a conversation based on previous conversation history (e.g., conversation history between the Al system and the user or conversation history between the user and an associate (e.g., family member, friend, clinician, etc.)) and / or metrics associated with the user, such as auditory progression level, academic progression level, results from previous speech tests, etc.

[0097] At 607, the Al system can evaluate whether the user is engaged and / or responding positively. In certain embodiments, the user’s response, captured via one or more sensors, can include a verbal response, a hand gesture, an eye movement, or any suitable response. When the Al system determines the user’s response does not match an expected response (e.g., engagement level fails to meet a threshold), the Al system can be configured to adjust the personality map at 609. That is, for example, the personality map can be adjusted to reflect the one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) provided by the Al system fail to elicit an expected response. The personality map can further reflect that one or more characteristics (e.g., communication style, tone of voice, etc.) of the Al system’s personality fail to elicit an expected response. Then, the method 600A returns to 605, where the Al system continues to engage the user in a conversation until the user becomes engaged and / or respond positively.

[0098] However, when the Al system determines the user’s response matches an expected response (e.g., engagement level meets a threshold), the Al system can be configured to adjust the personality map at 611 to reflect the one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) provided by the Al system elicited an expected / positive response. The personality map can further reflect that one or more characteristics (e.g., communication style, tone of voice, etc.) of the Al system’s personality elicited an expected / positive response. After the personality map is updated at 611, one or more lesson plans, the user’ s auditory progression level, and / or the user’s academic progression level can be obtained. In certain embodiments, the one or more lesson plans can include preprogrammed lesson plans obtained from a database, such as lesson plans curated by one or more experts (e.g., clinicians, researchers, subject matter experts, etc.). In one example, the pre-curated lesson plans are specifically designed for developing (e.g., habitualizing or rehabilitating) an individual’s aural language competency. Based on the information obtained at 613, the Al system can be configured to begin or continue an educational routine at 615.

[0099] FIG. 6B is a flowchart illustrating further details of the method 600, in accordance with certain embodiments presented herein. Continuing from FIG. 6A, the Al system can beAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1configured to begin or continue an educational routine at 615. In certain embodiments, the educational routine can include one or more of lessons, exercises, quizzes, games, or any suitable interaction for developing (e.g., habitualizing or rehabilitating) an individual’s aural language competency. For example, a trained machine learning model can be leveraged to select an educational routine from one or more pre-curated lesson plans or generate an educational routine that adheres to requirements / guidelines provided in the one or more precurated lesson plans. After engaging the user in the educational routine, the Al system can be configured to evaluate whether the user is engaging with the educational routine at 617. In certain embodiments, the user’s response, captured via one or more sensors, can include a verbal response, a hand gesture, an eye movement, or any suitable response. When the Al system determines the user’s response does not match an expected response (e.g., engagement level fails to meet a threshold), the Al system can be configured to adjust the personality map at 619. That is, for example, the personality map can be adjusted to reflect the one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) provided by the Al system fail to elicit an expected response. The personality map can further reflect that one or more characteristics (e.g., communication style, tone of voice, etc.) of the Al system’s personality fail to elicit an expected response. Then, at 621, the Al system can be configured to record that the educational routine has been completed. At 623, the Al system can be configured to engage the user in another educational routine.[ooioo] However, when the Al system determines the user’s response matches an expected response (e.g., engagement level meets a threshold) at 617, the Al system can be configured to determine whether the user shows improvement with a currently tested ability (e.g., aural language competency) at 625 when compared to a baseline (e.g., user’s previous performance, average performance of individuals in the user’s age cohort, etc.). When improvement has been detected at 625, the user’s auditory change can be recorded at 627. However, when improvement has not been detected at 625, the Al system is configured to note the educational routine’s targeted skill difficulty for a later interaction at 629. That is, for example, the Al system can re-teach the noted skill with the targeted difficulty level in a later interaction. Then, the user’s auditory change can be recorded at 627. After the user’s auditory change is recorded at 627, the Al system can be configured to engage the user in another educational routine at 623. In certain embodiments, the Al system can select and / or generate an educational routine that targets certain skills / tasks in which the user has not shown improvement.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1[ooioi] FIG. 7A is a flowchart illustrating a method 700A for determining a user’s favorite interaction. Method 700A begins at 741 where an artificial intelligence (Al) system engages a user in an interaction. In certain embodiments, the Al system can include a sleeve structure (e.g., character object) having an intelligent agent as described in various embodiments herein. For example, the interaction can include an educational routine, a conversation, or any suitable interaction for developing (e.g., habitualizing or rehabilitating) an individual’s aural language competency. In certain embodiments, the educational routine can include one or more of lessons, exercises, quizzes, games, etc. At 743, the Al system can be configured to obtain one or more features of the user’s response to one or more stimuli (e.g., one or more conversational dialogue parts, sounds, or sound sequences, etc.) provided by the Al system in the interaction. For example, the one or more features of the user’s response can include response time, tone of voice, response matter, etc. Then, at the end of the interaction, the Al system can be configured to ask if the user would like to continue the interaction (e.g., educational game) and obtain the user’s response at 745. For example, the Al system can ask whether the user wants to play the educational game again and the response from the user provides an indicator of whether the user enjoyed the educational game. Then, at 747, the user’s response data (e.g., whether the user indicated a willingness to play an educational game again) is provided to train a machine learning model / algorithm for predicting a user’s favorite interaction (e.g., exercise, game, etc.).

[0102] FIG. 7B is a flowchart illustrating a method 700B for generating one or more lesson plans adapted to incorporate one or input words or sounds. Method 700B begins at 751 where one or more input words or sounds are obtained. For example, the one or more input words or sounds can be obtained from a parent or guardian, a teacher, a medical practitioner (e.g., a clinician), a subject matter expert, a child’s weekly word list from school, or any suitable source. At 753, an artificial intelligence (Al) system is trained to generate one or more lesson plans adapted to incorporate the one or more input words or sounds. In certain embodiments, the Al system can include a sleeve structure (e.g., character object) having an intelligent agent as described in various embodiments herein. Then, at 755, one or more lesson plans adapted to incorporate the input words are generated. At 757, one or more lesson plans adapted to incorporate the input sounds (e.g., vowel sounds) are generated. In certain embodiments, the lesson plans can be used to guide interactions (e.g., educational games / activities, conversation, etc.) between the Al system and the user.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0103] In one example, the one or more input words or sounds are extracted from a weekly word list associated with a child. The weekly word list can be provided by the school and uploaded by a parent to a software application that controls a sleeve structure that interacts with the child. Based on the weekly word list, the Al system can be configured to generate one or more lesson plans that incorporate the words and / or sounds (e.g., “cat”, “and”, and “pet”) provided in the weekly word list. For example, the Al system can be configured to use the word “cat” when testing the child’s ability to understand the use of “and” / “buf ’ in a sentence. For example, the Al system can be configured to generate the question “Harry, I know you like cats and dogs, but which do you prefer?” for a lesson plan. Further, the Al system can be configured to use the vowel sound in “pet” when testing the child’s ability to notice when words rhyme. For example, instead of using the word ‘pet’, the Al system can be configured to use “bed” and “net” if the child is already familiar with the words. For example, the Al system can be configured to generate the question “Harry, can you tell me which of these three words rhyme? Bed, net, and pat” for a lesson plan.

[0104] In another example, the one or more input words or sounds can be provided by a teacher or parent of the user (e.g., a child) who is practicing accurate production of vowel sounds, such as the ‘ai’ sound in “bye-bye,” hi,” “dry,” and “fly.” Based on the one or more input words or sounds, the AI system can be configured to generate the question “Harry, can you go over and say hi to the monkey (on your playmat).” The AI system can be configured to detect a response from the user, such as the user turning to the monkey to say “Hi.” Based on the user response, the AI system can be configured to generate the following conversational dialogue parts: “Good job Harry. Now let’s say bye-bye to the monkey.” Then, a user response with “Bye-bye monkey” can be detected. Thus, the AI system can be leveraged to engage the user in learning vowel sounds via one or more interactions.

[0105] FIG. 7C is a flowchart illustrating a method 700C for generating one or more conversations adapted to incorporate one or more words used by a user. Method 700C begins at 761, where one or more words (e.g., “soft,” “dog,” or “milk”) used by a user in an interaction (e.g., conversation) with an artificial intelligence (AI system) or another individual (e.g., teacher, parent, etc.) are obtained. In certain embodiments, the AI system can include a sleeve structure (e.g., character object) having an intelligent agent as described in various embodiments herein. At 763, the AI system can be configured to record the use of the one or more words by the user and train a machine learning model to adapt one or more lesson plans based on the one or more words.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0106] At 765, based on the one or more lesson plans, the Al system can be configured to generate one or more conversational dialogue parts that include a theme associated with the one or more words. For example, the Al system can be configured to generate one or more conversational dialogue parts that discuss pets / animals based on the input word “dog” in order to expand the user’s vocabulary. At 767, based on the one or more lesson plans, the Al system can be configured to generate one or more conversational dialogue parts that include an antonym or synonym associated with the one or more words. For example, the Al system can be configured to generate one or more conversational dialogue parts that include the word “hard” based on the input word “soft.” At 769, based on the one or more lesson plans, the Al system can be configured to generate one or more conversational dialogue parts that include a context associated with the one or more words. For example, the Al system can be configured to generate one or more conversational dialogue parts that discuss whether the child had milk with his cereal based on the input word “milk.” The generation of conversational dialogue parts that incorporate words / concepts the user has used can further reinforce understanding of the words / concepts and provide greater engagement, thus improving learning outcomes.

[0107] FIG. 8 A illustrates a system 800 that is configured to monitor a user’s aural language competency development. The system 800 comprises an aural language competency monitoring module / application 820 that can be configured to provide, via a graphical user interface, information on the aural language competency development of an individual to a user. In certain embodiments, the individual being monitored and the user of the aural language competency monitoring module / application 820 can be the same person or different people. In one example, the aural language competency monitoring module / application 820 can be configured to provide information on the aural language competency development of a child named Nathan who is four years old to Isabella, who is a parent / guardian of Nathan. In another example, the aural language competency monitoring module / application 820 can be configured to provide an adult individual information on their own aural language competency development.

[0108] In certain embodiments, the aural language competency monitoring module / application 820 can be connected to a sleeve structure (e.g., character object) via a wired or wireless connection. The aural language competency monitoring module / application 820 includes an interactive element 822A (e.g., “Progress” tab) configured to provide information on the learning progression of the individual, an interactive element 822B (e.g., “Lessons” tab) configured to provide information on the lessons taken or planned for the user, and anAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1interactive element 822C (e.g., “History” tab) configured to provide information on the conversation history between the individual and a sleeve structure (e.g., character object). That is, for example, the interactive element 822A (e.g., “Progress” tab) can be configured to provide information on the individual’s progress on a specific lesson and overall progress, time spent on the lessons, and other relevant information (e.g., links to articles on learning strategies). In certain embodiments, one or more rewards or discounts can be provided to the individual being monitored based on the amount of time spent with the sleeve structure. For example, the individual can be provided with a reward of additional sleeve structures (e.g., toys) when the individual has spent a threshold amount of time with the current sleeve structure or when the individual reaches a certain goal (e.g., above 90% overall progress on the lessons).

[0109] The aural language competency monitoring module / application 820 further includes one or more controls 828, such as an “Awake” button, an “Asleep” button, and a “Mode” button, that can be configured to control the sleeve structure. For example, “Awake” button can be configured to activate the sleeve structure while the “Asleep” button can be configured to deactivate the sleeve structure. The “Mode” button can be configured to specify one or more modes at which the sleeve structure can be configured. For example, the user (e.g., a parent) can select the sleeve structure to be in a “Friend” mode or “Educator” mode to the individual (e.g., child) being monitored. Moreover, the aural language competency monitoring module / application 820 can be configured to enable the user to select a character voice (e.g., voice of a cartoon character, voice of a parent, etc.) for the sleeve structure. Further, in certain embodiments, the aural language competency monitoring module / application 820 can be configured to enable the user to tune background noises and volume for the sleeve structure.

[0110] FIG. 8B illustrates the aural language competency monitoring module / application 820 where the interactive element 822C (e.g., “History” tab) is configured to display a transcript of an exemplary conversation between the individual being monitored (e.g., a child named Nathan) and a sleeve structure named “Rex.” In certain embodiments, the conversation transcript can be provided to train a machine learning model for developing future lesson plans to develop the aural language competency of the individual. Further, the data (e.g., lesson progress, conversation history, etc.) collected by the aural language competency monitoring module / application 820 can be provided to a machine learning model configured to generate a data analysis report of the individual’s aural language competency. The data analysis report, highlighting areas that need further improvement, can be provided to a medical practitioner (e.g., audiologist) for further assessment.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1[oom] FIG. 9 illustrates a system 900 that is configured to manage one or more hearing devices. The system 900 comprises an external component 904 of a hearing device (e.g., cochlear implant) and an external device 910. The external device 910 can wirelessly communicate with the external component 904 via a bi-directional communication link 926. The bi-directional communication link 926 can comprise, for example, a short-range communication, such as Bluetooth link, Bluetooth Low Energy (BLE) link, a proprietary link, etc. In certain embodiments, the external component 904 is configured to be directly or indirectly attached to the body of a user and managed and / or programmed by the external device 910. For example, the external device 910 can include a mobile device such as a mobile phone, tablet, laptop, etc.

[0112] The external device 910 can be configured to execute a device management application 947. In certain embodiments, the device management application 947 can be a mobile application, a computer application, or any suitable software application. The device management application 947 includes one or more interactive elements, such as an interactive element 949, that can be configured to enable a user to perform one or more device management operations. For example, the interactive element 949 can enable the user to access an aural language competency monitoring module / application (e.g., aural language competency monitoring module / application 820). In certain embodiments, the user can, via interactive element 949, access the aural language competency monitoring module / application to review the user’s learning progress, determine whether additional development (e.g., habitualization or rehabilitation) is needed, and / or activate the sleeve structure to initiate one or more developmental exercises to reach an aural language competency development goal. Further, the one or more device management operations can include controlling the volume, changing the program, or adjusting the audio source settings.

[0113] FIG. 10 is a flowchart illustrating a method 1000, in accordance with certain embodiments presented herein. Method 1000 beings at 1001 where one or more conversational dialogue parts for interaction with a hearing device user is generated via at least one acoustic transducer. At 1003, one or more responses from the hearing device user to the one or more conversational dialogue parts are obtained via at least one sound input unit. At 1005, one or more subsequent conversational dialogue parts for interaction with the hearing device user based on the one or more responses from the hearing device user are generated.

[0114] FIG. 11 is a flowchart illustrating a method 1100, in accordance with certain embodiments presented herein. Method 1100 beings at 1101 where one or more interactiveAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1exercises are delivered to a hearing device user via at least one acoustic transducer. At 1103, one or more responses of the hearing device user to the one or more interactive exercises are obtained. At 1105, the one or more responses of the hearing device user are used to assess at least one of a preference of the hearing device user or an aural language competency development of the hearing device user. At 1107, the one or more interactive exercises delivered to the hearing device user are adapted over time based on the assessment of at least one of the preference of the hearing device user or the aural language competency development of the hearing device user.

[0115] FIG. 12 is a flowchart illustrating a method 1200, in accordance with certain embodiments presented herein. Method 1200 beings at 1201 where a personality map associated with a character object is obtained, wherein the personality map identifies one or more characteristics forming a personality of the character object. At 1203, based on the personality map, train an intelligent agent to generate one or more interactive exercises to elicit a behavioral response from a user. At 1205, the personality map is updated based on the behavioral response.

[0116] FIG. 13 is a flowchart illustrating a method 1300, in accordance with certain embodiments presented herein. Method 1300 beings at 1301 where at least one response is obtained from a user to at least one stimulus provided via an acoustic transducer. A 1303, whether the at least one response matches an expected response is determined. At 1305, upon determining that the at least one response matches the expected response, a personality map of a character object is updated to generate an updated personality map. At 1307, an intelligent agent to generate a subsequent stimulus is trained based on the updated personality map.

[0117] As previously described, the technology disclosed herein can be applied in any of a variety of circumstances and with a variety of different devices. Example devices that can benefit from technology disclosed herein are described in more detail in FIGs. 14-17 for a vestibular stimulator system, a retinal prosthesis system, a tinnitus therapy device, and an upper airway stimulation device, respectively. However, the techniques of the present disclosure can be applied to other devices, such as neurostimulators, cardiac pacemakers, cardiac defibrillators, seizure therapy stimulators, as well as other medical devices that deliver stimulation to tissue. Further, technology described herein can also be applied to consumer devices. These different systems and devices can benefit from the technology described herein.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0118] FIG. 14 illustrates an example vestibular stimulator system 1402, with which embodiments presented herein can be implemented. As shown, the vestibular stimulator system 1402 comprises an implantable component (vestibular stimulator) 1412 and an external device / component 1404 (e.g., external processing device, battery charger, remote control, etc.). The external device 1404 comprises a transceiver unit 1460. As such, the external device 1404 is configured to transfer data (and potentially power) to the vestibular stimulator 1412.

[0119] The vestibular stimulator 1412 comprises an implant body (main module) 1434, a lead region 1436, and a stimulating assembly 1416, all configured to be implanted under the skin / tissue (tissue) 1415 of the user. The implant body 1434 generally comprises a hermetically-sealed housing 1438 in which RF interface circuitry, one or more rechargeable batteries, one or more processors, and a stimulator unit are disposed. The implant body 1434 also includes an intemal / implantable coil 1414 that is generally external to the housing 1438, but which is connected to the transceiver via a hermetic feedthrough (not shown).

[0120] The stimulating assembly 1416 comprises a plurality of electrodes 1444(l)-(3) disposed in a carrier member (e.g., a flexible silicone body). In this specific example, the stimulating assembly 1416 comprises three (3) stimulation electrodes, referred to as stimulation electrodes 1444(1), 1444(2), and 1444(3). The stimulation electrodes 1444(1), 1444(2), and 1444(3) function as an electrical interface for delivery of electrical stimulation signals to the user’s vestibular system.

[0121] The stimulating assembly 1416 is configured such that a surgeon can implant the stimulating assembly adjacent the user’s otolith organs via, for example, the user’s oval window. It is to be appreciated that this specific embodiment with three stimulation electrodes 1444 is merely illustrative and that the techniques presented herein can be used with stimulating assemblies having different numbers of stimulation electrodes, stimulating assemblies having different lengths, etc.

[0122] In certain examples, the vestibular stimulator 1412 can be implemented with the systems, methods, etc. described above, for example, with reference to FIGs. 2A- 13. In such examples, the system could be configured to develop physiological competency or development of user in relation to the vestibular stimulator 1412. That is, in such embodiments, the systems, methods, etc., could be modified in a manner that facilitates the user’s acceptance, use, etc. of the vestibular stimulator 1412.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0123] FIG. 15 illustrates a retinal prosthesis system 1501 that comprises an external device 1510 configured to communicate with an implantable retinal prosthesis 1500 via signals 1551. The retinal prosthesis 1500 comprises an implanted processing module 1525, and a retinal prosthesis sensor-stimulator 1590 is positioned proximate the retina of a user. The external device 1510 and the processing module 1525 can communicate via coils 1508, 1514.

[0124] In an example, sensory inputs (e.g., photons entering the eye) are absorbed by a microelectronic array of the sensor-stimulator 1590 that is hybridized to a glass piece 1592 including, for example, an embedded array of microwires. The glass can have a curved surface that conforms to the inner radius of the retina. The sensor-stimulator 1590 can include a microelectronic imaging device that can be made of thin silicon containing integrated circuitry that convert the incident photons to an electronic charge.

[0125] The processing module 1525 includes an image processor 1523 that is in signal communication with the sensor-stimulator 1590 via, for example, a lead 1588 that extends through surgical incision 1589 formed in the eye wall. In other examples, processing module 1525 is in wireless communication with the sensor-stimulator 1590. The image processor 1523 processes the input into the sensor-stimulator 1590 and provides control signals back to the sensor-stimulator 1590 so the device can provide an output to the optic nerve. That said, in an alternate example, the processing is executed by a component proximate to, or integrated with, the sensor-stimulator 1590. The electric charge resulting from the conversion of the incident photons is converted to a proportional amount of electronic current which is input to a nearby retinal cell layer. The cells fire and a signal is sent to the optic nerve, thus inducing a sight perception.

[0126] The processing module 1525 can be implanted in the user and function by communicating with the external device 1510, such as a BTE unit, a pair of eyeglasses, etc. The external device 1510 can include an external light / image capture device (e.g., located in / on a behind-the-ear device or a pair of glasses, etc.), while, as noted above, in some examples, the sensor-stimulator 1590 captures light / images, in which sensor-stimulator 1590 is implanted in the user.

[0127] In certain examples, the implantable retinal prosthesis 1500 can be implemented with the systems, methods, etc. described above, for example, with reference to FIGs. 2 A- 13. In such examples, the system could be configured to develop physiological competency or development of user in relation to the implantable retinal prosthesis 1500. That is, in suchAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1embodiments, the systems, methods, etc., could be modified in a manner that facilitates the user’s acceptance, use, etc. of the implantable retinal prosthesis 1500.

[0128] FIG. 16 illustrates a tinnitus therapy device 1600 (e.g., a tinnitus implant, a tinnitus management stimulator) including a sound input unit 1602 (e.g., a microphone) configured to receive acoustic inputs. In some embodiments, the sound input unit 1602 is implanted adjacent to an outer ear 1603 to position a diaphragm 1616 of the sound input unit 1602 such that the diaphragm 1616 is configured to be displaced (vibrate) in response to the acoustic inputs. The tinnitus therapy device 1600 further includes an implant body 1604 in which circuitry, such as a processor and / or a memory, is disposed. The implant body 1604 is also coupled to a coil 1608 to enable transfer of power / data between the tinnitus therapy device 1600 and an external device. The implant body 1604 is electrically coupled to the sound input unit 1602 to receive the acoustic input. The tinnitus therapy device 1600 is then configured to convert the acoustic input to tinnitus therapy control signals (e.g., based on a classification of the acoustic input).

[0129] The tinnitus therapy control signals are provided to an actuator 1606 electrically coupled to the implant body 1604 for delivery to the user. By way of example, a coupling member 1640 couples the actuator 1606 to an ossicular chain 1636 (i.e., the malleus, the incus, and the stapes bones) positioned in a middle ear cavity between a tympanic membrane 1613 and a cochlea 1638 of the user, and the actuator 1606 is configured to deliver the tinnitus therapy control signals. The actuator 1606 is attached to a temporal bone 1615 of the user via a fixation system 1642 and is configured to impart motion to (e.g., vibrate) the ossicular chain 1636, which is typically configured to amplify sound waves received via an ear canal 1611. In operation, the actuator 1606 is configured to impart motion based on the tinnitus therapy control signals, and such vibration creates waves of fluid motion of perilymph within the cochlea 1638 to activate hair cells within the cochlea 1638. Activation of the hair cells causes nerve impulses to be generated and transferred through spiral ganglion cells, an auditory nerve, and a brain, where the vibration is perceived as sounds to provide relief of tinnitus symptoms experienced by the user.

[0130] In certain examples, the tinnitus therapy device 1600 can be implemented with the systems, methods, etc. described above, for example, with reference to FIGs. 2A- 13. In such examples, the system could be configured to develop physiological competency or development of user in relation to the tinnitus therapy device 1600. That is, in such embodiments, the systems, methods, etc., could be modified in a manner that facilitates the user’s acceptance, use, etc. of the tinnitus therapy device 1600.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0131] FIG. 17 illustrates an upper airway stimulation device 1700 (e.g., an upper airway implant, a sleep apnea management stimulator, a sleep disorder system) that includes an implant body 1702, a sensor 1704, and a stimulator 1706. The upper airway stimulation device 1700 is implantable in a user 1708 to position the sensor 1704 adjacent to lungs 1710 of the user 1708. Thus, the sensor 1704 is able to receive input that indicates breathing performed by the user 1708. The implant body 1702 includes a housing in which circuitry, such as a processor and / or a memory, is disposed. The sensor 1704 transmits electrical signals in response to receipt of the input, and the upper airway stimulation device 1700 is configured to convert the electrical signals to stimulation signals, which are provided to the stimulator 1706. The stimulator 1706 is positioned adjacent to a hypoglossal nerve 1712 of the user 1708 and is configured to deliver the stimulation signals to the hypoglossal nerve 1712, which fires nerve cells of a tongue of the user 1708, thereby causing the tongue to contract and move (e.g., in an anterior direction) and increase a size of an opening of an airway of the user 1708. Consequently, the upper airway stimulation device 1700 generates stimulation signals based on the input to help the user 1708 breathe more easily (e.g., while the user 1708 is asleep to mitigate sleep apnea).

[0132] In certain examples, the upper airway stimulation device 1700 can be implemented with the systems, methods, etc. described above, for example, with reference to FIGs. 2 A- 13. In such examples, the system could be configured to develop physiological competency or development of user in relation to the upper airway stimulation device 1700. That is, in such embodiments, the systems, methods, etc., could be modified in a manner that facilitates the user’s acceptance, use, etc. of the upper airway stimulation device 1700.

[0133] The above embodiments have been primarily described with reference to use of the techniques presented herein during implantation of a specific implantable component, namely a cochlear implant stimulating assembly. However, as described above with reference to FIGs.14, 15, 16, and 17, the techniques presented herein can be used to provide flexibility to improve implantation of a variety of implantable components, including sleep disorder devices (e.g., sleep apnea devices), balance or movement disorder devices (e.g., vestibular stimulation devices), tinnitus management devices, visual implants (e.g., bionic eyes), and other neuromodulation devices (e.g., brain-computer interfaces).

[0134] As should be appreciated, while particular uses of the technology have been illustrated and discussed above, the disclosed technology can be used with a variety of devices in accordance with many examples of the technology. The above discussion is not meant toAtty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1suggest that the disclosed technology is only suitable for implementation within systems akin to that illustrated in the figures. In general, additional configurations can be used to practice the processes and systems herein and / or some aspects described can be excluded without departing from the processes and systems disclosed herein.

[0135] This disclosure described some aspects of the present technology with reference to the accompanying drawings, in which only some of the possible aspects were shown. Other aspects can, however, be embodied in many different forms and should not be construed as limited to the aspects set forth herein. Rather, these aspects were provided so that this disclosure was thorough and complete and fully conveyed the scope of the possible aspects to those skilled in the art.

[0136] As should be appreciated, the various aspects (e.g., portions, components, etc.) described with respect to the figures herein are not intended to limit the systems and processes to the particular aspects described. Accordingly, additional configurations can be used to practice the methods and systems herein and / or some aspects described can be excluded without departing from the methods and systems disclosed herein.

[0137] According to certain aspects, systems and non-transitory computer readable storage media are provided. The systems are configured with hardware configured to execute operations analogous to the methods of the present disclosure. The one or more non-transitory computer readable storage media comprise instructions that, when executed by one or more processors, cause the one or more processors to execute operations analogous to the methods of the present disclosure.

[0138] Similarly, where steps of a process are disclosed, those steps are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps. For example, the steps can be performed in differing order, two or more steps can be performed concurrently, additional steps can be performed, and disclosed steps can be excluded without departing from the present disclosure. Further, the disclosed processes can be repeated.

[0139] Although specific aspects were described herein, the scope of the technology is not limited to those specific aspects. One skilled in the art will recognize other aspects or improvements that are within the scope of the present technology. Therefore, the specific structure, acts, or media are disclosed only as illustrative aspects. The scope of the technology is defined by the following claims and any equivalents therein.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1

[0140] It is also to be appreciated that the embodiments presented herein are not mutually exclusive and that the various embodiments can be combined with another in any of a number of different manners.

Claims

Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1CLAIMSWhat is claimed is:

1. A system, comprising:at least one acoustic transducer;at least one sound input unit;an aural language competency development device, comprising:a memory; andat least one processor operably coupled to the memory, the at least one acoustic transducer, and the at least one sound input unit, wherein the at least one processor is configured to execute an intelligent agent to:generate one or more conversational dialogue parts for interaction with a user device user via the at least one acoustic transducer;obtain one or more responses from the user device user to the one or more conversational dialogue parts via the at least one sound input unit; and generate one or more subsequent conversational dialogue parts for interaction with the user device user based on the one or more responses from the user device user.

2. The system of claim 1, further comprising:a sleeve structure,wherein the sleeve structure is configured to have the aural language competency development device disposed therein.

3. The system of claim 2, wherein the sleeve structure includes the at least one acoustic transducer and the at least one sound input unit.

4. The system of claim 2, wherein the sleeve structure is a character object.

5. The system of claim 1, 2, 3, and 4, further comprising at least one secondary transducer in spatial separation from the at least one sound input unit, and wherein the at least one processor is configured to use the at least one secondary transducer to deliver a stimulus that recruits hearing faculties of the user device user to elicit an expected response.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC16. The system of claim 5, wherein the stimulus is an acoustic prompt that, when perceived correctly by the user device user, prompts the user device user to behave in a predetermined manner, and wherein the at least one processor is configured to execute the intelligent agent to:capture a behavioural response of the user device user to the acoustic prompt; and determine whether the behavioural response corresponds with an expected behavioural response.

7. The system of claim 6, wherein the expected behavioural response is an expected verbal response.

8. The system of claim 6, wherein the expected behavioural response is an expected gesture.

9. The system of claim 6, wherein the expected behavioural response is an expected movement.

10. The system of claim 5, wherein the at least one secondary transducer comprises at least two secondary transducers each in spatial separation from the at least one sound input unit.

11. A system, comprising:at least one acoustic transducer;at least one sound input unit;an aural language competency development device, comprising:a memory storing an intelligent agent; andat least one processor operably coupled to the memory, the at least one acoustic transducer, and the at least one sound input unit, wherein the at least one processor is configured to execute the intelligent agent to:deliver one or more interactive exercises to a user device user via the at least one acoustic transducer;obtain one or more responses of the user device user to the one or more interactive exercises;Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC1use the one or more responses of the user device user to assess at least one of a preference of the user device user or an aural language competency development of the user device user; andadapt the one or more interactive exercises delivered to the user device user over time based on the assessment of at least one of the preference of the user device user or the aural language competency development of the user device user.

12. The system of claim 11, wherein the system includes a database of pre-programmed lessons pre-curated by a clinician for enabling the intelligent agent to create the one or more interactive exercises.

13. The system of claim 11, wherein the one or more interactive exercises are generated or selected by the intelligent agent based on an engagement level of the user device user.

14. The system of claim 13, wherein the engagement level is determined based on one or more of: response time associated with the one or more responses, response length associated with the one or more responses, response accuracy associated with the one or more responses, or response repeatability associated with the one or more responses.

15. The system of claim 11, 12, 13, or 14, further comprising:a sleeve structure,wherein the sleeve structure is configured to have the aural language competency development device disposed therein.

16. The system of claim 15, wherein the sleeve structure is a character object.

17. The system of claim 16, wherein the one or more interactive exercises are delivered in a voice associated with the character object.

18. The system of claim 17, wherein the voice associated with the character object is determined based on a personality map associated with the character object or an engagement goal associated with the one or more interactive exercises.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC119. The system of claim 11, 12, 13, or 14, wherein the one or more interactive exercises are associated with a predetermined theme provided by the user device user.

20. The system of claim 11, 12, 13, or 14, wherein the one or more interactive exercises incorporate one or more words previously spoken by the user device user.

21. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to:obtain a personality map associated with a character object, wherein the personality map identifies one or more characteristics forming a personality of the character object; train, based on the personality map, an intelligent agent to generate one or more interactive exercises to elicit a behavioural response from a user; andupdate the personality map based on the behavioural response.

22. The one or more non-transitory computer readable storage media of claim 21, wherein the one or more characteristics include one or more of a tone of voice, a dialect, an accent, or a type of vocabulary.

23. The one or more non-transitory computer readable storage media of claim 21, wherein the behavioural response includes one or more of a hand gesture, a verbal response, or an eye movement.

24. The one or more non-transitory computer readable storage media of claim 21, 22, or 23, wherein the one or more interactive exercises are delivered to the user via at least one transducer.

25. The one or more non-transitory computer readable storage media of claim 21, 22, or 23, wherein the one or more interactive exercises are delivered to the user via at least two spatially-separated transducers.

26. The one or more non-transitory computer readable storage media of claim 21, 22, or 23, wherein the intelligent agent includes a machine learning model.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC127. The one or more non-transitory computer readable storage media of claim 26, wherein the machine learning model is trained to select the one or more interactive exercises from a plurality of pre-curated lesson plans.

28. The one or more non-transitory computer readable storage media of claim 21, 22, or 23, wherein the one or more interactive exercises adhere to one or more guidelines identified in one or more pre-curated lesson plans.

29. The one or more non-transitory computer readable storage media of claim 21, 22, or 23, wherein an engagement level of the user is determined based on the behavioral response.

30. The one or more non-transitory computer readable storage media of claim 29, wherein the personality map is updated based on the engagement level reaching a predetermined threshold.

31. A method, comprising:obtaining at least one response from a user to at least one stimulus provided via an acoustic transducer;determining whether the at least one response matches an expected response; upon determining that the at least one response matches the expected response, updating a personality map of a character object to generate an updated personality map; and training, based on the updated personality map, an intelligent agent to generate a subsequent stimulus.

32. The method of claim 31, wherein the acoustic transducer is spatially separated from a sensor configured to capture the at least one response.

33. The method of claim 31, wherein the acoustic transducer is disposed within an object located in a same environment as the user.

34. The method of claim 33, wherein the at least one stimulus is an acoustic prompt.

35. The method of claim 31, 32, 33, or 34, wherein the intelligent agent is included in an aural language competency development device disposed within a sleeve structure.Atty. Docket No. 3065.088 li Client Ref. No. CID04040WOPC136. The method of claim 35, wherein the sleeve structure is a first character object having a first personality.

37. The method of claim 36, further comprising:upon determining that the at least one response fails to match the expected response, transferring the aural language competency development device from the first character object to a second character object having a second personality.

38. The method of claim 37, wherein a first voice associated with the first personality is different from a second voice associated with the second personality.

39. The method of 31, 32, 33, or 34, wherein at least one response is captured via at least one sensor.

40. The method of claim 39, wherein the at least one sensor includes a microphone or a motion sensor.

41. The use of a device according to 31, 32, 33, or 34, in a cochlear implant, sleep disorder device, a seizure device, a balance or movement disorder device, a tinnitus management device, or a visual device.

42. A system according to 31, 32, 33, or 34, wherein the system is a cochlear implant system, a sleep disorder system, a seizure system, a balance or movement disorder system, a tinnitus management system, or a visual system.

Images