Linguistic context in hearing device systems

Abstract

Presented herein are techniques for determining a linguistic context from speech (e.g., a conversation) captured by a system and using the "linguistic context" or "second-order meaning" to initiate / generate an action at / by the system. More specifically, a hearing device (e.g., cochlear implant) or medical device is configured to capture / detect speech which has a primary / explicit, or first-order, meaning. An intelligence module that includes an artificial intelligence (AI) model, e.g., a large language model (LLM) or a large action model (LAM), is configured to obtain the speech and determine a linguistic context (second-order meaning) of the speech and determine the relevancy of the second order-meaning to the system and / or to the recipient (recipient-specific second-order meaning of the speech). Using the recipient- specific second-order meaning of the speech, one or more instructions can be generated and / or produced for implementation at the system.

Description

Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1LINGUISTIC CONTEXT IN HEARING DEVICE SYSTEMSBACKGROUNDField of the Invention[ooot] Aspects of the techniques presented herein are generally directed to systems and methods for determining the linguistic context associated with speech and using the linguistic context to perform an action with respect to a hearing device or medical device.Related Art

[0002] Medical devices have provided a wide range of therapeutic benefits to recipients over recent decades. Medical devices can include internal or implantable components / devices, external or wearable components / devices, or combinations thereof (e.g., a device having an external component communicating with an implantable component). Medical devices, such as traditional hearing aids, partially or fully-implantable hearing prostheses (e.g., bone conduction devices, mechanical stimulators, cochlear implants, etc.), pacemakers, defibrillators, functional electrical stimulation devices, and other medical devices have been successful in performing lifesaving and / or lifestyle enhancement functions and / or recipient monitoring for a number of years.

[0003] The types of medical devices and the ranges of functions performed thereby have increased over the years. For example, many medical devices, sometimes referred to as “implantable medical devices,” now often include one or more instruments, apparatus, sensors, processors, controllers or other functional mechanical or electrical components that are permanently or temporarily implanted in a recipient. These functional devices are typically used to diagnose, prevent, monitor, treat, or manage a disease / injury or symptom thereof, or to investigate, replace or modify the anatomy or a physiological process. Many of these functional devices utilize power and / or data received from external devices that are part of, or operate in conjunction with, implantable components.SUMMARY

[0004] In one aspect, a method is provided. The method comprises: receiving sound signals at a hearing device, wherein the sound signals include speech, the speech having a first-order meaning; determining a recipient-specific second-order meaning associated with the speech;Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 and generating at least a first instruction based on the recipient-specific second-order meaning, wherein the first instruction is not included in the speech.

[0005] In another aspect, a system is provided. The system comprises: a display screen; a memory; and at least one processor operable coupled to the display screen and the memory, wherein the at least one processor is configured to: receive sound signals at a hearing device, wherein the sound signals include speech, the speech having a first-order meaning; determine a recipient-specific second-order meaning associated with the speech; and generate at least a first instruction based on the recipient-specific second-order meaning, wherein the first instruction is not included in the speech.

[0006] In still other aspects, the techniques described herein relate to non-transitory computer readable storage media comprising instructions that, when executed by one or more processors, cause the one or more processors to: obtain sound signals at a medical device, wherein the sound signals include speech; process the speech to identify a contextual meaning associated with the speech, wherein the meaning is not expressly included in the speech; interpret the contextual meaning; and generate at least a first instruction based on the interpreting of the contextual meaning.

[0007] In another aspect, a system is provided. The system comprises: a display screen; a memory; and at least one processor operable coupled to the display screen and the memory, wherein the at least one processor is configured to: obtain sound signals at a medical device, wherein the sound signals include speech; process the speech to identify a contextual meaning associated with the speech, wherein the meaning is not expressly included in the speech; interpret the contextual meaning; and generate at least a first instruction based on the interpreting of the contextual meaning.

[0008] In another aspect, a method is provided. The method comprises: obtaining sound signals at a first device of a hearing device system, wherein the sound signals include speech; extracting at least one contextual cue from the sound signals; determining, based on the at least one contextual cue, a linguistic context associated with the speech; and adjusting operation of the first device based on the linguistic context.

[0009] In still other aspects, the techniques described herein relate to non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to: obtain sound signals at a first device, wherein the sound signals include speech; extract at least one contextual cue from the sound signals; determine, based on the at least oneAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 contextual cue, a context associated with the speech; and adjust operation of the first device based on the context.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments of the present invention are described herein in conjunction with the accompanying drawings, in which:

[0011] FIG. 1A is a schematic diagram illustrating a cochlear implant system with which aspects of the techniques presented herein can be implemented;

[0012] FIG. IB is a side view of a recipient wearing a sound processing unit of the cochlear implant system of FIG. 1A;

[0013] FIG. 1C is a schematic view of components of the cochlear implant system of FIG. 1 A;

[0014] FIG. ID is a block diagram of the cochlear implant system of FIG. 1A;

[0015] FIG. IE is a schematic diagram illustrating a computing device with which aspects of the techniques presented herein can be implemented;

[0016] FIG. 2A is a diagrammatic representation of an intelligence module and an instruction implementation model, e.g.., intelligence module 131 and instruction implementation module 135 of FIG. ID, in which the intelligence module includes an artificial intelligence model that accesses an external dataset in accordance with an embodiment;

[0017] FIG. 2B is a diagrammatic representation of an intelligence module and an instruction implementation model, e.g.., intelligence module 131 and instruction implementation module 135 of FIG. ID, in which the intelligence module includes an artificial intelligence model that accesses an internal dataset in accordance with an embodiment;

[0018] FIG. 3A is a diagrammatic representation of intelligence logic and instruction implementation logic, e.g.., intelligence logic 195 and instruction implementation logic 196 of FIG. IE, in which the intelligence logic includes an artificial intelligence model that accesses an external dataset in accordance with an embodiment;

[0019] FIG. 3B is a diagrammatic representation of an intelligence logic and an instruction implementation logic, e.g.., intelligence logic 195 and instruction implementation logic 196 of FIG. IE, in which the intelligence logic includes an artificial intelligence model that accesses an internal dataset in accordance with an embodiment;Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0020] FIG. 4A is a diagrammatic representation of a server arrangement that is arranged to cooperate with an intelligence module and an instruction implementation model, e.g.., intelligence module 131 and instruction implementation module 135 of FIG. ID, in accordance with an embodiment;

[0021] FIG. 4B is a diagrammatic representation of a server arrangement, e.g., server arrangement 493 of FIG. 4A, that is arranged to cooperate with intelligence logic and instruction implementation logic, e.g.., intelligence logic 195 and instruction implementation logic 196 of FIG. IE, in accordance with an embodiment;

[0022] FIG. 5 is a process flow diagram which illustrates an overall method of causing an action to be taken based on contextual audio information, e.g., speech from a conversation, in accordance with an embodiment;

[0023] FIGs. 6A and 6B are a process flow diagram which illustrates an exemplary method of causing an action to be taken based on speech detected during a conversation in accordance with an embodiment;

[0024] FIG. 7 is a diagrammatic representation of a process of generating an output based on speech content that includes identifying a second-order meaning and an action behind the second-order meaning associated with the speech content in accordance with an embodiment;

[0025] FIG. 8 is a diagrammatic representation of a process of analyzing speech content provided to an artificial intelligence system that includes determining a second-order meaning and an action behind the second-order meaning in accordance with an embodiment;

[0026] FIG. 9 is a diagrammatic representation of an example of generating an output from speech content in accordance with an embodiment;

[0027] FIG. 10 is a schematic diagram illustrating a vestibular stimulator system with which aspects of the techniques presented herein can be implemented; and

[0028] FIG. 11 is a schematic diagram illustrating a retinal prosthesis system with which aspects of the techniques presented herein can be implemented.DETAILED DESCRIPTION

[0029] Presented herein are techniques for determining a linguistic context from speech (e.g., a conversation, from the recipient themselves, etc.) captured by a system (e.g., hearing device system or medical device system) and using the “linguistic context” or “second-order meaning” to initiate / generate an action at / by the system. As used herein, “linguistic context” refers to aAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 second-order meaning associated with the speech (e.g., a meaning behind the language relating to things that aren’t explicitly expressed). More specifically, a hearing device (e.g., hearing aid, cochlear implant, etc.) or medical device is configured to capture / detect speech which has a primary / explicit, or first-order, meaning. An intelligence module that includes an artificial intelligence (Al) model, e.g., a large language model (LLM) or a large action model (LAM), is configured to obtain the speech and determine a linguistic context (second-order meaning) associated with the speech and determine the relevancy of the second order-meaning to the system and / or to the recipient. The second-order meaning, and the relevancy of the second order-meaning to the system and / or to the recipient, are sometimes collectively referred to herein as the “recipient-specific second-order meaning” associated with the speech. Using the recipient-specific second-order meaning associated with the speech, one or more instructions can be generated and / or produced for implementation at the system (e.g., at the hearing device, medical device, and / or other associated device).

[0030] The one or more instructions generated based on the recipient-specific second-order meaning associated with the speech can take any of a number of different forms. For example, in certain embodiments, the one or more instructions can address an issue or a problem identified by the linguistic context. The instructions can include, but are not limited to, an instruction for an action to be taken by a user of the hearing device system, an instruction for an action taken to substantially automatically adjust a configuration of the hearing device system, an instruction for an action to be taken to substantially automatically adjust a configuration of an external system, etc.

[0031] There are a number of different types of devices in / with which embodiments of the present invention can be implemented. Merely for ease of description, the techniques presented herein are primarily described with reference to a specific hearing device system in the form of a cochlear implant system. However, it is to be appreciated that the techniques presented herein can also be partially or fully implemented by any of a number of different types of systems or devices, including consumer electronic device (e.g., mobile phones), wearable devices (e.g., smartwatches), hearing devices, implantable medical devices, consumer electronic devices, etc. As used herein, the term “hearing device” is to be broadly construed as any device that acts on an acoustical perception of an individual, including to improve perception of sound signals, to reduce perception of sound signals, etc. In particular, a hearing device can deliver sound signals to a user in any form, including in the form of acoustical stimulation, mechanical stimulation, electrical stimulation, etc., and / or can operateAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 to suppress all or some sound signals. As such, a hearing device can be a device for use by a hearing-impaired person (e.g., hearing aids, middle ear auditory prostheses, bone conduction devices, direct acoustic stimulators, electro-acoustic hearing prostheses, auditory brainstem stimulators, bimodal hearing prostheses, bilateral hearing prostheses, dedicated tinnitus therapy devices, tinnitus therapy device systems, combinations or variations thereof, etc.), a device for use by a person with normal hearing (e.g., consumer devices that provide audio streaming, consumer headphones, earphones, and other listening devices), a hearing protection device, etc. In other examples, the techniques presented herein can be implemented by, or used in conjunction with, various implantable medical devices, such as visual devices (i.e., bionic eyes), sensors, pacemakers, drug delivery systems, defibrillators, functional electrical stimulation devices, catheters, seizure devices (e.g., devices for monitoring and / or treating epileptic events), sleep apnea devices, electroporation devices, etc.

[0032] FIGs. 1A-1D illustrate an example cochlear implant system 102 with which aspects of the techniques presented herein can be implemented. The cochlear implant system 102 comprises an external component 104 that is configured to be directly or indirectly attached to the body of the user, and an intemal / implantable component 112 that is configured to be implanted in or worn on the head of the user. In the examples of FIGs. 1A-1D, the implantable component 112 is sometimes referred to as a “cochlear implant.” FIG. 1A illustrates the cochlear implant 112 implanted in the head 154 of a user, while FIG. IB is a schematic drawing of the external component 104 worn on the head 154 of the user. FIG. 1C is another schematic view of the cochlear implant system 102, while FIG. ID illustrates further details of the cochlear implant system 102. For ease of description, FIGs. 1A-1D will generally be described together.

[0033] In the examples of FIGs. 1A-1D, the external component 104 comprises a sound processing unit 106, an external coil 108, and generally, a magnet fixed relative to the external coil 108. The cochlear implant 112 includes an implantable coil 114, an implant body 134, and an elongate stimulating assembly 116 configured to be implanted in the user’s cochlea. In one example, the sound processing unit 106 is an off-the-ear (OTE) sound processing unit, sometimes referred to herein as an OTE component, that is configured to send data and power to the implantable component 112. In general, an OTE sound processing unit is a component having a generally cylindrically shaped housing 111 and which is configured to be magnetically coupled to the user’s head 154 (e.g., includes an integrated external magnet 150 configured to be magnetically coupled to an intemal / implantable magnet 152 in the implantable componentAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1112). The OTE sound processing unit 106 also includes an integrated external (headpiece) coil 108 (the external coil 108) that is configured to be inductively coupled to the implantable coil 114.

[0034] It is to be appreciated that the OTE sound processing unit 106 is merely illustrative of the external devices that could operate with implantable component 112. For example, in alternative examples, the external component 104 can comprise a behind-the-ear (BTE) sound processing unit configured to be attached to, and worn adjacent to, the recipient’s ear. A BTE sound processing unit comprises a housing that is shaped to be worn on the outer ear of the user. In certain examples, the BTE is connected to a separate external coil assembly via a cable, where the external coil assembly is configured to be magnetically and inductively coupled to the implantable coil 114, while in other embodiments the BTE includes a coil disposed in or on the housing worn on the outer ear of the user. It is also to be appreciated that alternative external components could be located in the user’s ear canal, worn on the body, etc.

[0035] Although the cochlear implant system 102 includes the sound processing unit 106 and the cochlear implant 112, as described below, the cochlear implant 112 can operate independently from the sound processing unit 106, for at least a period, to stimulate the user. For example, the cochlear implant 112 can operate in a first general mode, sometimes referred to as an “external hearing mode,” in which the sound processing unit 106 captures sound signals which are then used as the basis for delivering stimulation signals to the user. The cochlear implant 112 can also operate in a second general mode, sometimes referred as an “invisible hearing” mode, in which the sound processing unit 106 is unable to provide sound signals to the cochlear implant 112 (e.g., the sound processing unit 106 is not present, the sound processing unit 106 is powered-off, the sound processing unit 106 is malfunctioning, etc.). As such, in the invisible hearing mode, the cochlear implant 112 captures sound signals itself via implantable sound sensors and then uses those sound signals as the basis for delivering stimulation signals to the user. Further details regarding operation of the cochlear implant 112 in the external hearing mode are provided below, followed by details regarding operation of the cochlear implant Oin the invisible hearing mode. It is to be appreciated that reference to the external hearing mode and the invisible hearing mode is merely illustrative and that the cochlear implant 112 could also operate in alternative modes.

[0036] In FIGs. 1A and 1C, the cochlear implant system 102 is shown with an external device 110, configured to implement aspects of the techniques presented. The external device 110, which is shown in greater detail in FIG. IE, is a computing device, such as a personal computerAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1(e.g., laptop, desktop, tablet), a mobile phone (e.g., smartphone), a remote control unit, etc. The external device 110 and the cochlear implant system 102 (e.g., sound processing unit 106 or the cochlear implant 112) wirelessly communicate via a bi-directional communication link 126. The bi-directional communication link 126 can comprise, for example, a short-range communication, such as Bluetooth link, Bluetooth Low Energy (BLE) link, a proprietary link, etc.

[0037] Returning to the example ofFIGs. 1A-1D, the sound processing unit 106 of the external component 104 also comprises one or more input devices configured to capture and / or receive input signals (e.g., sound or data signals) at the sound processing unit 106. The one or more input devices include, for example, one or more sound input devices 118 (e.g., one or more external microphones, audio input ports, telecoils, etc.), one or more auxiliary input devices 128 (e.g., audio ports, such as a Direct Audio Input (DAI), data ports, such as a Universal Serial Bus (USB) port, cable port, etc.), and a short-range wireless transmitter / receiver (wireless transceiver) 120 (e.g., for communication with the external device 110), each located in, on or near the sound processing unit 106. However, it is to be appreciated that one or more input devices can include additional types of input devices and / or less input devices (e.g., the short- range wireless transceiver 120 and / or one or more auxiliary input devices 128 could be omitted).

[0038] The sound processing unit 106 also comprises the external coil 108, a charging coil 130, a closely-coupled radio frequency transmitter / receiver (RF transceiver) 122, at least one rechargeable battery 132, and an external sound processing module 124. The external sound processing module 124 can be configured to perform a number of operations that are represented in FIG. ID by an intelligence module 131, a sound processor 133, and an instruction implementation module 135. Each of the intelligence module 131, the sound processor 133, and the instruction implementation module 135 can be formed by one or more processors (e.g., one or more Digital Signal Processors (DSPs), one or more uC cores, etc.), firmware, software, etc. arranged to perform operations described herein. That is, the intelligence module 131, the sound processor 133, and the instruction implementation module 135 can each be implemented as firmware elements, partially or fully implemented with digital logic gates in one or more application-specific integrated circuits (ASICs), partially or fully in software, etc. Although FIG. ID illustrates the intelligence module 131, the sound processor 133, and the instruction implementation module 135 as being implemented / performed at the external sound processing module 124, it is to be appreciated that these elements (e.g.,Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 functional operations) could also or alternatively be implemented / performed as part of the implantable sound processing module 158, as part of the external device 110, etc. As such, the arrangement shown in FIG. ID is merely illustrative.

[0039] Returning to the example of FIGs. 1A-1D, the implantable component 112 comprises an implant body (main module) 134, a lead region 136, and the stimulating assembly 116, all configured to be implanted under the skin (tissue) 115 of the user. The implant body 134 generally comprises a hermetically-sealed housing 138 that includes, in certain examples, at least one power source 125 (e.g., one or more batteries, one or more capacitors, etc.), in which the RF interface circuitry 140 and a stimulator unit 142 are disposed. The implant body 134 also includes the intemal / implantable coil 114 that is generally external to the housing 138, but which is connected to the RF interface circuitry 140 via a hermetic feedthrough (not shown in FIG. ID).

[0040] As noted, the stimulating assembly 116 is configured to be at least partially implanted in the user’s cochlea. The stimulating assembly 116 includes a plurality of longitudinally spaced intra-cochlear electrical stimulating contacts (electrodes) 144 that collectively form a contact array (electrode array) 146 for delivery of electrical stimulation (current) to the recipient’s cochlea. The stimulating assembly 116 extends through an opening in the recipient’s cochlea (e.g., cochleostomy, the round window, etc.) and has a proximal end connected to stimulator unit 142 via lead region 136 and a hermetic feedthrough (not shown in FIG. ID). Lead region 136 includes a plurality of conductors (wires) that electrically couple the electrodes 144 to the stimulator unit 142. The implantable component 112 also includes an electrode outside of the cochlea, sometimes referred to as the extra-cochlear electrode (ECE) 139.

[0041] As noted, the cochlear implant system 102 includes the external coil 108 and the implantable coil 114. The external magnet 150 is fixed relative to the external coil 108 and the intemal / implantable magnet 152 is fixed relative to the implantable coil 114. The external magnet 150 and the intemal / implantable magnet 152 fixed relative to the external coil 108 and the intemal / implantable coil 114, respectively, facilitate the operational alignment of the external coil 108 with the implantable coil 114. This operational alignment of the coils enables the external component 104 to transmit data and power to the implantable component 112 via a closely-coupled wireless link 148 formed between the external coil 108 with the implantable coil 114. In certain examples, the closely-coupled wireless link 148 is an RF link. However, various other types of energy transfer, such as infrared (IR), electromagnetic, capacitive andAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 inductive transfer, can be used to transfer the power and / or data from an external component to an implantable component and, as such, FIG. ID illustrates only one example arrangement.

[0042] As noted above, the sound processing unit 106 includes the external sound processing module 124. The external sound processing module 124 is configured to process the received input audio signals (received at one or more of the input devices, such as sound input devices 118 and / or auxiliary input devices 128) and convert the received input audio signals into output control signals for use in stimulating a first ear of a recipient or user (i.e., the external sound processing module 124 is configured to perform sound processing on input signals received at the sound processing unit 106). Stated differently, the one or more processors (e.g., processing element(s) implementing firmware, software, etc.) in the external sound processing module 124 are configured to execute sound processing logic in memory to convert the received input audio signals into output control signals (stimulation signals) that represent electrical stimulation for delivery to the recipient.

[0043] As noted, FIG. ID illustrates an embodiment in which the external sound processing module 124 in the sound processing unit 106 generates the output control signals. In an alternative embodiment, the sound processing unit 106 can send less processed information (e.g., audio data) to the implantable component 112, and the sound processing operations (e.g., conversion of input sounds to output control signals 156) can be performed by a processor within the implantable component 112.

[0044] In FIG. ID, according to an example embodiment, output control signals (stimulation signals) are provided to the RF transceiver 122, which transcutaneously transfers the output control signals (e.g., in an encoded manner) to the implantable component 112 via the external coil 108 and the implantable coil 114. That is, the output control signals (stimulation signals) are received at the RF interface circuitry 140 via the implantable coil 114 and provided to the stimulator unit 142. The stimulator unit 142 is configured to utilize the output control signals to generate electrical stimulation signals (e.g., current signals) for delivery to the user’s cochlea via one or more of the stimulating contacts 144. In this way, cochlear implant system 102 electrically stimulates the user’s auditory nerve cells, bypassing absent or defective hair cells that normally transduce acoustic vibrations into neural activity, in a manner that causes the recipient to perceive one or more components of the input audio signals (the received sound signals).Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0045] As detailed above, in the external hearing mode, the cochlear implant 112 receives processed sound signals from the sound processing unit 106. However, in the invisible hearing mode, the cochlear implant 112 is configured to capture and process sound signals for use in electrically stimulating the user’s auditory nerve cells. In particular, as shown in FIG. ID, an example embodiment of the cochlear implant 112 can include a plurality of implantable sound sensors 165(1), 165(2) that collectively form a sensor array 160, and an implantable sound processing module 158. Similar to the external sound processing module 124, the implantable sound processing module 158 can comprise, for example, one or more processors and a memory device (memory) that includes sound processing logic. The memory device can comprise any one or more of: Non-Volatile Memory (NVM), Ferroelectric Random Access Memory (FRAM), read only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. The one or more processors are, for example, microprocessors or microcontrollers that execute instructions for the sound processing logic stored in memory device.

[0046] In the invisible hearing mode, the implantable sound sensors 165(1), 165(2) of the sensor array 160 are configured to detect / capture input sound signals 166 (e.g., acoustic sound signals, vibrations, etc.), which are provided to the implantable sound processing module 158. The implantable sound processing module 158 is configured to convert received input sound signals 166 (received at one or more of the implantable sound sensors 165(1), 165(2)) into output control signals 156 for use in stimulating the first ear of a recipient or user (i.e., the implantable sound processing module 158 is configured to perform sound processing operations). Stated differently, the one or more processors (e.g., processing element(s) implementing firmware, software, etc.) in the implantable sound processing module 158 are configured to execute sound processing logic in memory to convert the received input sound signals 166 into output control signals 156 that are provided to the stimulator unit 142. The stimulator unit 142 is configured to utilize the output control signals 156 to generate electrical stimulation signals (e.g., current signals) for delivery to the user’s cochlea, thereby bypassing the absent or defective hair cells that normally transduce acoustic vibrations into neural activity.

[0047] It is to be appreciated that the above description of the so-called external hearing mode and the so-called invisible hearing mode are merely illustrative and that the cochlear implant system 102 could operate differently in different embodiments. For example, in one alternativeAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 implementation of the external hearing mode, the cochlear implant 112 could use signals captured by the sound input devices 118 and the implantable sound sensors 165(1), 165(2) of sensor array 160 in generating stimulation signals for delivery to the user.

[0048] According to the techniques of the present disclosure, the external sound processing module 124 can also include an inertial measurement unit (IMU) 170. The IMU 170 is configured to measure the inertia of the user's head, that is, motion of the user's head. As such, the IMU 170 comprises one or more sensors 175 each configured to sense one or more of rectilinear or rotatory motion in the same or different axes. Examples of sensors 175 that can be used as part of inertial measurement unit 170 include accelerometers, gyroscopes, inclinometers, compasses, and the like. Such sensors can be implemented in, for example, micro electromechanical systems (MEMS) or with other technology suitable for the particular application.

[0049] As also illustrated in FIG. ID, in certain examples, a second IMU 180 including one or more sensors 185 is incorporated into implantable sound processing module 158 of implant body 134. The second IMU 180 can serve as an additional or alternative inertial measurement unit to the IMU 170 of external sound processing module 124. Like sensors 175, sensors 185 can each be configured to sense one or more of rectilinear or rotatory motion in the same or different axes. Examples of sensors 185 that can be used as part of inertial measurement unit 180 include accelerometers, gyroscopes, inclinometers, compasses, and the like. Such sensors can be implemented in, for example, MEMS or with other technology suitable for the particular application. For hearing devices that include an implantable sound processing module, such as implantable sound processing module 158, that includes an IMU, such as the IMU 180, the techniques presented herein can be implemented without an external processor. Accordingly, a hearing device that includes an implant body 134 and lacks an external component 104 can be configured to implement the techniques presented herein.

[0050] FIG. IE is a block diagram illustrating one example arrangement for an external device 110 (external computing device) configured to perform one or more operations in accordance with certain embodiments presented herein. As shown in FIG. IE, in its most basic configuration, the external device 1 10 includes at least one processing unit 183 and a memory 184. The processing unit 183 includes one or more hardware or software processors (e.g., Central Processing Units) that can obtain and execute instructions. The processing unit 183 can communicate with and control the performance of other components of the external device 110. The memory 184 is one or more software or hardware-based computer-readableAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 storage media operable to store information accessible by the processing unit 183. The memory 184 can store, among other things, instructions executable by the processing unit 183 to implement applications or cause performance of operations described herein, as well as other data. The memory 184 can be volatile memory (e.g., RAM), non-volatile memory (e.g., ROM), or combinations thereof. The memory 184 can include transitory memory or non-transitory memory. The memory 184 can also include one or more removable or non-removable storage devices. In examples, the memory 184 can include RAM, ROM) EEPROM (Electronically- Erasable Programmable Read-Only Memory), flash memory, optical disc storage, magnetic storage, solid state storage, or any other memory media usable to store information for later access. By way of example, and not limitation, the memory 184 can include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, other wireless media, or combinations thereof. In certain embodiments, the memory 184 comprises logic 195 and logic 196 that, when executed, enables the processing unit 183 to perform aspects of the techniques presented. By way of example, in lieu of, or in addition to, intelligence module 131 of FIG. ID providing intelligence functionality, logic 195 can include intelligence logic that provides intelligence functionality. Also, by way of example, in lieu of, or in addition to, instruction implementation module 135 of FIG. ID providing instruction implementation functionality, logic 196 can include instruction implementation logic that provides instruction implementation functionality.

[0051] In the illustrated example of FIG. IE, the external device 110 further includes a network adapter 186, one or more input devices 187, and one or more output devices 188. The external device 110 can include other components, such as a system bus, component interfaces, a graphics system, a power source (e.g., a battery), among other components. The network adapter 186 is a component of the external device 110 that provides network access (e.g., access to at least one network 189). The network adapter 186 can provide wired or wireless network access and can support one or more of a variety of communication technologies and protocols, such as Ethernet, cellular, Bluetooth, near-field communication, and RF, among others. The network adapter 186 can include one or more antennas and associated components configured for wireless communication according to one or more wireless communication technologies and protocols. The one or more input devices 187 are devices over which the external device 110 receives input from a user. The one or more input devices 187 can include physically- actuatable user-interface elements (e.g., buttons, switches, or dials), a keypad, keyboard, mouse, touchscreen, and voice input devices, among other input devices that can accept userAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 input. The one or more output devices 188 are devices by which the external device 110 is able to provide output to a user. The output devices 188 can include a display 190 (e.g., a liquid crystal display (LCD)) and one or more speakers 191, among other output devices for presentation of visual or audible information to the recipient, a clinician, an audiologist, or other user.

[0052] It is to be appreciated that the arrangement for the external device 110 shown in FIG. IE is merely illustrative and that aspects of the techniques presented herein can be implemented at a number of different types of systems / devices including any combination of hardware, software, and / or firmware configured to perform the functions described herein. For example, the external device 110 can be a personal computer (e.g., a desktop or laptop computer), a handheld device (e.g., a tablet computer), a mobile device (e.g., a smartphone), a surgical system, and / or any other electronic device having the capabilities to perform the associated operations described elsewhere herein.

[0053] As noted, a hearing device or hearing device system, such as cochlear implant system 102, can detect speech which has a primary meaning, and the detected speech can effectively be processed by an intelligence module to cause an instruction for an action to be taken to be generated based on a “recipient-specific second-order meaning associated with the speech.” That is, the intelligence module includes an intelligence system (e.g., an artificial intelligence (Al) model, such as a large language model (LLM) or a large action model (LAM), is configured to determine a linguistic context (second-order meaning) derived from the primary meaning and is further configured to determine a relevancy of the second-order meaning to the hearing device and / or hearing device system (i.e., determine a recipient-specific second-order meaning associated with the speech). Using the recipient-specific second-order meaning associated with the speech (the recipient-specific second-order meaning associated with the speech and the determined relevancy to the recipient / hearing device) one or more instructions for actions to be taken can be generated. It should be appreciated that an instruction can generally include one or more actions or actionable steps.

[0054] In general, as used herein, a “primary meaning” or a “first-order meaning” is a meaning that is substantially directly expressed or otherwise indicated by speech or language. For example, the conventional definitions of words used in speech can indicate a primary meaning associated with the speech. Also as used herein, “linguistic context” or a “second-order meaning” can be a meaning that is derived from, or otherwise determined from, speech but is not substantially directly expressed in the speech. That is, a linguistic context is a contextAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 implied or suggested by speech but is not based on conventional definitions of words used in the speech. In one embodiment, the linguistic context can be considered to be a meaning of speech that is not included in or otherwise directly evident in the actual words contained in the speech but can be understood or inferred based on the context associated with the words. As noted above, the “recipient-specific second-order meaning associated with the speech” refers to how the linguistic context is relevant to the recipient and / or the hearing device system (e.g., if and / or how the linguistic context / second order meaning applies to the recipient and / or the hearing device system). The recipient-specific second-order meaning associated with the speech is used to generate an instruction or a command that is not expressed or otherwise included in the speech.

[0055] By way of example, if speech is detected in sound signals obtained from a conversation that is captured by a hearing device includes a statement “we are at a library,” the primary meaning or first-order meaning of that statement is generally that at least one participant in the conversation is physically located at a library. A linguistic context or second-order meaning can be that at least one participant is in a relatively quiet environment, and the recipient-specific second-order meaning could be a determination that at least one participant in the conversation will begin speaking more softly or can stop speaking (as a library is typically a quiet environment). An instruction generated from the recipient-specific second-order meaning can be an instruction to recipient of the hearing device to speak more quietly, or an instruction to at least one component of the hearing device to adjust a configuration of the hearing device to account for softer voices.

[0056] In general, certain aspects of the techniques presented herein are described with reference to linguistic context determined from a conversation between a recipient and another individual. As noted above, it is to be appreciated that the embodiments presented herein include situations in which the recipient is not participating in a conversation, but instead uses the techniques presented herein to make adjustments to his / her device (e.g., linguistic context is determined from the recipient’s own speech). For example, the techniques presented herein can be beneficial for recipients who have reduced manual dexterity or cognitive capacity to otherwise make adjustments themselves. In such embodiment, the recipient need not explicitly state an instruction or adjustment to the device, but instead the system can determine the specific device adjustment based on the linguistic context of the recipient’s speech.

[0057] An intelligence system is configured to effectively interpret audio information such as speech that has a primary meaning such that a linguistic context can be identified and used toAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 generate an instruction. The intelligence system can be hosted on, or implemented as part of, a hearing system such as cochlear implant system 102 of FIGs. 1A-1D, an external device such as external device 110 of FIGs. 1A-1E, and / or a server on a network such as network 189 of FIG. IE.

[0058] Referring next to FIG. 2A, an intelligence system that is hosted on a hearing device such as a cochlear implant system will be described in accordance with an embodiment. An intelligence system that is hosted on a cochlear implant system, as for example on external component 104 of FIGs. 1A-1D, includes intelligence module 131 and instruction implementation module 135. Intelligence module 131 includes an Al model 210a, and instruction implementation module 135 includes a processing arrangement 235a, a device adjustment arrangement 235b, and a user instruction arrangement 235c.

[0059] Al model 210a is configured to process language or speech obtained in a sound signal from a conversation, and to determine a linguistic context or second-order meaning of the language or speech. In one embodiment, Al model 210a can utilize natural language processing (NLP) to determine the linguistic context. The linguistic context is generally a contextual meaning of the language or speech which is not expressly included or otherwise specified in the speech. Al model 210a, which can be, but is not limited to being, an LLM or an LAM can further process the linguistic context to determine a recipient-specific second- order meaning associated with the speech. Al model 210a can access a dataset 237 that is hosted on network 189, e.g., on a server within network 189, to obtain data that can be used to determine the linguistic context and / or the recipient-specific second-order meaning associated with the speech, respectively. It should be appreciated that intelligence module 131 may not be in direct communication with network 189 and can generally indirectly communicate with network 189 through external device 110 of FIGs. 1A-1E.

[0060] Intelligence module 131 provides information regarding the recipient-specific second- order meaning associated with the speech to instruction implementation module 135, which can be an interface that allows an instruction or a command to be implemented. Processing arrangement 235a can process the information regarding the recipient-specific second-order meaning associated with the speech to effectively determine an action that is responsive to the recipient-specific second-order meaning associated with the speech. For example, when the recipient-specific second-order meaning associated with the speech provides a solution set for an issue identified in the linguistic context, the action can be arranged to cause the solution set to be implemented.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0061] Processing arrangement 235a can provide the action to device adjustment arrangement 235b and / or user instruction arrangement 235c. Device adjustment arrangement 235b is configured to transform the action into an output which can be an instruction or command that can be used to adjust a configuration or setting on a device, e.g, cochlear implant system 102 of FIGs. 1A-1D. User instruction arrangement 235c is configured to effectively transform the action into an output which can be instruction or command that can be used to instruct a user of the cochlear implant system e.g., cochlear implant system 102 of FIGs. 1A-1D, to perform the action.

[0062] In one embodiment, dataset 237 can be part of intelligence module 131, as shown in FIG. 2B. That is, rather than being accessed on network 189, dataset 237 can instead be included as part of intelligence module 131, i.e., stored in intelligence module 131.

[0063] As mentioned above, an intelligence system that includes an Al model can be hosted on an external device such as external device 110 of FIGs. 1A-1E. FIG. 3A is a diagrammatic representation of intelligence logic and instruction implementation logic, e.g., intelligence logic 195 and instruction implementation logic 196 of FIG. IE, in which the intelligence logic includes an Al model in accordance with an embodiment. Intelligence logic 195 and instruction implementation logic 196, as shown in FIG. IE, are included on external component 104 of cochlear implant system 102. As shown, an Al model 395a can be implemented as part of intelligence logic 195. Intelligence logic 195 can communicate with network 189 to obtain data from dataset 237 that can be used by intelligence logic 195 to determine a linguistic context associated with speech obtained from a conversation, to identify the recipient-specific second- order meaning associated with the speech, and to identify one or more actions to take based on the linguistic context.

[0064] Instruction implementation logic 196 can include a processing arrangement 396a, a device adjustment arrangement 396b, and a user instruction arrangement 396c that cooperate to generate one or more instructions to implement based on one or more actions identified by intelligence logic 195.

[0065] Intelligence logic 195 can communicate with intelligence module 131”, which can include an Al model endpoint arrangement 310c that is configured either to obtain one or more actions identified by intelligence logic 195 and / or to provide information associated with speech to intelligence logic 195. Intelligence module 131” can also communicate with an implementation endpoint arrangement 335a of instruction implementation module 135’,Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 although instruction implementation logic 196 can also, or alternatively, communicate with implementation endpoint arrangement 335a. Implementation endpoint arrangement 335a can be arranged to cause at least one configuration or parameter associated with cochlear implant system 102 of FIGs. 1 A- ID to be changed.

[0066] Dataset 237 can be incorporated into intelligence logic 195, rather than being accessed by on network 189. As shown in FIG. 3B, intelligence logic 195’ can include dataset 237. That is, dataset 237 can be locally stored with respect to Al model 395a.

[0067] In one embodiment, an intelligence system can be implemented on a server arrangement in a network, e.g, network 189 ofFIG. IE. That is, the resources that can be used to determine a linguistic context of speech, actions to take based on the linguistic context, etc. can be provided by one or more servers rather than by cochlear implant system 102 and / or external device 110 of FIGs. 1A-1E. FIG. 4A is a diagrammatic representation of a server arrangement that is arranged to cooperate with an intelligence module and an instruction implementation model, e.g., intelligence module 131 and instruction implementation module 135 of FIG. ID, in accordance with an embodiment. Intelligence module 131”” includes an Al model endpoint arrangement 410c, and instruction implementation module 135”’ includes an implementation endpoint arrangement 435a. Intelligence module 131”” is in communication with a server arrangement 493 on network 189.

[0068] Server arrangement 493, which can include one or more servers, includes an Al model 493a, a processing arrangement 493b, a device adjustment arrangement 493c, a user instruction arrangement 493d, and dataset 237. Al model 493a can interpret speech which has a primary or first-order meaning, to identify a linguistic context or second-order meaning. Al model 493a can further process the linguistic context to determine one or more actions to be taken. Processing arrangement 493b can process the one or more actions identified by Al model 493a and provide instructions to device adjustment arrangement 493c and / or user instruction arrangement 493d as appropriate. Processing arrangement 493b can, for example, provide an instruction to device adjustment arrangement 493c which indicates how a particular configuration on cochlear implant system 102 of FIGs. 1A-D are to be adjusted, or can provide an instruction to user instruction arrangement 493d which instructs how a user of cochlear implant system 102 can be instructed to perform an act. Dataset 237 can be used by Al model 493a to facilitate determining one or more actions that can be undertaken based on speech obtained from a conversation.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0069] Information relating to one or more actions to be taken and / or instructions to be implemented can be provided by server arrangement 493 to intelligence module 131”” and instruction implementation module 135”’. Upon obtaining instructions, for example, implementation endpoint arrangement 435a can cause instructions to be implemented.

[0070] In lieu of communicating substantially directly with intelligence module 131”” and / or instruction implementation module 135”’, server arrangement 493 can instead communicate with intelligence logic 195 and / or instruction implementation logic 196 of FIG. IE. FIG. 4B is a diagrammatic representation of server arrangement 493 communication with intelligence logic 195 and instruction implementation logic 196 of FIG. IE, in accordance with an embodiment. Intelligence logic 195” includes an Al model endpoint arrangement 495c, and instruction implementation logic 196’ includes an implementation endpoint arrangement 496d. Artificial intelligence model endpoint arrangement 495c can obtain one or more actions associated with a linguistic context of speech from server arrangement 493, and implementation endpoint arrangement 496d can obtain one or more instructions to be implemented from server arrangement 493.

[0071] Intelligence logic 195” can communicate with intelligence module 131’””, which can include an Al model endpoint arrangement 410c that is configured either to obtain one or more actions identified by intelligence logic 195” and / or to provide information associated with speech to intelligence logic 195”. Intelligence module 131”’” can also communicate with an implementation endpoint arrangement 435a of instruction implementation module 135’”, although instruction implementation logic 196’ can also, or alternatively, communicate with implementation endpoint arrangement 435a.

[0072] FIG. 5 is a process flow diagram which illustrates an overall method of causing an action to be taken based on contextual audio information, e.g., speech from a conversation, in accordance with an embodiment. A method 505 of causing an action to be taken based on contextual audio information begins at a step 509 in which a cochlear implant system obtains audio information or sound signals, e.g., language or speech from a conversation. When the audio information includes speech, the actual meaning of words included in the speech are effectively a first-order meaning associated with the speech.

[0073] Once the cochlear implant system obtains the audio information, the cochlear implant provides the audio information to an intelligence system, e.g., an Al model such as an LLM or an LAM, in a step 513. As discussed above, the intelligence system can be included on theAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 cochlear implant, an external device that is in communication with the cochlear implant system, and / or on a server in a network that is in communication with the cochlear implant system.

[0074] The intelligence system is generally configured to determine a recipient-specific second-order meaning of the audio information in a step 517. That is, the intelligence system interprets the audio information to deduce, infer, identify, surmise, understand, reason, and / or suggest a second-order meaning of the audio information, and determines how / why the second- order meaning is relevant to / for the recipient and / or the cochlear implant system. The intelligence system can, in one embodiment, apply natural language processing to the audio information to determine a sentiment that is associated with the audio information. In one embodiment, the second-order meaning can effectively be extracted from the audio information using one or more contextual cues.

[0075] Upon determining the recipient-specific second-order meaning, the Al model determines an action behind the recipient-specific second-order meaning in a step 525. The Al model can process the recipient-specific second-order meaning to identify the appropriate action, which in certain examples can include a solution set to one or more issues associated with the second-order meaning. When the action behind the recipient-specific second-order meaning includes a solution set, the solution set is effectively a solution to an issue or problem that was not specifically articulated in the audio information. For example, the action behind the recipient-specific second-order meaning can include identifying a setting for a cochlear implant that can address or substantially mitigate an issue that, while not directly articulated in the audio information, was contextually implied in the audio information. Identifying a setting can include, but is not limited to including, selecting a suitable setting from a set of possible settings.

[0076] In a step 529, the Al model provides the action behind the recipient-specific second- order meaning to an instruction implementation arrangement or interface. Once the instruction implementation arrangement obtains the action behind the recipient-specific second-order meaning, the instruction implementation arrangement takes an action in a step 533. For example, the instruction implementation arrangement can issue an instruction or command to adjust a configuration of a cochlear implant system, or the instruction implementation arrangement can issue an audible instruction or command to a user of the cochlear implant system to perform a task. In one embodiment, a user or recipient of the cochlear implant system can be asked to either accept or reject the instruction or command to adjust, e.g., substantially automatically adjust, the configuration of the cochlear implant system. After the instructionAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 implementation arrangement takes an action, the method of causing an action to be taken based on contextual audio information is completed.

[0077] Referring next to FIGs. 6A and 6B, an exemplary method of causing an action to be taken based on speech detected during a conversation will be described in accordance with an embodiment. A method 605 of causing an action to be taken based on speech detected during a conversation begins at a step 609 in which a cochlear implant system obtains, or picks up, speech or speech information. The cochlear implant system provides the speech information to an intelligence system, as for example an Al model, in a step 613. The Al model can be hosted on the cochlear implant system, on an external device that is in communication with the cochlear implant system, and / or on a server in a network that can be accessed either directly or indirectly by the cochlear implant system.

[0078] In an optional step 615, the Al model can obtain non-linguistic context. For example, the Al model can obtain, through the cochlear implant system or an external device in communication with the cochlear implant, environmental parameters including, but not limited to including information on whether the conversation is occurring outdoors or indoors, a temperature of the environment, noise levels, and sentiments associated with tones in voices. In one embodiment, the non-linguistic context can include at least one state associated with the cochlear implant system or the external device.

[0079] The Al model determines a recipient-specific second-order meaning associated with the speech information in a step 617. Such a determination can be made using the speech information and the optional non-linguistic context. The speech information and the optional non-linguistic context provide contextual cues to the Al model that can be used to identify the recipient-specific second-order meaning. The speech information and, optionally the non- linguistic context, can be processed by the Al model, e.g., using NLP, to create or determine the recipient-specific second-order meaning associated with the speech. For example, when speech information from a conversation contains the words “we have arrived at the library,” the linguistic context or second-order meaning can indicate that participants in the conversation will begin to speak more quietly, as a library is generally a relatively quiet location.

[0080] After the recipient-specific second-order meaning associated with the speech is determined, the Al model determines an action behind the recipient-specific second-order meaning associated with the speech in a step 625. For example, when the recipient-specific second-order meaning associated with the speech is that a conversation is to become quieter,Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 the action behind the recipient-specific second-order meaning associated with the speech can identify a suitable mitigation for the quieter conversation that includes adjusting settings on the cochlear implant system by selecting suitable settings from a group of possible settings. In other words, a solution to a conversation that is to become quieter can involve providing a user of the cochlear implant system with an ability to hear the quieter conversation.

[0081] Once the Al model determines or identifies the action behind the recipient-specific second-order meaning associated with the speech, the Al model provides the action to an instruction implementation arrangement or interface in a step 629. The instruction implementation arrangement can be hosted on the cochlear implant system, on an external device in communication with the cochlear implant system, and / or on a server on a network that is in direct or indirect communication with the cochlear implant system. The instruction implementation arrangement can effectively determine an action to take in response to the recipient-specific second-order meaning associated with the speech. For example, when the action behind the recipient-specific second-order meaning associated with the speech is that the cochlear implant system can be configured to compensate for a quieter conversation, the instruction implementation arrangement can determine which settings to adjust on the cochlear implant system, and how much the settings can be adjusted.

[0082] In a step 631, the instruction implementation arrangement issues an instruction based on the action behind the recipient-specific second-order meaning associated with the speech. A determination is then made in a step 635 as to whether there is an action for a user, z.e., the recipient of the cochlear implant system, to take. For example, if the recipient-specific second- order meaning associated with the speech involves a determination that a conversation is to become quieter, the user can be asked to take an action that involves talking more quietly or whispering. If there is an action for the user to take, process flows from step 635 to a step 639 in which the instruction implementation arrangement notifies the user of an action for the user to take. A notification to the user can be an audible notification, or can be a visual notification, e.g., a text, that is provided on an external device.

[0083] After the user is notified of an action to take, or when the determination in step 635 is that there is no action for the user to take, a determination is made in a step 643 as to whether there is an action that involves an adjustment to the cochlear implant system. If it is determined that there is an adjustment, e.g., an automatic adjustment, to be made to the cochlear implant system, the instruction implementation arrangement adjusts the configuration of, or a program executing on, the cochlear implant system in a step 647. Adjusting the configuration canAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 include, in one embodiment, selecting an appropriate setting from a set of settings and causing the selected setting to be implemented based on the recipient-specific second-order meaning associated with the speech. For example, a configuration that either decreases or increases noise-cancellation can be adjusted. It should be appreciated that in some embodiments, a user or recipient of the cochlear implant system can be asked to approve or reject an adjustment before the adjustment is made.

[0084] In one embodiment, an external system can be arranged to be substantially controlled through the cochlear implant system and / or an external device that is in communication with the cochlear implant system. The external system can be a smart home integration, for example. The external system can also involve telecoil activation and / or synchronizing Bluetooth® streams such as Auracast™ streams. For example, in a loud stadium environment, the external system can be associated with a commentators’ box, and turning on Auracast™ streaming can enable a connection to the commentators’ box to be made.

[0085] After the instruction implementation arrangement adjusts the configuration of the cochlear implant system in step 647, or if it is determined in step 643 that there is no adjustment to be made to the cochlear implant system, process flow proceeds to an optional step 651 in which it is determined whether there is an action that involves an external system. If the determination is that there is no action that involves an external system, the method of causing an action to be taken based on speech detected during a conversation is completed. Alternatively, if the determination in optional step 651 that there is an action that involves an external system, the process flow moves from step 651 to an optional step 655 in which the instruction implementation arrangement causes an adjustment to be made to an external system, and the method of causing an action to be taken based on speech detected during a conversation is completed.

[0086] FIG. 7 is a diagrammatic representation of a process of generating an output based on speech content having a first-order meaning that includes identifying a recipient-specific second-order meaning associated with the speech and an action behind the recipient-specific second-order meaning associated with the speech associated with the speech content in accordance with an embodiment. A system 713, which can be a framework or a platform, is arranged to generate an output 727 from speech 717 having a first-order meaning or primary meaning, and includes cochlear implant system 102, device 110, and network 189. Cochlear implant system 102 is partially implanted in, and partially mounted on, head 154 of a user, as discussed above with respect to FIGs. 1A and IB. System 713 generally includes intelligenceAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 logic that can include, but is not limited to including, logic that includes an Al model such as an LLM or an LAM. Speech 717, or language content, which has a first-order meaning is obtained by cochlear implant system 102 and can be processed within system 713. Within system 713, a recipient-specific second-order meaning associated with the speech 717 can be determined. Non-speech context 719 can be obtained by system 713 and can used to facilitate a determination of the recipient-specific second-order meaning associated with the speech 717. An action behind recipient-specific second-order meaning associated with the speech 717 can be transformed or otherwise converted into one or more actions for which instructions or commands can be provided. Generating the actions or instructions can include identifying at least one configuration or state associated with cochlear implant system 102 and based on second-order meaning of speech 717.

[0087] Output 727, which can generally include instructions or commands that correspond to actions associated with the action behind recipient-specific second-order meaning associated with the speech 717, can be provided to, but is not limited to being provided to, cochlear implant system 102, head 154 of a user, and / or an external system 741. Output 727 can be provided to cochlear implant system 102 as an instruction for cochlear implant system 102 to adjust, modify, change, or update a state or a setting. When output 727 is effectively provided to head 154 of the user using cochlear implant system 102, output 727 can be provided audibly or visually such that the user can hear and / or see an instruction for an act the user is to take or follow. For example, a visual indication can be provided on a display or display screen associated with cochlear implant system 102. Output 727 that is provided to external system 741 can generally be arranged to cause a setting or configuration of external system 741 to be adjusted, modified, changed, or updated.

[0088] Referring next to FIG. 8. a process of analyzing speech content provided to an artificial intelligence system that includes determining a second-order meaning and an action behind the second-order meaning in accordance with an embodiment. A system 813 includes intelligence logic 895 and instruction implementation logic 896. Intelligence logic 895 and instruction implementation logic 896 can generally be located on a server arrangement in a network, on an external device associated with a cochlear implant system, or on a cochlear implant system. Intelligence logic 895 includes an Al model 895a. Instruction implementation logic 896 includes a processing arrangement 896a, a device adjustment arrangement 896b, and a user instruction arrangement 896c.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0089] Speech 817 or language content which has a first-order meaning is obtained by Al model 895a which processes speech 817 to determine a recipient-specific second-order meaning 857. Recipient-specific second-order meaning 857 is further processed by Al model 895a to determine an action 859 behind recipient-specific second-order meaning 857. Al model 895a provides action 859 to processing arrangement 896a. In one embodiment, Al model 895a can obtain non-speech context 819, and process non-speech context 819 along with speech to generate recipient-specific second-order meaning 857. Non-speech context 819 can include a state or a configuration of a cochlear implant system.

[0090] Processing arrangement 896a is configured to process action 859 to determine at least one instruction 861. Once instruction 861 is generated, instruction 861 is provided to device adjustment arrangement 896b and / or user instruction arrangement 896c such that an output 827 can be generated. Output 827 can include, but is not limited to including, an instruction to cause a configuration or a setting of a cochlear implant system to be adjusted, an instruction for a user of a cochlear implant system to act or to otherwise perform a task, and / or an instruction to cause a configuration of an external system to be adjusted.

[0091] FIG. 9 is a diagrammatic representation of an example of generating an output from speech content, as processed by system 813 of FIG. 8, in accordance with an embodiment. In one example, speech 817’ includes a statement in which a party to a conversation asks “did we pack everything for the roadtrip today?” The recipient-specific second-order meaning 857’ for speech 817’ can be based on an interpretation of speech 817’, along with non-speech context 819’, to determine that “roadtrips have different parameters which can affect what to bring.” Non-speech context 819’ can include context including, but not limited to including, environment, temperature, noise level, sentiment, and at least one state or configuration associated with a cochlear implant system.

[0092] The recipient-specific second-order meaning 857’ can be used to determine action 859’ . Action 859’ indicates that an approximate time frame for the roadtrip can determine whether a battery to charge a cochlear implant system should be packed for the roadtrip.

[0093] Instruction 861’ can be generated based on an estimated battery life for a cochlear implant system, and an inferred length of the roadtrip. Prior information, e.g., information obtained from context building, can be used to facilitate determining an inferred length of the roadtrip. Instruction 861’ can be provided in output 827’. Output 827’ can be an instructionAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 to a user of a cochlear implant system or any other recipient of output 827’ to pack a charger for the cochlear implant system.

[0094] As recipient-specific second-order meanings are determined, the recipient-specific second-order meanings can be used as training data to improve the accuracy with which recipient-specific second-order meanings are determined in the future. For example, using information generated by a context interpreter to train the context interpreter can improve future performance of the context interpreter.

[0095] The use of an LLM or an LAM for understanding the linguistic context or second-order meaning of sound signals containing speech enables actionable commands which are not directly articulated in the speech to be identified. The actionable commands can involve a hearing device or medical device such as a cochlear implant system substantially automatically adjusting device configurations and programs based on the linguistic context. It should be appreciated that in lieu of automatic adjustments being made, a user can instead accept or reject adjustments.

[0096] Certain aspects have generally been described herein with reference to a conversation between a recipient and another individual. As noted above, it is to be appreciated that the embodiments presented herein include situations in which the recipient is not participating in a conversation, but instead uses the techniques presented herein to make adjustments to his / her device. For example, the techniques presented herein can be beneficial for recipients who have reduced manual dexterity or cognitive capacity to otherwise make adjustments themselves.

[0097] As previously described, the technology disclosed herein can be applied in any of a variety of circumstances and with a variety of different devices. Example hearing devices that can benefit from technology disclosed herein are described in more detail in FIGS. 10 and 11. The techniques of the present disclosure can be applied to other devices, such as neurostimulators, cardiac pacemakers, cardiac defibrillators, sleep apnea management stimulators, seizure therapy stimulators, tinnitus management stimulators, and vestibular stimulation devices, as well as other medical devices that deliver stimulation to tissue. Further, technology described herein can also be applied to consumer devices. These different systems and devices can benefit from the technology described herein.

[0098] FIG. 10 illustrates an example vestibular stimulator system 1002, with which embodiments presented herein can be implemented. As shown, the vestibular stimulator system 1002 comprises an implantable component (vestibular stimulator) 1012 and an externalAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 device / component 1004 (e.g., external processing device, battery charger, remote control, etc.). The external device 1004 comprises a transceiver unit 1060. As such, the external device 1004 is configured to transfer data (and potentially power) to the vestibular stimulator 1012.

[0099] The vestibular stimulator 1012 comprises an implant body (main module) 1034, a lead region 1036, and a stimulating assembly 1016, all configured to be implanted under the skin / tissue (tissue) 1015 of the recipient. The implant body 1034 generally comprises a hermetically-sealed housing 1038 in which RF interface circuitry, one or more rechargeable batteries, one or more processors, and a stimulator unit are disposed. The implant body 134 also includes an intemal / implantable coil 1014 that is generally external to the housing 1038, but which is connected to the transceiver via a hermetic feedthrough (not shown).[ootoo] The stimulating assembly 1016 comprises a plurality of electrodes 1044( l)-(3) disposed in a carrier member (e.g., a flexible silicone body). In this specific example, the stimulating assembly 1016 comprises three (3) stimulation electrodes, referred to as stimulation electrodes 1044(1), 1044(2), and 1044(3). The stimulation electrodes 1044(1), 1044(2), and 1044(3) function as an electrical interface for delivery of electrical stimulation signals to the recipient’s vestibular system.[ooiot] The stimulating assembly 1016 is configured such that a surgeon can implant the stimulating assembly adjacent the recipient’s otolith organs via, for example, the recipient’s oval window. It is to be appreciated that this specific embodiment with three stimulation electrodes is merely illustrative and that the techniques presented herein can be used with stimulating assemblies having different numbers of stimulation electrodes, stimulating assemblies having different lengths, etc.

[0102] In operation, the vestibular stimulator 1012, the external device 1004, and / or another external device can be configured to implement the techniques presented herein. That is, the vestibular stimulator 1012, possibly in combination with the external device 1004 and / or another external device, can include an evoked biological response analysis system, as described elsewhere herein.

[0103] FIG. 11 illustrates a retinal prosthesis system 1101 that comprises an external device 1110 (which can correspond to the wearable device 100) configured to communicate with an implantable retinal prosthesis 1100 via signals 1151. The retinal prosthesis 1100 comprises an implanted processing module 1125, and a retinal prosthesis sensor-stimulator 1190 isAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 positioned proximate the retina of a recipient. The external device 1110 and the processing module 1125 can communicate via coils 1108, 1114.

[0104] In an example, sensory inputs (e.g., photons entering the eye) are absorbed by a microelectronic array of the sensor-stimulator 1190 that is hybridized to a glass piece 1192 including, for example, an embedded array of microwires. The glass can have a curved surface that conforms to the inner radius of the retina. The sensor-stimulator 1190 can include a microelectronic imaging device that can be made of thin silicon containing integrated circuitry that convert the incident photons to an electronic charge.

[0105] The processing module 1125 includes an image processor 1123 that is in signal communication with the sensor-stimulator 1190 via, for example, a lead 1188 that extends through surgical incision 1189 formed in the eye wall. In other examples, processing module 1125 is in wireless communication with the sensor-stimulator 1190. The image processor 1123 processes the input into the sensor-stimulator 1190 and provides control signals back to the sensor-stimulator 1190 so the device can provide an output to the optic nerve. That said, in an alternate example, the processing is executed by a component proximate to, or integrated with, the sensor-stimulator 1190. The electric charge resulting from the conversion of the incident photons is converted to a proportional amount of electronic current which is input to a nearby retinal cell layer. The cells fire and a signal is sent to the optic nerve, thus inducing a sight perception.

[0106] The processing module 1125 can be implanted in the recipient and function by communicating with the external device 1110, such as a BTE unit, a pair of eyeglasses, etc. The external device 1110 can include an external light / image capture device (e.g., located in / on a behind-the-ear device or a pair of glasses, etc.), while, as noted above, in some examples, the sensor-stimulator 1190 captures light / images, in which sensor-stimulator 1190 is implanted in the recipient.

[0107] As should be appreciated, while particular uses of the technology have been illustrated and discussed above, the disclosed technology can be used with a variety of devices in accordance with many examples of the technology. The above discussion is not meant to suggest that the disclosed technology is only suitable for implementation within systems akin to that illustrated in the figures. In general, additional configurations can be used to practice the processes and systems herein and / or some aspects described can be excluded without departing from the processes and systems disclosed herein.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1

[0108] This disclosure described some aspects of the present technology with reference to the accompanying drawings, in which only some of the possible aspects were shown. Other aspects can, however, be embodied in many different forms and should not be construed as limited to the aspects set forth herein. Rather, these aspects were provided so that this disclosure was thorough and complete and fully conveyed the scope of the possible aspects to those skilled in the art.

[0109] As should be appreciated, the various aspects (e.g., portions, components, etc.) described with respect to the figures herein are not intended to limit the systems and processes to the particular aspects described. Accordingly, additional configurations can be used to practice the methods and systems herein and / or some aspects described can be excluded without departing from the methods and systems disclosed herein.[oono] According to certain aspects, systems and non-transitory computer readable storage media are provided. The systems are configured with hardware configured to execute operations analogous to the methods of the present disclosure. The one or more non-transitory computer readable storage media comprise instructions that, when executed by one or more processors, cause the one or more processors to execute operations analogous to the methods of the present disclosure.[oom] Similarly, where steps of a process are disclosed, those steps are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps. For example, the steps can be performed in differing order, two or more steps can be performed concurrently, additional steps can be performed, and disclosed steps can be excluded without departing from the present disclosure. Further, the disclosed processes can be repeated.

[0112] Although specific aspects were described herein, the scope of the technology is not limited to those specific aspects. One skilled in the art will recognize other aspects or improvements that are within the scope of the present technology. Therefore, the specific structure, acts, or media are disclosed only as illustrative aspects. The scope of the technology is defined by the following claims and any equivalents therein.

[0113] It is also to be appreciated that the embodiments presented herein are not mutually exclusive and that the various embodiments can be combined with another in any of a number of different manners.

Claims

Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1CLAIMSWhat is claimed is:

1. A method, comprising: receiving sound signals at a hearing device, wherein the sound signals include speech, the speech having a first-order meaning; determining a recipient-specific second-order meaning associated with the speech; and generating at least a first instruction based on the recipient-specific second-order meaning, wherein the first instruction is not included in the speech.

2. The method of claim 1, wherein the first instruction is a command, and wherein the method further includes: providing the command to an interface, wherein the command is arranged to cause the interface to change a configuration associated with the hearing device.

3. The method of claim 1, wherein the first instruction is a command, the method further including: providing the command to an interface, wherein the command is a selection of one of a plurality of settings associated with the hearing device.

4. The method of claim 1, further including: providing the first instruction to a user of the hearing device, wherein providing the instruction includes audibly providing the first instruction.

5. The method of claim 1, wherein determining the recipient-specific second-order meaning associated with the speech includes: processing the sound signals on a second device, the second device being in communication with the hearing device across a network.

6. The method of claim 1, wherein generating the at least first instruction based on the recipient-specific second-order meaning includes identifying at least one state associated withAtty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 the hearing device and generating the first instruction based on the recipient-specific second- order meaning and the at least one state.

7. The method of claim 1, 2, 3, 4, 5, or 6, wherein determining the recipient-specific second-order meaning associated with the speech comprises: processing the sound signals with natural language processing.

8. The method of claim 1, 2, 3, 4, 5, or 6 wherein determining the recipient-specific second-order meaning associated with the speech comprises: processing the sound signals with a Large Language Model (LLM) or a Large Action Model (LAM).

9. The method of claim 1, 2, 3, 4, 5, or 6, wherein determining the recipient-specific second-order meaning associated with the speech comprises: determining a second-order meaning associated with the speech; and determining a relevancy of the second order-meaning to the hearing device or to a recipient of the hearing device.

10. A system, comprising: a display screen; a memory; and at least one processor operable coupled to the display screen and the memory, wherein the at least one processor is configured to: receive sound signals at a hearing device, wherein the sound signals include speech, the speech having a first-order meaning; determine a recipient-specific second-order meaning associated with the speech; and generate at least a first instruction based on the recipient-specific second-order meaning, wherein the first instruction is not included in the speech.

11. The system of claim 10, wherein the first instruction is a command, and wherein the processor is further configured to: provide the command to an interface, wherein the command is arranged to cause the interface to change a configuration associated with the hearing device.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC112. The system of claim 10, wherein the first instruction is a command, and wherein the processor is further configured to: provide the command to an interface, wherein the command is a selection of one of a plurality of settings associated with the hearing device.

13. The system of claim 10, wherein the processor is further configured to: provide the first device instruction to a user of the hearing device, wherein the instructions operable to cause the processor to provide the instruction are operable to audibly provide the first instruction.

14. The system of claim 10, wherein the processor configured to determine the recipientspecific second-order meaning associated with the speech is further configured to: process the sound signals on a second device, the second device being in communication with the hearing device across a network.

15. The system of claim 10, wherein the processor configured to generate the at least first device instruction based on the recipient-specific second-order meaning is further configured to identify at least one state associated with a device and generating the first device instruction based on the recipient-specific second-order meaning and the at least one state.

16. The system of claim 10, 11, 12, 13, 14, or 15, wherein the processor operable to determine the recipient-specific second-order meaning associated with the speech is further operable to process the sound signals with natural language processing.

17. The system of claim 10, 11, 12, 13, 14, or 15, wherein the processor operable to determine the recipient-specific second-order meaning associated with the speech is further operable to cause the processor to process the sound signals with a Large Language Model (LLM) or a Large Action Model (LAM).

18. One or more non-transitory computer readable storage media comprising instructions that, when executed by one or more processors, cause the one or more processors to: obtain sound signals at a medical device, wherein the sound signals include speech;Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 process the speech to identify a contextual meaning associated with the speech, wherein the meaning is not expressly included in the speech; interpret the contextual meaning; and generate at least a first instruction based on the interpreting of the contextual meaning.

19. The one or more non-transitory computer readable storage media of claim 18, wherein the instructions that cause the one or more processors to interpret the contextual meaning comprise instructions that cause the one or more processors to: determine a relevancy of the contextual meaning to the medical device or to a recipient of the medical device.

20. The one or more non-transitory computer readable storage media of claim 18 or 19, further comprising instructions that cause the one or more processors: transform the at least first instruction into one selected from a group including a command to the medical device and an audible instruction provided to a recipient of the medical device.

21. The one or more non-transitory computer readable storage media of claim 20, wherein the command to the device is a command to adjust a first setting of the medical device.

22. The one or more non-transitory computer readable storage media of claim 18 or 19, wherein the at least first instruction includes a plurality of actionable steps.

23. The one or more non-transitory computer readable storage media of claim 18 or 19, wherein the instructions that cause the one or more processors to process the speech to identify the contextual meaning comprise instructions that cause the one or more processors to: apply natural language processing to the speech.

24. The one or more non-transitory computer readable storage media of claim 18 or 19, wherein the meaning includes an identified issue, and wherein the at least first device instruction addresses the identified issue.

25. The one or more non-transitory computer readable storage media of claim 24, wherein the at least first device instruction includes a solution to the identified issue.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC126. A system, comprising: a display screen; a memory; and at least one processor operable coupled to the display screen and the memory, wherein the at least one processor is configured to: obtain sound signals at a medical device, wherein the sound signals include speech; process the speech to identify a contextual meaning associated with the speech, wherein the meaning is not expressly included in the speech; interpret the contextual meaning; and generate at least a first instruction based on the interpreting of the contextual meaning.

27. The system of claim 26, wherein the at least one processor is further configured to: transform the at least first device instruction into one selected from a group including a command to the medical device and an audible instruction provided to a recipient of the medical device.

28. The system of claim 27, wherein the command to the device is a command to adjust a first setting of the medical device.

29. The system of claim 26, wherein the at least first instruction includes a plurality of actionable steps.

30. The system of claim 26, 27, 28, or 29, wherein the processor configured to process the speech to identify the contextual meaning is further configured to: apply natural language processing to the speech.

31. The system of claim 26, 27, 28, or 29, wherein the meaning includes an identified issue, and wherein the at least first instruction addresses the identified issue.

32. The system of claim 31, wherein the at least first instruction includes a solution to the identified issue.Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC133. The system of claim 26, 27, 28, or 29, wherein to interpret the contextual meaning, the at least one processor is configured to determine a relevancy of the contextual meaning to the medical device or to a recipient of the medical device.

34. A method, comprising: obtaining sound signals at a first device of a hearing device system, wherein the sound signals include speech; extracting at least one contextual cue from the sound signals; determining, based on the at least one contextual cue, a linguistic context associated with the speech; and adjusting operation of the first device based on the linguistic context.

35. The method of claim 34, wherein adjusting operation of the first device based on the linguistic context includes: determining a relevancy of the linguistic context associated with the speech to the hearing device system and / or to a recipient of the hearing device system.

36. The method of claim 34 or 35, further including: generating an instruction using the linguistic context, wherein the instruction is not included in the speech.

37. The method of claim 36, further including: transforming the instruction into one selected from a group including a command to the first device, an audible instruction provided to a recipient of the hearing device system, and a command to a second device.

38. The method of claim 37, wherein the first device is a hearing device, and wherein the method further includes: implementing the instruction, wherein implementing the instruction includes adjusting at least a first setting of a plurality of settings on the hearing device.

39. One or more non-transitory computer readable storage media comprising instructions that, when executed by a processor, cause the processor to: obtain sound signals at a first device, wherein the sound signals include speech;Atty. Docket No. 3065.0810i Client Ref. No. CID03835WOPC1 extract at least one contextual cue from the sound signals; determine, based on the at least one contextual cue, a context associated with the speech; and adjust operation of the first device based on the context.

40. The one or more non-transitory computer readable storage media of claim 39, wherein the speech has a first-order meaning and the context associated with the speech is a recipientspecific second-order meaning associated with the speech.

41. The one or more non-transitory computer readable storage media of claim 39 or 40, wherein the instructions are further operable to cause the processor to: generate a device instruction using the recipient-specific second-order meaning, wherein the device instruction is not included in the speech.

42. The one or more non-transitory computer readable storage media of claim 41 wherein the instructions are further operable to cause the processor to: transform the device instruction into one selected from a group including a command to the first device, an audible instruction provided to a user of the first device, and a command to a second device.

43. The one or more non-transitory computer readable storage media of claim 41, wherein the first device is a hearing device, the instructions further being operable to cause the processor to: implement the device instruction by adjusting at least a first setting of a plurality of settings on the hearing device.

Images