Auxiliary device for prosthesis-assisted communication

Abstract

The invention relates to an assist device for prosthetic assisted communication. A system includes a signal input, a processor, and a signal output, where the processor is configured to generate instructions related to data regarding a recipient of a sensory prosthesis based on an input of the signal input, and the signal output is configured to output data indicative of the instructions.

Description

[0001] This application is a divisional application of the Chinese invention patent application with an international application date of October 22, 2018, a national application number of 201880068125.4, and an invention name of “Assistive device for prosthesis-assisted communication”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to U.S. Provisional Application No. 62 / 575,567, filed on October 23, 2017, entitled “ADVANCED ASSISTANCE FOR PROSTHESIS ASSISTED COMMMUNICATION,” whose inventor is Kenneth OPLINGER of Macquarie University, Australia, the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0004] The present disclosure relates to assistive devices for prosthesis-assisted communication. Background Art

[0005] Hearing loss can be caused by many different reasons, generally of two types: conductive and sensorineural. Sensorineural hearing loss is caused by the absence or destruction of hair cells in the cochlea, which convert sound signals into nerve impulses. Various hearing prostheses are commercially available to provide individuals with sensorineural hearing loss with the ability to perceive sound. One example of a hearing prosthesis is a cochlear implant.

[0006] Conductive hearing loss occurs when damage to, for example, the ossicular chain or the ear canal blocks the normal mechanical pathway that provides sound to the hair cells in the cochlea. Individuals with conductive hearing loss may retain some form of residual hearing because the hair cells in the cochlea may not be damaged.

[0007] Individuals with hearing loss typically receive acoustic hearing aids. Conventional hearing aids rely on the principle of air conduction to transmit acoustic signals to the cochlea. Specifically, hearing aids typically use an arrangement positioned in the recipient's ear canal or on the outer ear to amplify the sound received by the recipient's outer ear. This amplified sound reaches the cochlea, causing movement of the perilymph and stimulating the auditory nerve. Cases of conductive hearing loss are typically treated with the aid of bone conduction hearing aids. In contrast to conventional hearing aids, these devices use a mechanical actuator coupled to the skull to apply the amplified sound.

[0008] In contrast to hearing aids, which rely primarily on the principle of air conduction, certain types of hearing prostheses, commonly referred to as cochlear implants, convert received sounds into electrical stimulation, which is applied to the cochlea, which enables the perception of received sounds.

[0009] Many devices, such as medical devices that interface with a recipient, have structural and / or functional features where there is utility in adjusting such features for an individual recipient. The process of tailoring or customizing or otherwise adjusting a device that interfaces with or is otherwise used by a recipient to a particular need or particular need or particular features of the recipient is generally referred to as fitting. One class of medical devices that has utility in fitting these aspects to an individual recipient is the cochlear implants described above. That said, there are other types of medical devices, such as other types of hearing prostheses, where there is utility in fitting these medical devices to a recipient.

[0010] There are other types of medical devices that enhance or otherwise provide sensory stimulation, such as, by way of example only and not limitation, visual prostheses such as retinal implants. These devices (hearing, vision, etc.) will be collectively referred to herein as sensory prostheses or sensory medical devices. Some embodiments of some such sensory prostheses include one or more sensory stimulation-induced phenomenon capture devices, such as, by way of example only and not limitation, a microphone or camera, etc. Note that sensory stimulation-induced phenomenon does not require that the phenomenon induce stimulation in all people (e.g., the phenomenon exists regardless of whether it can be seen by, for example, a blind person or heard by a deaf person). Summary of the invention

[0011] According to an exemplary embodiment, a system comprises: a signal input kit; a processor; and a signal output, wherein the processor is configured to generate instructions related to data about a recipient of a sensory prosthesis based on input of the signal input, and the signal output is configured to output data indicative of the instructions.

[0012] According to another exemplary embodiment, a method includes: capturing sensory input during an interaction between two or more individuals, one of whom is using a sensory prosthesis to at least enhance sensory capabilities; processing the captured sensory input to identify instructions to one or more individuals in the interaction so that the person using the sensory prosthesis can have at least one of an enhanced sensation or an adequate sensation for future sensory input; and outputting the instructions to the one or more individuals.

[0013] According to another exemplary embodiment, a component includes: a device configured to receive input indicative of a captured sensory stimulation phenomenon and provide output regarding the captured sensory stimulation phenomenon, the output enhancing a person's future sensory input from future sensory stimulation phenomena.

[0014] According to another exemplary embodiment, a device comprises: a prosthesis, which is configured to operate in conjunction with a remote sensory induced phenomenon capture device that also includes an indicator, wherein the prosthesis is configured to provide input to the remote device related to the captured sensory stimulus induced phenomenon captured by the prosthesis and / or the remote device, so that the remote device provides an indication related to the phenomenon through the indicator.

[0015] According to another exemplary embodiment, a portable electronic device comprises: a visual indicator device; and a wireless communication device; wherein the portable electronic device is configured to display instructions in an interactive format for a person within the visual range of the visual indicator to take actions to enhance future sensory input of a recipient of a sensory prosthesis.

[0016] According to another exemplary embodiment, a method includes: enabling a hearing impaired person to participate in a conversation; using a first electronic device to capture at least a portion of the sound of the conversation at a certain point in time; using the first electronic device and / or a second electronic device to analyze the captured sound; and manually providing information related to the captured sound to one party to the conversation based on the analysis during the conversation to enhance aspects of the conversation at a subsequent point in time.

[0017] According to another exemplary embodiment, a method of managing a conversation includes: utilizing a portable electronic device; electronically analyzing sounds captured during a conversation; and based on the analysis, manually providing indicators to participants of the conversation regarding how the participants should speak to improve the conversation, wherein at least one participant of the conversation is using a hearing prosthesis to hear the sounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Embodiments are described below with reference to the accompanying drawings, in which:

[0019] Figure 1 is a perspective view of an exemplary hearing prosthesis to which at least some of the teachings detailed herein may be applied;

[0020] Figure 2A and Figure 2B An exemplary system including a hearing prosthesis and a remote device in the form of a portable handheld device is presented;

[0021] Figure 3 An exemplary system including a hearing prosthesis and a remote device in the form of a smartwatch is presented;

[0022] Figure 4 An exemplary functional arrangement detailing communication between a black box of a hearing prosthesis and a black box of a remote device is presented;

[0023] Figure 5 A functional representation of an exemplary system is presented;

[0024] Fig. 6A A functional representation of another exemplary system is presented; and

[0025] Figure 6B-13 An exemplary algorithm according to an exemplary method is presented. DETAILED DESCRIPTION

[0026] Figure 1 is a perspective view of a cochlear implant, referred to as cochlear implant 100, implanted into a recipient, to which some of the embodiments and / or variations thereof detailed herein are applicable. In some embodiments, cochlear implant 100 is part of system 10, which may include external components, as will be described in detail below. In addition, it is noted that the teachings detailed herein are also applicable to other types of hearing prostheses, such as, by way of example only and not limitation, bone conduction devices (transcutaneous, active transcutaneous, and / or passive transcutaneous), direct acoustic cochlear stimulators, middle ear implants, and conventional hearing aids, among others. In fact, it is noted that the teachings detailed herein are also applicable to so-called multimodal devices. In exemplary embodiments, these multimodal devices apply both electrical stimulation and acoustic stimulation to a recipient (sometimes referred to as electroacoustic stimulators). In exemplary embodiments, these multimodal devices evoke hearing perception via electrical hearing and bone conduction hearing. Thus, any disclosure herein regarding one of these types of hearing prostheses corresponds to disclosure regarding another of these types of hearing prostheses or any medical device for that matter, unless otherwise indicated or unless its disclosure is incompatible with a given device based on the current state of the art. Thus, in at least some embodiments, the teachings detailed herein are applicable to partially implantable and / or fully implantable medical devices that provide a wide range of therapeutic benefits to a recipient, patient, or other user, including hearing implants with implanted microphones, auditory brain stimulators, pacemakers, visual prostheses (e.g., bionic eyes), sensors, drug delivery systems, defibrillators, functional electrical stimulation devices, catheters, and the like.

[0027] In view of the above, it is to be understood that at least some of the embodiments and / or variations thereof described herein are directed to body-worn sensory supplemental medical devices (e.g., Figure 1The present invention also relates to a hearing prosthesis that supplements hearing even in cases where all natural hearing ability has been lost. It is noted that at least some exemplary embodiments of some sensory supplementation medical devices relate to devices such as conventional hearing aids that supplement hearing perception in cases where some natural hearing ability has been retained, and to visual prostheses (both applicable to recipients who retain some natural visual ability as well as recipients who do not retain any natural visual ability). Therefore, the teachings detailed herein are applicable to any type of sensory supplementation medical device, and the teachings detailed herein can be used in a practical manner with the device. In this regard, the phrase "sensory supplementation medical device" refers to any device used to provide sensation to a recipient, regardless of whether the applicable natural perception is only partially impaired or completely impaired.

[0028] The recipient has an outer ear 101, a middle ear 105, and an inner ear 107. The components of the outer ear 101, the middle ear 105, and the inner ear 107 are described below, and then the cochlear implant 100 is described.

[0029] In a fully functional ear, the outer ear 101 includes the auricle 110 and the ear canal 102. Sound pressure or sound waves 103 are collected by the auricle 110 and pass through the ear canal 102. A tympanic membrane 104 is disposed across the distal end of the ear canal 102, which vibrates in response to the sound waves 103. This vibration is coupled to the oval or elliptical window 112 through three bones of the middle ear 105, collectively referred to as the ossicles 106, and including the malleus 108, the incus 109, and the stapes 111. The bones 108, 109, and 111 of the middle ear 105 serve to filter and amplify the sound waves 103, thereby causing the oval window 112 to articulate or vibrate in response to the vibration of the tympanic membrane 104. This vibration sets up waves of fluid motion of the perilymph within the cochlea 140. This fluid motion, in turn, activates tiny hair cells (not shown) inside the cochlea 140. Activation of the hair cells causes appropriate nerve impulses to be generated and transmitted via spiral ganglion cells (not shown) and the auditory nerve 114 to the brain (also not shown) where they are perceived as sound.

[0030] As shown, cochlear implant 100 includes one or more components that are temporarily or permanently implanted in a recipient. Figure 1 1 shows a cochlear implant 100 having an external device 142 that is part of system 10 (together with cochlear implant 100) that is configured to provide power to the cochlear implant as described below, wherein the implanted cochlear implant includes a battery that is charged by power provided from external device 142.

[0031] exist Figure 1In the illustrative arrangement of , external device 142 may include a power source (not shown) disposed in behind-the-ear (BTE) unit 126. External device 142 also includes components of a transcutaneous energy transfer link referred to as an external energy transfer assembly. The transcutaneous energy transfer link is used to transfer power and / or data to cochlear implant 100. Various types of energy transfer, such as infrared (IR), electromagnetic, capacitive, and inductive transfer, may be used to transfer power and / or data from external device 142 to cochlear implant 100. In the embodiment of FIG. Figure 1 In the illustrative embodiment of the invention, the external energy transfer assembly includes an external coil 130 that forms part of an inductive radio frequency (RF) communication link. The external coil 130 is typically a wire antenna coil consisting of multiple turns of electrically insulated single or multiple strands of platinum or gold wire. The external device 142 also includes a magnet (not shown) positioned within the turns of the external coil 130. It should be understood that Figure 1 The external devices shown in are merely illustrative, and other external devices may be used with embodiments of the present invention.

[0032] The cochlear implant 100 includes an internal energy transfer assembly 132 that can be positioned in a recess of the temporal bone adjacent to the recipient's auricle 110. As described in detail below, the internal energy transfer assembly 132 is a component of a transcutaneous energy transfer link and receives power and / or data from an external device 142. In the illustrative embodiment, the energy transfer link includes an inductive RF link, and the internal energy transfer assembly 132 includes a primary internal coil 136. The internal coil 136 is typically a wire antenna coil composed of multiple turns of electrically insulated single or multiple strands of platinum or gold wire.

[0033] The cochlear implant 100 also includes a main implantable component 120 and an elongated electrode assembly 118. In some embodiments, the internal energy transfer assembly 132 and the main implantable component 120 are tightly sealed within a biocompatible housing. In some embodiments, the main implantable component 120 includes an implantable microphone assembly (not shown) and a sound processing unit (not shown) to convert sound signals received by the implantable microphone in the internal energy transfer assembly 132 into data signals. That is, in some alternative embodiments, the implantable microphone assembly can be located in a separate implantable component (e.g., which has its own housing assembly, etc.) that is in signal communication with the main implantable component 120 (e.g., via leads between the separate implantable component and the main implantable component 120, etc.). In at least some embodiments, the teachings and / or variations thereof detailed herein can be used with any type of implantable microphone arrangement.

[0034] The main implantable component 120 also includes a stimulator unit (also not shown) that generates electrical stimulation signals based on the data signals. The electrical stimulation signals are delivered to the recipient via the elongated electrode assembly 118.

[0035] The elongated electrode assembly 118 has a proximal end connected to the main implantable member 120 and a distal end implanted in the cochlea 140. The electrode assembly 118 extends from the main implantable member 120 through the mastoid bone 119 to the cochlea 140. In some embodiments, the electrode assembly 118 can be implanted at least in the basal region 116, and sometimes more deeply. For example, the electrode assembly 118 can extend toward the top of the cochlea 140 (referred to as the cochlear apex 134). In some cases, the electrode assembly 118 can be inserted into the cochlea 140 through the cochlear fenestration 122. In other cases, the cochlear fenestration can be formed by the round window 121, the oval window 112, the promontory 123, or through the apical gyrus 147 of the cochlea 140.

[0036] The electrode assembly 118 includes a longitudinally aligned and distally extending array 146 of electrodes 148 disposed along its length. As noted, the stimulator unit generates stimulation signals that are applied by the electrodes 148 to the cochlea 140, thereby stimulating the auditory nerve 114.

[0037] Figure 2A and Figure 2B An exemplary system 210 is depicted according to an exemplary embodiment and includes a hearing prosthesis 100 , which in the exemplary embodiment corresponds to the cochlear implant 100 described in detail above, and a portable handheld device 240 . Figure 2B The embodiment has a wireless link 230 with the hearing prosthesis 100, and Figure 2A The alternative embodiment depicted in FIG. 1 has no such link. In the exemplary embodiment, the hearing prosthesis 100 is an implant (e.g., a device functionally connected to a hearing aid) implanted in a recipient 99. Figure 2A / 2B). In an exemplary embodiment (as shown in the dashed box 100 in FIG. Figure 2B), the system 210 is configured such that the cochlear implant 100 and the portable handheld device 240 (e.g., a portable cellular telephone, such as, by way of example only and not limitation, a smartphone, as the term is generally used) have a relationship. By way of example only and not limitation, in an exemplary embodiment, the relationship is the ability of the smartphone to act as a remote device for the prosthesis 100 via the wireless link 230. Alternatively or in addition, the relationship is the transmission of only an audio signal stream captured by the smartphone's microphone to the hearing prosthesis so that the hearing prosthesis can evoke hearing perception based on the audio stream (as will be described in detail, other relationships exist). In some embodiments, the portable handheld device 240 merely extends the hearing prosthesis system, but is not a control device for the hearing prosthesis system. That is, in some embodiments, a different relationship exists between the two devices. Instead, for example, the two devices may be used simultaneously to achieve practical value, as will be described below. The two devices operate completely autonomously relative to each other, although in some such exemplary embodiments, one or both of the devices may be "aware" that one or both devices are being used simultaneously with the other device. Some additional details of this point will be described below. To be clear, in some embodiments, the remote device cannot be used to actively adjust the prosthesis 100, but this does not preclude the ability of the remote device to provide prompts to the recipient, which prompts may be of practical value to the recipient in adjusting the hearing prosthesis 100. In some embodiments, pairing between the devices occurs during operation of one or more or all of the devices, and this pairing is recognized by one or more or all of the devices.

[0038] It is noted that while the embodiments detailed herein will generally be described in terms of use with cochlear implants, alternative embodiments may be used with other types of hearing prostheses, such as, by way of example only and not limitation, bone conduction devices (transcutaneous, active transcutaneous, and / or passive transcutaneous), direct acoustic cochlear implants (DACIs), and conventional hearing aids. Thus, any disclosure herein with respect to one of these types of hearing prostheses corresponds to disclosure with respect to another of these types of hearing prostheses, or any other prosthetic medical device for that matter, unless otherwise indicated or unless its disclosure is incompatible with a given hearing prosthesis based on the current state of the art.

[0039] Figure 3 An exemplary system 211 is depicted according to an exemplary embodiment, comprising a hearing prosthesis 100 (which in the exemplary embodiment corresponds to the cochlear implant 100 detailed above) and a portable device 241 having an optional wireless link 230 with the hearing prosthesis 100, wherein the portable device 241 is a smart watch. In the exemplary embodiment, the hearing prosthesis 100 is an implant implanted in a recipient 99 (e.g., a device that is functionally connected to the hearing prosthesis 100). Figure 2A and Figure 2Bdenoted by the dashed box 100 in the figure). In an exemplary embodiment, the system 211 is configured so that the cochlear implant 100 and the portable device 241 in the smart watch embodiment can have a relationship. By way of example only and not limitation, in an exemplary embodiment, the relationship is the ability of the smart watch 241 to serve as a remote microphone for the prosthesis 100 and / or a control for the prosthesis via the wireless link 230. However, as was the case with the embodiments detailed above with respect to a smartphone, in some embodiments, no relationship exists. To be clear, any disclosure of features of a smartphone herein may correspond to disclosure of features of a smart watch, and / or vice versa, unless otherwise noted, assuming the technology enables it. Note also that while Figure 2A , Figure 2B and Figure 3 The embodiments are presented according to the remote device being a multi-purpose smart portable device, but in some embodiments, the remote device is a device dedicated to implementing the teachings detailed herein. It is also noted that, as will be described in detail below, in some embodiments, one or more or all of the aforementioned devices may be used simultaneously in a given system and / or as a substitute for another component of the system.

[0040] It is clear that in exemplary embodiments, the teachings detailed herein may be performed in whole or in part by a multi-purpose smart portable device configured to perform the teachings detailed herein. In some exemplary embodiments, there is a multi-purpose smart portable device, such as the one described above in Figure 2A , Figure 2B and Figure 3 Those described in, which include an interactive display screen, which may be Apple's commercially available as of July 4, 2017 TM (For example, iPhone 6 TM ) or Samsung (e.g. Galaxy S7 TM ) on a smartphone. In an exemplary embodiment, the multi-purpose smart portable device is a body-worn device, such as, by way of example only and not limitation, Figure 3An embodiment of the present invention is a smart watch, which includes a base. In some embodiments, this base can be a plastic and / or metal base that supports these exemplary components, such as: an LCD screen on which images (e.g., text, pictures, graphics, etc.) can be presented, wherein in some embodiments, the LCD screen can be a touch screen; one or more microphones (e.g., 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or more microphones); one or more speakers (e.g., 1, 2, 3, 4, 5 speakers) and / or one or more vibrators, including actuators and their weights (if used); a central processing unit (CPU), which can be a computer chip or computer processor, etc.; one or more printed circuit boards; lugs for attaching a strap; an RF transmitter, an RF receiver (e.g., a Wi-Fi and / or Bluetooth transmitter / receiver system), etc. Note that in at least some exemplary embodiments, the body-worn device 241 corresponds to an Apple Watch. TM Series 1 or Series 2, which are commercially available in the United States as of July 4, 2017. In an exemplary embodiment, the body-worn device 241 corresponds to a Samsung Galaxy Gear TM Gear 2, which is commercially available in the United States as of July 4, 2017. In exemplary embodiments, the aforementioned base carries one or more of all components available in the Samsung and / or Apple devices just detailed. Note that in at least some exemplary embodiments, the base is a single integral component, while in other embodiments, the base is an assembly of components that are integrated with respect to each other. Note that the body-worn device may include two or more bases. Note that in the case where the multi-purpose intelligent portable device is a body-worn device, the interactive display may correspond to the display of the aforementioned smart watch.

[0041] In at least some exemplary embodiments of this embodiment, the multi-purpose smart portable device also includes a wireless communication kit. In an exemplary embodiment, the wireless communication kit includes an FM / RF receiver and / or transmitter, or an IR receiver and / or transmitter, etc. In an exemplary embodiment, the wireless communication kit is based on Bluetooth technology and includes a Bluetooth-compatible device, such as a Bluetooth transceiver. Therefore, in some exemplary embodiments, the multi-purpose smart portable device is configured to perform signal communication (RF communication, but in some other embodiments, IR and / or wired communication) with the hearing prosthesis, and perform at least one of receiving signals from it or sending signals to it.

[0042] In at least some exemplary embodiments, the multi-purpose smart portable device also includes an operating system, which may include a processor and memory, as well as software and / or firmware and / or hardware to perform one or more of the teachings detailed herein.

[0043] In at least some exemplary embodiments, a multi-purpose smart portable device (which may or may not be a smart phone and thus may or may not have a portable cellular phone kit) is configured to analyze a signal input indicative of captured sound and provide an output regarding the captured sound. In an exemplary embodiment, the input is the captured sound, which may be captured by a microphone of the multi-purpose smart portable device or another microphone in wired communication with the multi-purpose smart portable device. In an exemplary embodiment, the input is a signal from a prosthesis based on ambient sound captured by the microphone of the prosthesis. By way of example only and not limitation, the microphone 126 of the BTE device may capture sound, and the BTE device may output a wireless signal via an RF transmitter / transceiver, which will be received by an RF receiver of the multi-purpose smart portable device, the signal corresponding to the signal input indicative of the captured sound. In conclusion to this, in exemplary embodiments, a component, including a device, configured to receive input indicative of a captured sensory stimulation phenomenon and provide output regarding the captured sensory stimulation phenomenon that enhances a person's future sensory input from future sensory stimulation phenomena, corresponds to a smartphone, a personal computer having a computer-readable medium programmed to perform the teachings detailed herein, a dedicated consumer electronic product, and the like. In addition, some embodiments of this component include a microphone component that wirelessly communicates with a hearing prosthesis / is configured to communicate with the hearing prosthesis. Indeed, in exemplary embodiments, the component is modified or otherwise has such features as those described herein with respect to a device remote from a hearing prosthesis (e.g., by way of example only and not limitation, Figure 5 or Fig. 6A The microphone may be a device configured to receive input indicative of the captured sensory stimulation phenomenon, and in other embodiments may be a dedicated remote microphone in combination with a wireless receiver / transceiver that receives a signal from the hearing prosthesis indicative of the captured sensory stimulation phenomenon. The assembly may include a processor that analyzes the input, whether from the microphone or from the prosthesis (or in some embodiments, from both), and forms an output regarding the captured sensory stimulation phenomenon that, based on the analysis, enhances the person's future sensory input from future sensory stimulation phenomena.

[0044] In an exemplary embodiment, a multi-purpose smart portable device includes the aforementioned device and system, and is additionally configured to perform the methods detailed herein in order to analyze a signal input indicative of captured sound.

[0045] In some cases, the phrase "detrimentally affects the hearing perception of another sound" or variations thereof will be used herein. This means that the sound makes it difficult to actually hear the other sound. This is in contrast to the general sound present in the environment. For example, wind noise may cause a detrimental effect on another sound, but only in some circumstances, e.g., depending on how fast the wind is blowing. For example, a breeze of one kilometer per hour may have no detrimental effect on another sound, while a breeze of 22.7 kilometers per hour will almost certainly cause a detrimental effect on the other sound.

[0046] In short, as will be seen below, in some embodiments, the multi-purpose smart portable device is configured to communicate signals with a hearing prosthesis (e.g., the prosthesis 100 described in detail above), and the output regarding the captured sound is a signal to the hearing prosthesis to adjust its settings, to notify the recipient to adjust its settings, or to notify the recipient of at least one of a feature of the surrounding environment in which the device is located. In this regard, in an exemplary embodiment, a party to a conversation (the context of this conversation may be described below) may input a control command into the multi-purpose smart portable device, which will adjust the settings of the hearing prosthesis, such as adjusting the volume or gain, etc., and / or based on the analysis, the multi-purpose smart portable device may be configured to automatically output the signal, thereby adjusting the settings of the prosthesis. Further, the multi-purpose device may be configured to notify the recipient to adjust its settings, wherein the recipient may manually adjust the hearing prosthesis through its input kit, such as by pressing a button or turning a knob on the BTE device. Still further, the device may be configured to inform the recipient of characteristics of the surrounding environment in which the device is located, such as indicating to the recipient that the surrounding environment contains background noise that is detrimental to sounds of interest (e.g., the voice of a person conversing with the recipient).

[0047] In exemplary embodiments where the device is configured to control a hearing prosthesis, in at least some exemplary embodiments, the device is configured to display control settings on a display screen to control the hearing prosthesis based on the analysis of the signal so that the recipient can adjust the hearing prosthesis based on the output. By way of example only and not limitation, in exemplary embodiments where the analysis determines that there is too much background noise in the received signal, the hearing prosthesis automatically presents controls for beamforming on the display screen. The recipient can use their finger to input the desired control input to adjust the beamforming so that the microphone of the hearing prosthesis (e.g., the microphone on the BTE device) beamforms / exits omnidirectional service to the speaker. In exemplary embodiments, by way of example only and not limitation, if the analysis determines that the speaker is not speaking loud enough, in exemplary embodiments, the multi-purpose smart portable device can automatically bring up a volume control and display it on the screen so that the recipient can increase the volume of the prosthesis. Note also that in exemplary embodiments of this scenario, the volume control that can appear can be a volume control limited to certain frequencies, such as frequencies associated with speech.

[0048] It is briefly noted that in some exemplary embodiments, the multi-purpose smart portable device may utilize a learning algorithm that may learn over time which types of sounds or types of voices or which actions or otherwise which scenarios have a detrimental effect on other sounds. In some embodiments, this is performed using a machine learning algorithm that may be executed using a remote processor that may be periodically accessed via the Internet to update the algorithm and the smart portable device, while in other embodiments, this may be performed by a simple input from a recipient that indicates to the multi-purpose smart portable device that a given scenario should be ignored or otherwise not taken seriously. In the latter case, since there are many cases in which a particular scenario causes the device to produce an output, and then the recipient subsequently inputs that the scenario should be ignored or otherwise not taken any action, the multi-purpose smart portable device will learn that this scenario should not result in an indication to one or more speakers of the conversation. Conversely, the reverse is also true: in an exemplary embodiment, the multi-purpose smart portable device may learn that a given scenario is a scenario that should result in the device providing an indication, whereas prior to learning, no indication was provided when the given scenario occurred.

[0049] It is briefly noted that in some embodiments, a smartwatch represents any body-worn device that may have utility in whole or in part with respect to a smartwatch. Thus, in exemplary embodiments, any smartwatch disclosed herein corresponds to disclosure of another type of body-worn device, such as a pendant on the neck, a ring configured to be worn on a person's finger, etc. In an exemplary embodiment, the base of the smartwatch is mounted in a neck chain, wherein there is no wristband. In an embodiment, the base is a modified base to be tactilely and / or visually consistent with a pendant worn on the neck, etc.

[0050] In an exemplary embodiment, the system 210 / 211 is configured to enable the portable electronic device to replicate the functionality of a given input device of the hearing prosthesis 100 (e.g., the input device is a button (such as a push button, a knob), a heat sensitive pad, etc., alone or in combination with another input device (such as an LED) at the portable electronic device. By way of example only and not limitation, the input device may be a knob on the BTE device 246 that is adjusted by the recipient to increase or decrease the perceived volume of the resulting hearing perception evoked by the hearing prosthesis 100. The function of this knob is thus to control or at least adjust the perceived volume perceived by the recipient or the volume associated with the output of the prosthesis (in some embodiments, any volume phenomenon that is quantifiable and / or limitable with respect to the device and / or the recipient may be adjusting the volume). In an exemplary embodiment, the hearing prosthesis is configured to enable the portable electronic device to have this functionality. Further, in an exemplary embodiment, functionality such as those functions that cause the processor of the hearing prosthesis to turn on and off, change the mapping of the hearing prosthesis, and / or start and / or stop streaming are present in the portable electronic device. That is, in some embodiments, the system 210 / 211 is not configured to enable the portable electronic device to replicate the functionality of a given input device of the hearing prosthesis 100 , as described above.

[0051] In an exemplary embodiment, the hearing prosthesis 100 captures sound through, for example, a microphone on the BTE 126 or off-the-ear sound processor, or through a remote microphone in signal communication therewith, and transmits data to a remote device 240 in this exemplary embodiment, which data may correspond to the raw output signal of the microphone via link 230. The off-the-ear (OTE) sound processor is held against the recipient's skin between the 9 o'clock position and the 12 o'clock position from the ear canal (e.g., at approximately the 10, 10:30, or 11 o'clock position in persons over the age of 10, at a distance greater than 2 inches and less than 5 inches therefrom for at least 50 percent of such persons). This is accomplished by Figure 4 Functionally shown, Figure 4 The hearing prosthesis 100 and remote device 240 / 241 are depicted in a black box format, where input 3144 corresponds to input to a microphone of the prosthesis 100 .

[0052] The remote device 240 / 241 receives the signal via link 230 (if present) and processes the data in a useful manner, some of the details of which will be described below. However, in brief, in an exemplary embodiment, the remote device processes the data to evaluate the sound captured by the microphone of the prosthesis 100 and automatically determines whether changes should be made to the prosthesis or to more general features that can enhance the hearing perception delivered to the recipient. In some embodiments, the remote device is configured to indicate which changes should be made. In an exemplary embodiment, such changes are made, and the hearing prosthesis 100 thus evokes hearing perception through output 3246 to the recipient's tissue in a more useful manner than if such changes were not made (wherein the output 3246 corresponds to electrical stimulation in the case of the hearing prosthesis 100 being a cochlear implant, and the output 3246 corresponds to vibration in the case of a bone conduction device, etc.).

[0053] In view of the above, in an exemplary embodiment, there is a system, such as system 210 / 211. However, in an exemplary embodiment, the system may also be limited to one or the other of prosthesis 100 or remote device 240 / 241.

[0054] Figure 5 Box 542 is depicted in a black box format and may correspond to the prosthesis 100 or the remote device 240 / 241. That is, box 542 may also functionally represent two components, bifurcated when appropriate. In an exemplary embodiment, the system includes a sound signal input kit, which, by way of example only and not limitation, may include a microphone and associated circuitry of the prosthesis 100 in some embodiments, which, by way of example only and not limitation, in an exemplary embodiment, may include an amplifier, etc., and in some cases, an analog and / or digital signal processor. Moreover, in some embodiments, the sound signal input may correspond to input from the microphone 126 on the BTE and / or the microphone of the remote device 240 / 241. In Figure 5, the sound input kit is represented in a black box format by element 543, where arrow 544 functionally represents the sound input through the surrounding environment to the sound input kit 543. In an exemplary embodiment, the sound input kit 543 includes a microphone. That is, in an embodiment where the sensory stimulus induced phenomenon is a phenomenon that induces visual perception (for example, the phenomenon is light), element 543 is a light input kit including a camera. It is to be clear that any disclosure of microphones, etc. herein corresponds to disclosure of alternative embodiments in which the sensory stimulus induced phenomenon capture device is another type of device (for example, a camera) or any other phenomenon capture device (for example, a device that captures fragrance, etc.). It is also noted that any disclosure of hearing prostheses herein corresponds to disclosure of alternative embodiments in which the prosthesis is another type of sensory prosthesis (for example, a visual prosthesis, a tactile induction prosthesis, an odor induction prosthesis, etc.). Therefore, in some exemplary embodiments, with respect to arrow 544, arrow 544 represents any phenomenon that induces sensory stimulation in a recipient. Furthermore, while the teachings detailed herein are generally focused on conversations and sounds, it is to be understood that any such disclosure herein corresponds to disclosure in alternative embodiments in which the conversation is instead a gathering of people that are only visible to a recipient aided by a visual prosthesis, and the teachings detailed herein are used to ultimately enhance the visual perception of the recipient of the visual prosthesis. Thus, the downstream device(s) of device 543 are configured to evaluate the input (regardless of its relevant form), or otherwise act in conjunction with the input, and provide output that attends the use of the teachings detailed herein to enhance the visual experience.

[0055] The system may also include a processor, which is Figure 5 The processor 545 in FIG. 5 is functionally shown. In an exemplary embodiment, the processor 545 is configured to: analyze the output of the sound signal input kit, the output of which is based on the signal received by the sound signal input kit 543, the directional signal emitted from the box 543, Figure 5 and outputting a signal indicating an instruction related to data about a recipient of a hearing prosthesis, represented by arrow 548 in the middle right; and outputting a signal indicating an instruction related to data about a recipient of a hearing prosthesis, represented by arrow 546 (instructions related to data about a recipient of a hearing prosthesis are described in more detail below). In an exemplary embodiment, output 546 is provided to either or both of the prosthesis and / or the remote device. In an exemplary embodiment, as Fig. 6AAs seen in FIG. 5 , one of the devices also includes an output kit 549 that outputs a signal 541. In an exemplary embodiment, the output kit 549 is a display on which text indicating instructions is presented. In an exemplary embodiment, the output kit 549 is an LED associated with a permanent text corresponding to the text of the instructions, wherein the illumination or alternatively the cancellation of the LED draws the attention of a person in a visible location of the particular device, who can read the particular text associated with the LED. That is, in an exemplary embodiment, the LED is simply an LED, the meaning of which is known in advance to the recipient. In an exemplary embodiment, the LED can change color from, for example, red to blue to green, with each different color representing a different instruction.

[0056] It should be noted that in at least some exemplary embodiments, the aforementioned display or the aforementioned indicator may be co-located with a remote microphone or any other remote sensory stimulation induced image capturing device (e.g., a remote camera for a visual prosthesis, etc.). For example, in at least some exemplary embodiments, a remote microphone may be provided for the aforementioned LED. In this way, a combined device is provided that provides remote sensory stimulation induced image capturing as well as the indications detailed herein. Any disclosure herein regarding an apparatus or system for providing information to one or more parties to a conversation also corresponds to the disclosure of an apparatus combined with an apparatus for capturing sensory stimulation induced images, and vice versa, unless otherwise indicated or otherwise not possible in the art.

[0057] Briefly note that in at least some exemplary embodiments, arrow 548 may represent a link between the hearing prosthesis and the remote portable device. Figure 5 The schematic diagram of is presented in functional terms to correspond to Figure 5 All the descriptions in Figure 2A and Figure 2B The case of only one of the two devices and Figure 5 An embodiment in which the associated components are distributed between the two devices.

[0058] In an exemplary embodiment, the processor 545 can be a standard microprocessor supported by software or firmware, etc., which is programmed to evaluate the signals received from the sound input kit 543. By way of example only and not limitation, in an exemplary embodiment, the microprocessor can access a lookup table, etc., for example, the lookup table has data associated with the spectral analysis of a given sound signal, and can compare the characteristics of the input signal and compare these characteristics with the characteristics in the lookup table, and make a determination about the input signal and therefore make a determination related to the sound 544 through the relevant data associated with these characteristics in the lookup table. In an exemplary embodiment, the processor is the processor of a sound analyzer. The sound analyzer can be based on FFT or on another operating principle. The sound analyzer can be a standard sound analyzer available on a smartphone, etc. The sound analyzer can be a standard audio analyzer. The processor can be part of a sound wave analyzer. In addition, it should be specifically pointed out that although Figure 5 The embodiment presents the processor 545 as one of the devices of the hearing prosthesis or part of the portable electronic device, but it should be noted that in some exemplary embodiments, the processor may be remote from both devices. By way of example only and not limitation, in an exemplary embodiment, one or both of the devices of the system 210 and / or 211 may communicate signals via Bluetooth technology or other RF signal communication systems with a remote server, which is linked to a remote processor via, for example, the Internet. The signal 548 is provided to this remote processor via the Internet, where the signal is analyzed, and then a signal indicating instructions related to data about the recipient of the hearing prosthesis may be provided to the device in question via the Internet so that the device may output the signal. It is also noted that in exemplary embodiments, the information received from the remote processor may be merely the result of the analysis, wherein the processor may analyze the result of this analysis and identify instructions and output this instruction. It is noted that the term "processor" as used herein may correspond to multiple processes linked together as well as a single processor.

[0059] In view of the above, it can be seen that in an exemplary embodiment, there is a system comprising a signal input, a processor, and a signal output. In this exemplary embodiment, the processor is configured to generate instructions related to data about a recipient of a sensory prosthesis based on the input of the signal input, and the signal output is configured to output data indicating the instructions. The data indicating the instructions can be data indicating any one of the instructions detailed herein. This embodiment can be modified or expanded or otherwise changed to include any teaching and / or variations thereof detailed herein. In some of these embodiments, the processor is configured to analyze data related to the recipient and determine whether a person speaking provides sensory input in a manner that enhances the sensory perception of the recipient of the sensory prosthesis. In some of these embodiments, the system is configured to provide the instructions to a person other than the recipient of the sensory prosthesis, who is part of a group of people providing sensory input captured by the system.

[0060] Also consider the above and consider Figure 4 It can be seen that in an exemplary embodiment, a device includes a prosthesis (e.g., 100) configured to operate with a remote sensory induced phenomenon capture device (e.g., 240 / 241) that also includes an indicator, wherein the prosthesis is configured to provide input related to the captured sensory stimulus induced phenomenon captured by the prosthesis and / or the remote device to the remote device, so that the remote device provides an indication related to the phenomenon through the indicator. As will be described in detail below, in some embodiments where the prosthesis is a hearing prosthesis, the remote device is a remote microphone device used with the hearing prosthesis, as opposed to a smartphone or smart watch. The remote microphone has an extended capability to perform the functional features just mentioned. In some embodiments, any arrangement that enables an indicator to enhance future sensory input from future sensory stimulus phenomena can be used, the indicator being configured to provide an indication of the captured sensory stimulus phenomenon, whether the arrangement is in the form of a remote microphone, a smartphone, or a dedicated consumer electronic product having this function (in some embodiments, only one or more or all of the functions herein, and no more functions), etc. Thus, some embodiments include a device interfaced with a prosthesis that is configured to analyze the captured phenomenon and form input to a remote device (e.g., via an onboard processor of the prosthesis), wherein the input is a command indicator indicating that one or more persons within the line of sight of the indicator should take action that affects future sensory input from the future sensory stimulation phenomenon.

[0061] In an exemplary embodiment, the system includes a speech analyzer, such as, by way of example only and not limitation, a speech analyzer configured to perform spectral measurements and / or spectral analysis measurements and / or duration measurements and / or fundamental frequency measurements. By way of example only and not limitation, this may correspond to a processor of a computer configured to execute SIL Language Technology Speech Analyzer TM (SIL Language Technologies Speech Analyzer TM ) program. In this regard, the program can be loaded onto the memory of the system, and the processor can be configured to access the program to analyze or otherwise evaluate speech. In an alternative embodiment, a speech analyzer can be obtained from Rose Medical, and the programming can be loaded into the memory of the system.

[0062] In an exemplary embodiment, the system includes an audio analyzer that can analyze one or more of the following parameters: harmonics, noise, gain, level, intermodulation distortion, frequency response, relative phase of the signal, etc. It should be noted that the above-mentioned sound analyzer and / or speech analyzer can also analyze one or more of the aforementioned parameters. In some embodiments, the audio analyzer is configured to form time domain information to identify instantaneous amplitudes that vary over time. In some embodiments, the audio analyzer is configured to measure intermodulation distortion and / or phase. In an exemplary embodiment, the audio analyzer is configured to measure signal-to-noise ratio and / or total harmonic distortion plus noise.

[0063] To be clear, in some exemplary embodiments, the processor is configured to access software, firmware and / or hardware that is "programmed" or otherwise configured to perform one or more of the foregoing analyses. By way of example only and not limitation, the system may include hardware in the form of a circuit configured to enable the analysis detailed above and / or below, the output of which is received by the processor so that the processor can utilize the output to perform the teachings detailed herein. In some embodiments, the processor utilizes analog circuitry and / or digital signal processing and / or FFT. In an exemplary embodiment, the analyzer engine is configured to provide a high-precision implementation of the values ​​(peak and RMS) of the AC / DC voltmeter, the analyzer engine includes a high-pass and / or low-pass and / or weighting filter, the analyzer engine may include a bandpass and / or notch filter and / or a frequency counter, all of which are arranged to perform analysis on an incoming signal in order to evaluate the signal and identify certain characteristics thereof, which are associated with a predetermined scenario or another predetermined instruction and / or predetermined indication, which will be described in more detail below. Note also that in a digital based system, the system is configured to implement the signal analysis using FFT based calculations, and in this regard, the processor is configured to perform the FFT based calculations.

[0064] Note that the above instructions are instructions from the machine telling the person to perform certain actions, as opposed to mere suggestions, which are instructions (which are a type of instruction).

[0065] It is noted that while the above embodiments have been described in terms of a black box 542 corresponding to one or both or all of a prosthesis, a smartphone, or a smartwatch, it is noted that the black box 542 may correspond to another type of device, such as, by way of example only and not limitation, a device that is limited to and dedicated solely to performing the methods detailed herein and / or otherwise implementing the functions detailed herein and does not implement other functions. By way of example only and not limitation, the box 542 may be a metal or plastic box that supports a microphone and / or an RF receiver and / or a line-in jack (e.g., which may be hooked to a microphone), includes the aforementioned processor, and includes an output kit configured to output the aforementioned instructions, which output may correspond to a speaker, may correspond to an LCD, LED with permanent text associated therewith, may correspond to an output jack that may be hooked to a speaker or television, etc.

[0066] The above system may have utility for providing instructions to one or more members of a conversation, where someone who is part of the conversation may speak in different ways to enhance the hearing perception of a person wearing a hearing prosthesis. In this regard, the aforementioned "instructions relating to data about a recipient of a hearing prosthesis" enables the data relating to the recipient of the hearing prosthesis to be the degree to which the recipient is able to hear sounds with the prosthesis, and thus, the instructions may be instructions to act in a specific way for a party to the conversation to improve hearing, and are therefore instructions relating to data.

[0067] In an exemplary embodiment, the processor is configured to analyze the signal received from the sound signal input kit and determine whether the person speaking is speaking in a manner that enhances the hearing perception of the recipient of the hearing prosthesis. By way of example only and not limitation, in an exemplary embodiment, the processor may evaluate the volume (amplitude) of the output associated with the time measurement and infer from this that the speaker often speaks in a manner that is not as practical as it would otherwise be. Therefore, based on this evaluation, the processor may determine whether the speaker is speaking toward the microphone, and therefore determine whether the speaker is speaking in a manner that enhances the hearing perception of the recipient with the hearing prosthesis. Note that determining whether the speaker is speaking in a manner that enhances the hearing perception includes determining that the speaker is speaking in this manner, determining that the speaker is not speaking in this manner, and / or determining both. That is, determining whether the speaker is speaking toward the microphone includes determining whether the speaker is speaking toward the microphone, determining that the speaker is not speaking toward the microphone, and / or determining both.

[0068] It may be determined whether a speaker is speaking in a manner that enhances hearing perception when there is a baseline between when the speaker is speaking so and when the processor determines that the speaker is not speaking so. It may be determined whether a speaker is speaking in a manner that enhances hearing perception when there is a baseline between when the speaker is not speaking so and when the processor determines that the speaker is speaking so. Consistent with the teachings in the previous paragraphs, determining whether a speaker is speaking in a manner that enhances hearing perception may be done in two ways.

[0069] In an exemplary embodiment, the system is configured such that if the speaker is speaking in a manner that enhances the hearing perception of the recipient, the second signal may or may not be output. By way of example only and not limitation, if the baseline of the system is that the system only provides instructions when the speaker is speaking in a manner that enhances the hearing perception, this is the only time the second signal will be output, which may cause the system to output instructions to the speaker to continue speaking in the manner in which it has been speaking (e.g., the green LED will be illuminated). In such a system with such a baseline, if the speaker is not speaking in a manner that enhances the hearing perception (e.g., the green LED will not be illuminated), then the second signal will not be output. Conversely, if the baseline of the system is that the system only provides instructions when the speaker is not speaking in a manner that enhances the hearing perception (this includes scenarios where the speaker can speak in a manner different from the manner in which he is speaking, and the different manner enhances the hearing perception relative to the manner in which the speaker is currently speaking), then the second signal will be output, and this may cause the system to output instructions to the speaker to speak in a manner different from the manner in which he is speaking (e.g., the red LED will be illuminated). In such a system with such a baseline, if the speaker speaks in a manner that enhances hearing perception (e.g., the red LED will not be illuminated), then the second signal will not be output. That is, in an exemplary embodiment where there is no real baseline but the system outputs instructions in two scenarios, the second signal may be output, causing the system to provide instructions to keep speaking in the same manner (e.g., the green LED is illuminated), and then the second signal will be output, causing the system to provide instructions to the speaker to speak in a different manner (e.g., the red LED is illuminated).

[0070] Such scenarios in which the processor determines, based on analysis of the first signal, whether a speaker is speaking in a manner that enhances the recipient's hearing perception may include using relative analysis, such as analyzing whether the speaker can and / or cannot speak louder or softer, slower or faster, deeper (Darth Vader) or less deeply, etc.

[0071] Note that in some embodiments, the system described above may have practical value for providing one or more members of a conversation with the following indications: too many speakers are speaking at the same time and / or too many speakers are speaking closely together, with the speakers speaking in close temporal proximity to each other. This may correspond to a scenario where it is more difficult for the recipient to manage or otherwise understand the meaning of the sound captured by the prosthesis than when one or more speakers are not speaking at the same time as one or more other speakers and / or one or more speakers are not speaking in close temporal proximity to one or more other speakers. In an exemplary embodiment, the system may be configured to provide an output indicating an indication that too many people are speaking in close temporal proximity to each other. The indication may be a general indication of this situation, and / or may be an indication that one or more specific speakers in a group of speakers are causing a "problem". In an exemplary embodiment, the system may output instructions to a group of speakers or one or more individual speakers, the one or more individual speakers having a total less than all individual speakers, to avoid speaking in close temporal proximity to other speakers.

[0072] Note that in an exemplary embodiment, the above identification of whether the person speaking is a person with a hearing prosthesis may be performed using a processor program to receive the various data detailed above and analyze the data to make the identification. The identification may be performed using an if-then-else algorithm.

[0073] In an exemplary embodiment, the system is configured to provide instructions to a recipient of a hearing prosthesis. By way of example only and not limitation, the system may provide instructions to the recipient to speak louder or softer, slower or faster, deeper or less deep, etc. The instructions may be text instructions to the recipient, for example, text presented on the system's LCD, the instructions may be symbol-based (up arrow indicates to speak louder, down arrow indicates to speak softer, left arrow indicates slower, right arrow indicates faster, etc.), the instructions may be light / text related, or may simply be like-related, where the recipient understands what certain lights mean before using the system. By way of example only and not limitation, an instruction manual may be provided to the system, where the recipient reads the instruction manual and memorizes what three or four or five or six different LED colors and / or their positions mean, etc., and so when a light color and / or light position is illuminated, this will have meaning to the recipient. The recipient may be the only person who understands the output of the system, which is an exemplary embodiment where the system is configured to provide instructions to the recipient of a hearing prosthesis.

[0074] Alternatively and / or in addition, the system is configured to provide instructions to a person who is part of the conversation other than the recipient of the hearing prosthesis. In this regard, any or all of the aforementioned methods above can be used to provide instructions. Note that in some exemplary embodiments, the system is configured to provide instructions only to non-recipients, while in other embodiments, the system is configured to provide instructions only to recipients, and in other embodiments, the system is configured to provide instructions to both recipients and non-recipients. Because some embodiments of the system detailed herein are configured to provide instructions only to non-recipients, the system can be configured to identify whether the person speaking is a person without a hearing prosthesis. In an exemplary embodiment, this can be accomplished by one or more of the aforementioned methods detailed above regarding the embodiment in which the system is configured to identify whether the person speaking is a person with a hearing prosthesis. Note that in at least some exemplary embodiments, the system can identify both whether the person speaking is a person with a hearing prosthesis and whether the person speaking is a person without a hearing prosthesis.

[0075] It has practical value for determining whether the speaker is a speaker with hearing prosthesis. In an exemplary embodiment, the system can be configured to form instructions based only on the voice of the hearing prosthesis, and the instructions can be explicitly directed only to the recipient. As an example only and not as a limitation, in an exemplary embodiment, the system can be configured as a discrete system that only provides instructions to the recipient, and the instructions are provided in a manner that is transparent to other speakers in the conversation or unobservable / unobservable / imperceptible to the speaker in the conversation of the recipient. In this regard, in an exemplary embodiment, the system can be configured so that the system is configured to output tactile and / or auditory instructions to the recipient, and the tactile and / or auditory instructions may only be noticed by other speakers or may be difficult to be noticed by other speakers. As an example only and not as a limitation, for example, in the case where the system includes a BTE device, the BTE device may include a vibration device that vibrates, and the vibration is transmitted to the skin of the recipient, thereby providing the recipient with instructions that the recipient should speak differently and do things differently.

[0076] Note also that this haptic system can be implemented in a smartphone. By way of example only and not limitation, in an exemplary usage scenario, a recipient holds a smartphone that includes Figure 5 One or more of the components of the phone may be touched and the smartphone vibrator may vibrate, and because the recipient is holding the smartphone, the recipient will sense these vibrations, while perhaps no one else will.

[0077] In an exemplary embodiment, for example, where the system is configured to induce auditory instructions, the prosthesis may be configured to automatically induce hearing perception in a recipient indicating instructions, such as to speak louder or to speak slower, etc. Depending on the type of hearing prosthesis, such instructions may not be heard by other recipients, such as, by way of example only and not limitation, where hearing perception is induced using a cochlear implant.

[0078] Still with respect to embodiments where the system is configured to determine who is the speaker, in exemplary embodiments where the system is configured to determine that the speaker is a speaker other than the recipient of the hearing prosthesis, the system may be configured to provide instructions based only on the speech of the non-recipient. In exemplary embodiments, the system may be implemented in a BTE device having an ear hook extending around the front of the recipient's ear. In exemplary embodiments, the BTE device may include one or more LEDs, which in some embodiments may change to different colors. In exemplary embodiments, during a conversation, the LEDs may be illuminated in different colors and / or different LEDs may be illuminated to provide instructions to the non-recipient speaker. Because the speaker typically looks at the other person's face while speaking, the speaker is able to see the recipient's ear hook and, therefore, is able to see these visual cues in a relatively non-distracting manner. In this regard, in exemplary embodiments, the recipient may not be aware that the prosthesis is providing instructions to the speaker, such as because the LED is not within the recipient's field of view. This may have practical value in terms of avoiding the recipient's self-awareness. In an exemplary embodiment, at the beginning of a conversation, the recipient can explain to the non-recipient speaker what the various indicators mean, so the non-recipient speaker can pick up on these clues during the conversation. This can have practical value in situations where people frequently speak to the same people, which occurs in many situations in the post-industrial world. In this regard, at least some people will learn the meaning of the indicators over time, at least based on repetition, and will be able to better converse with the recipient.

[0079] As described above, in some embodiments, the system can be configured to analyze the received signal to form data related to microphone placement. In some embodiments, the microphone being used to capture sound is a microphone on a hearing prosthesis, such as microphone 126 on a BTE device. In some embodiments, the microphone being used to capture sound is a remote microphone. In some cases, the microphone can be a microphone of a smartphone. In other cases, the microphone is a dedicated remote microphone that communicates wirelessly (e.g., RF wireless communication) with the hearing prosthesis. In some cases, the system is configured so that the sound processor can rely on inputs from multiple microphones, such as any two or more microphones described in detail herein. In an exemplary embodiment, the sound processor of the system can compare sound inputs from multiple different sources at the same time, and based on determining that one signal has more practical value than other signals, the former and / or two signals are used to create a mixed signal with the best features from the signal (e.g., one signal is used for certain frequencies and other signals are used for other frequencies). Two processors may rely on comparisons between sound inputs from multiple different sources at the same time. In an exemplary embodiment, the system can be configured such that the processor processes algorithms based on statistical data related to microphone placement, and the processor can utilize these algorithms to determine, based on the received signals, that the microphone can be oriented or otherwise placed in a different position to improve the recipient's hearing perception. In some cases, it is determined that the recipient should place the remote microphone closer to the speaker and / or should rotate the microphone toward the speaker and / or away from the recipient.

[0080] In view of the above, in an exemplary embodiment, the system is configured to analyze the received signal to form data related to at least microphone placement, and the output signal causes the system to provide an indication of the microphone. The indication may correspond to any indication detailed above modified to indicate instructions related to microphone placement and / or microphone orientation. Different colored LEDs may be used, and arrows may be used to indicate the action to be taken / the action that should be taken. In some embodiments, the prosthesis is used to provide audio signals to the recipient. In at least some exemplary embodiments, these instructions are presented on the screen of a smartphone and / or a smart watch. With respect to some such embodiments, consistent with the above, it can be seen that the system may include a hearing prosthesis and a smart device, the smart device including an interactive display screen away from the hearing prosthesis. The system may be configured to display instructions thereon. That is, in some embodiments, the system is also configured to display one or more controls of the hearing prosthesis on the interactive display screen. Although some embodiments do not include such functionality, in some other embodiments, the smart device is configured to enable the recipient to input controls therein to control the hearing prosthesis. By way of example only and not limitation, in an exemplary embodiment, the touch screen of the smartphone may include graphics for volume control, gain control, noise cancellation system control, and the like.

[0081] In view of the above, it can be seen that in an exemplary embodiment, a portable electronic device, such as a smartphone, smart watch or extended remote microphone device, or a dedicated consumer electronic device, includes a visual indicator device and a wireless communication device, wherein the portable electronic device is configured to display instructions in an interactive format, the instructions being directed to a person within the visual range of the visual indicator to take actions to enhance future sensory input of a recipient of the sensory prosthesis. Such interactive format can be the result of the recipient providing input from their prosthesis and / or through their smartphone or smart watch, etc., or any other means capable of implementing the teachings herein. Such interactive format can be the result of another member of the conversation (e.g., through their smartphone / watch, etc.) providing such input.

[0082] Note also that in at least some example embodiments, an indication may be provided by the system that the recipient should use one or more additional microphones or otherwise not use one or more microphones currently in use in order to enhance the recipient's hearing perception.

[0083] In view of the above, it is to be understood that in at least some exemplary embodiments, the system is configured to provide instructions to the recipient on how to point the microphone / in which way the microphone should be pointed and / or where to place the microphone, all based on the analysis. This may have practical value for such a situation because the system will surely tell the recipient what to do. Likewise, in some embodiments, the system can be configured to automatically evoke hearing perception that cannot be heard by anyone other than the recipient providing such instructions.

[0084] Also note that in some embodiments, instructions can be directed to the recipient to adjust the sensitivity of a given microphone and / or which microphone should be used instead of another. For example, in an exemplary embodiment, the system can determine based on the analysis that a remote microphone does not provide as much practical value for a microphone that is part of a prosthesis, such as a microphone on a BTE device. The system can instruct the recipient which microphone to use. In addition, in at least some exemplary embodiments, the system can make adjustments automatically and then prompt the recipient or user or other party associated with or otherwise affected by the adjustments to confirm these changes. That is, in an embodiment, the system may prompt a "warning" that it is planning to perform changes or additional adjustments to the system unless an override is provided.

[0085] In view of the above, it can be appreciated that at least some exemplary embodiments include methods. To this end, Figure 6BAn exemplary flow chart of a method 600 according to an exemplary embodiment is presented. The method 600 includes a method action 610, which includes capturing sound during a conversation between two or more people, one of whom is using a hearing prosthesis to at least enhance hearing. By way of example only and not limitation, the hearing prosthesis may be a cochlear implant, or a middle ear implant or a bone conduction implant device, or in some embodiments may be a conventional acoustic hearing aid. The method 600 also includes a method action 620, which includes processing the captured sound to identify instructions to the participants in the conversation so that the person using the hearing prosthesis can hear better. Such actions may correspond to any of the actions detailed above or any other actions that can enable the person using the hearing prosthesis to hear better. These instructions may be pre-programmed into the processor or the system detailed above or any other device. The instructions may be present on a lookup table stored in a memory of the system, wherein the aforementioned processor is configured to access this lookup table. In an exemplary embodiment, the processor compares one or more features of the captured sound with one or more data points in the lookup table, and if there is a correlation between the two, selects the indicator of this data point. The method 600 also includes a method action 630, which includes outputting an indication to one or more of the two or more individuals. The action of outputting an indication to one or more of the two or more individuals may correspond to any output detailed above or any other manner that may have practical value for implementing the teachings detailed herein.

[0086] Consistent with the teachings detailed above, method action 630 may be performed such that the instructions are instructions to a conversation participant other than the person using the hearing prosthesis. Still consistent with the teachings detailed above, action 630 may be performed such that the instructions are instructions to the recipient using the hearing prosthesis. In some cases, method action 630 is performed to exclude one or more parties to the conversation, such as, by way of example only and not limitation, such that the person other than the recipient does not receive the instructions or such that the person other than the recipient is the only person receiving the instructions. Further, in embodiments where the instructions are provided to persons other than the recipient, one or more persons who are not the recipient may be excluded from a group of non-recipient persons, for example, by only illuminating an LED that can be seen by some persons but not others, or by providing a text message to only some persons' smartphones, etc.

[0087] Also, in at least some exemplary embodiments, method action 630 is performed so that an indication is provided to all members of the conversation. In an exemplary embodiment, method action 630 is performed so that all persons see the indication, wherein the indication is a visual indicator. In some exemplary embodiments, method action 630 is performed in a manner that distracts one or more speakers of the conversation and / or does not notify one or more speakers of the conversation (e.g., by way of example only and not limitation, a recipient of a hearing prosthesis or a speaker speaking to the recipient). To be clear, in some embodiments, the indication is provided to all parties of the conversation, while in other embodiments, the indication is provided to only some of them (e.g., only the speaker, only the person with the hearing prosthesis, only the person other than the hearing prosthesis, including all of these persons, etc.). In some embodiments, the indication distracts all parties of the conversation, while in other embodiments, the indication distracts only one or more parties (e.g., only the speaker, only the person with the hearing prosthesis, only the person other than the hearing prosthesis, including all of these persons, etc.).

[0088] Note that variations of method 600 are Fig.13 , method 1300 includes method action 1310, which includes an action of capturing sensory input during an interaction between two or more individuals, one of whom is using a sensory prosthesis to at least enhance sensory capabilities, consistent with the teachings above. Method 1300 also includes processing the captured sensory input to identify instructions to one or more individuals in the interaction so that the person using the sensory prosthesis can have at least one of an enhanced sensation or a sufficient sensation for future sensory input, as seen in box 1320. The future sensory input can be part of the same conversation that produced the original sensory input associated with method action 1310. Method 1300 also includes method action 1330, which includes outputting instructions for one or more individuals. Method 1300 can be implemented according to any of the teachings herein.

[0089] In an exemplary embodiment, as described above, the indication of method act 630 is an instruction indication. In an exemplary embodiment, this can be an instruction to a participant in the conversation other than the person using the hearing prosthesis. In an exemplary embodiment, this can be an instruction to the person in the conversation other than the person using the hearing prosthesis to speak differently. In keeping with the automatic nature of the system detailed above, in this exemplary embodiment, the instruction is not directly prompted by the person using the hearing prosthesis. That is, in at least some exemplary embodiments, the recipient is not prompted to enter the instruction. This is not to say that the recipient does not activate the system, just not directly prompted. All automatic systems must be activated by a person in some way or another in order to perform automatic operations. Note that in at least some exemplary embodiments, the instruction is not directly prompted by any party to the conversation. Note also that in certain other embodiments, the instruction is not indirectly prompted by any party to the conversation.

[0090] Note also that in some alternative embodiments, the indicators are actually prompted directly by the recipient of the hearing prosthesis. In this exemplary embodiment, this may be of practical value for scenarios where the recipient wants to control the indicators but does not want to blatantly interrupt the conversation (e.g., by saying "can't hear you" or "can you speak louder"). In this exemplary embodiment, the recipient has the ability to control the system to output indicators based on manual input.

[0091] In an exemplary embodiment, the system is configured so as to enable a recipient to fully or partially override the system, or otherwise prevent one or more or all of the indications from being provided in a scenario where such an indication would be provided all else being equal. In an exemplary embodiment, any device detailed herein may be provided with an input kit, or may otherwise be configured to receive input from a recipient, and be configured to override one or more or all features of the system, or otherwise override one or more or all activities of the system, such as preventing one or more of the indicators from being indicated, based on the received input. In an exemplary embodiment, the recipient's smartphone and / or the recipient's smartwatch may be used as an input kit for overriding purposes. Thus, in some embodiments, a system as detailed herein, wherein at least one of its components (prosthesis, remote device, etc.) is configured to enable a recipient of the prosthesis to override and / or adjust one or more indications.

[0092] In short, note that some embodiments include prostheses that are configured to adjust the functionality of a remote device that is independent of the indicators detailed herein. As an example, features of a remote microphone, or even features of a smartphone or smartwatch.

[0093] Additionally, note that in at least some exemplary embodiments, the remote display of the dedicated device of the system can be partially or fully copied or otherwise presented on a smartphone and / or smartwatch screen in a modified manner that still provides at least some of the information provided by the remote display. In exemplary embodiments, this can be used for control purposes to control one or more or all of the systems and / or subsystems detailed herein.

[0094] To be clear, in at least some exemplary embodiments, some or all of the teachings detailed herein relate to systems and methods that free a hearing prosthesis wearer from having to blatantly interrupt or otherwise barge into a conversation in which the wearer has difficulty hearing. Likewise, in at least some exemplary embodiments, the teachings may be implemented with respect to providing information on a display or otherwise through a device associated with one or more remote microphones. Any other device that can be operated or otherwise placed within the grasp of the recipient of the prosthesis may also be used, such as a smartphone or the like. Additionally, note that any such device (including the aforementioned remote microphones) may be combined with or otherwise include control components that can implement control of one or more of the devices associated with the system, wherein, in some embodiments, the controls may enable the recipient to override or otherwise minimize the information provided by implementing the teachings detailed herein.

[0095] In an exemplary embodiment, all parties to the conversation are aware of the content of the teachings detailed herein. In contrast, in another exemplary embodiment, only the recipient is aware that the system is being used and is otherwise aware of what is happening. In one or both of these embodiments, the teachings detailed herein can provide feedback to the recipient. The feedback can indicate whether the recipient is experiencing a problem and / or the device is causing a problem (in each case, the device should be changed, e.g., the volume control should be adjusted and / or the noise cancellation system should be engaged or disconnected, etc.).

[0096] That is, other discrete detectors may be used, such as, by way of example only and not limitation, the recipient simply taps or otherwise touches the screen of their smartphone and / or smartwatch. By way of example only and not limitation, the smartphone screen and smartwatch screen may be divided into two or four (or more) sections that may or may not be visible on the screen, wherein the recipient may discreetly tap one of these sections without looking at the phone, so that the smartphone may send a signal to the system to output an indicator that will be received by the recipient, e.g., the smartphone may communicate a signal with the BTE device, and the area of ​​the screen that the recipient tapped causes an LED to illuminate in a particular color, and the speaker knows what the indication means to him or her. This concept may also be extended to smartwatches, etc. Note also that in some cases, the number of taps may control the type of indication, as opposed to the location that was tapped (one tap means speaking louder, and thus the LED on the earhook facing the speaker lights up red, two taps means speaking slower, and thus the LED on the earhook facing the speaker lights up yellow, etc.).

[0097] Note that in some embodiments, method action 630 causes the indication to be instruction information, as noted above. In contrast, in some exemplary embodiments, the indication is non-instruction information. As an example only and not as a limitation, instruction information may correspond to an affirmative command to do something, while non-instruction information may correspond to providing only information relevant to a given scenario. As an example only and not as a limitation, the indication may be that the speaker is speaking too low or too fast. Further, as an example only and not as a limitation, the indication may be that the microphone placement is not optimized or that the microphone may be moved to a better position in addition. For example, the indication may be that the microphone may be rotated 20° to better capture the voice of the speaker on the right side of the microphone. If the speaker on the left side of the microphone is also speaking, and the speaker may wish to hear the voice at least as well as the other speakers, or in addition to speak in a manner that is not as clear as the speaker on the left, the recipient may or may not wish to adjust the microphone position to better capture the voice of the speaker on the right side of the microphone, and therefore ensure that the microphone is positioned so that it better captures the voice of the speaker on the left than the voice of the speaker on the right.

[0098] To further expand on some of the above teachings, and to provide specific examples of using some of the above teachings, note that in exemplary embodiments, the system may utilize different indicators to associate speech with microphone placement. In some embodiments, three separate instructions may be provided at exactly the same time. (Note that in some embodiments, more than three separate instructions may be provided at the same time, or two separate instructions may be provided at the same time.) In exemplary embodiments, the text screen may provide two or three or four more instructions at the same time.

[0099] With respect to background noise, in an exemplary embodiment, this involves instructions related to data about a recipient of a hearing prosthesis. In this regard, in an exemplary embodiment, if the system determines that background noise is interfering with the sound processing of the prosthesis, then the system may provide instructions that the recipient should move to an area where less background noise is present. Instructions may be provided by the system indicating that the recipient or another person should remove a given noise, turn off the current to a device that is generating the noise, etc. In an exemplary embodiment, the system may be configured to provide instructions that one or more microphones should be deactivated and / or the sensitivity of one or more microphones should be adjusted, and / or the output amplification from one or more microphones should be adjusted to accommodate or otherwise account for the noise. Further, in an exemplary embodiment, the system may be configured to automatically deactivate or otherwise make one or more of the aforementioned adjustments to one or more microphones if the system determines that background noise is interfering with the sound processing of the prosthesis.

[0100] Note that many other scenarios are conceivable with respect to instructions relating to data involving a recipient of a hearing prosthesis, such as potential electromagnetic interference with the prosthesis's wireless system.

[0101] Thus, in an exemplary embodiment, the indication of method act 630 indicates that one or more features in the surrounding environment are detrimental to optimal hearing perception of a person using a hearing prosthesis. Such indications include implicit indications, such as that certain actions may be taken to address a detrimental situation, even if the detrimental situation is not specifically identified. By way of example only and not limitation, the indication may correspond to a statement that “reducing background noise may help you hear.”

[0102] In some exemplary embodiments, the indication of method act 630 indicates that the sensitivity of a sound capture device remote from the hearing prosthesis (e.g., by way of example only and not limitation, a remote microphone, whether or not a dedicated remote microphone, and / or a remote microphone that is part of a portable consumer electronic product (e.g., a smartwatch or smartphone)) can be adjusted and / or another sound capture device can be used to improve the hearing of a person using a hearing prosthesis. By way of example only and not limitation, with respect to the latter, this can include indicating that the recipient should use a remote microphone that is in wireless communication with the hearing prosthesis or is otherwise positionable to be in wireless communication with the hearing prosthesis, rather than using a dedicated microphone of the prosthesis, such as a microphone on a BTE device. In exemplary embodiments, again with respect to the latter, this can include an indication commanding one or more parties to speak into a microphone of a smartphone or smartwatch that is in signal communication with the hearing prosthesis.

[0103] With respect to the former, such indications may be provided while the recipient is using a sound capture device that is remote from the hearing prosthesis. Such indications may be provided with respect to a scenario in which the recipient is using a dedicated remote microphone, for example, when the recipient has handed the remote microphone to one of the speakers and / or has placed the remote microphone on a table close to the speaker so that the sound capture device can better capture the speaker's voice.

[0104] Consistent with at least some of the exemplary embodiments detailed above, the output regarding the captured sound may be an indication of the presence of a noise source that detrimentally affects the auditory perception of another sound. By way of example only and not limitation, the multi-purpose smart portable device may be equipped with a memory that includes data indicating the sound spectra of different noises, such as, by way of example only and not limitation, a motorcycle engine running fast, a leaf blower, a jackhammer, etc., that have been previously identified as sounds that may have a detrimental effect on another sound. In an exemplary embodiment, the multi-purpose smart portable device is configured to compare the incoming sound with the data indicating the sound spectra, which may be located on a lookup table, etc., and make a determination that a sound with a detrimental effect exists. For example, a sound that has been identified as having a detrimental effect may be classified as such, and the algorithm used by the multi-purpose smart portable device is configured to output an indication when there is a correlation between the incoming sound and a catalog sound.

[0105] Consistent with the teachings detailed above, the output from the multi-purpose smart portable device regarding the captured sound may be a visual indicator. This may be a visual indication on the display of the smart portable device that may indicate that people within the viewing distance of the display (e.g., any party to a conversation that can see the display, which may be all people in the conversation in a scenario where, for example, the smart portable device is placed on a table) should take action, that people within the viewing distance of the display are acting in a useful manner (e.g., the display may indicate that the speaker should continue to speak in the manner in which he has been speaking or is otherwise currently speaking), or at least one of a characteristic of the surrounding environment.

[0106] Note that some embodiments may bi-split, tri-split, or quad-split the display so that the indicator is reproduced multiple times in orientations that can be read or otherwise evaluated by two or more people angularly away from each other. By way of example only and not limitation, in a scenario where two people are sitting at a table 180° apart from each other, the recipient may place the smart portable device at a location on the table where both parties can see the display. The smart portable device may have programming, for example in the form of an application thereon, that enables the recipient to activate a conversation application that performs one or more of the methods detailed herein, and bi-split the screen so that the top of the screen (e.g., the portion of the screen away from the recipient) presents characters that are upside down relative to the recipient but to the right of the person the recipient is speaking to, while the bottom of the screen (e.g., the portion of the screen closest to the recipient) presents characters that are up to the right of the recipient but to the right of the person the recipient is speaking to. Thus, in an exemplary embodiment, such as by way of example only and not limitation, in which a multi-purpose smart portable device provides visual indicators of characteristics of the surrounding environment, the words "Music is Loud" may be displayed on the screen both right-side up and upside down so that the recipient and the person the recipient is speaking to can read the words (this exemplary scenario is performed using a system configured to identify music using the aforementioned sound analyzer or other techniques and to determine whether the music is relatively loud, e.g., the volume may have a deleterious effect on the arousal of the auditory perception of speech).

[0107] It is also noted that the above exemplary embodiments can be implemented in a more simplified manner, such as when the screen is divided into two or three or four or five or six or more parts (e.g., a square pie chart), and each part is illuminated with red or yellow or green light according to a given scenario. For example, in a scenario where, for example, the user of a cochlear implant is playing poker (winning poker) with five other players, all of whom are generally equally spaced around a round table, the smartphone can display on the screen five different parts or six different parts (if the multi-purpose smart portable device is also used to analyze the recipient's voice), each part generally pointing to a given player or otherwise aligned with a given player. In a given scenario, everyone speaks in a practical way, and all five or six different parts will be illuminated by green. If one of the speakers speaks very quickly, and the multi-purpose smart portable device determines that this is the case, then that part of the screen can turn yellow, while all other parts will remain green. Further, if the other person speaks in a very soft way, then that other part can also turn red, while all other parts (such as the yellow part) remain green.

[0108] Of course, the embodiments detailed above have been directed to the context of using a multi-purpose smart portable device in a conversation with a person utilizing a hearing prosthesis. In this regard, in at least some exemplary embodiments, the multi-purpose smart portable device is configured to receive input indicating the presence of a person utilizing a hearing prosthesis, who is at least within the field of view and / or intelligent speech distance of the portable device. In exemplary embodiments, the device is configured to indicate to the person whether the person is speaking in a particular practical manner and / or not speaking in a particular practical manner.

[0109] Figure 7 An exemplary algorithm of an exemplary method - method 700 is shown. Method 700 includes method action 710, which includes an action of a hearing-impaired person participating in a conversation. In an exemplary embodiment, the hearing-impaired person is a person using a hearing prosthesis. Method 700 also includes method action 720, which includes capturing at least a portion of the conversation at a point in time using a first electronic device. In an exemplary embodiment, this may correspond to using a microphone on a BTE device or on an OTE sound processor, etc. In another exemplary embodiment, this may correspond to using a remote microphone specific to a prosthesis. In another exemplary embodiment, this may correspond to using a microphone of a smart watch or smartphone or another type of consumer electronic device including a microphone. Method 700 also includes method action 730, which includes analyzing the captured sound using a first electronic device and / or a second electronic device. In an exemplary embodiment, such as in the case of using a prosthesis for analysis, the prosthesis is configured to perform one or more method actions detailed above, a processor in the prosthesis performs method action 730, and if the prosthesis was used to perform method action 720 to capture sound, the first device is used to perform method 730. However, if the captured sound captured by the prosthesis is provided wirelessly to, for example, a multi-purpose smart portable device, such as a smartphone, and the smartphone analyzes the captured sound, method action 730 is performed using a second electronic device. Thus, in an exemplary embodiment, the first electronic device is a hearing prosthesis worn by a hearing-impaired person, and the action of analyzing the captured sound is performed using a second electronic device, wherein the second electronic device is a portable smart device. In this exemplary embodiment, the first electronic device provides a wireless signal to the second electronic device based on the captured sound (e.g., the BTE device or other part of the hearing prosthesis provides an RF signal to the smartphone, which is based on the captured sound), and the second electronic device analyzes the signal in an action of analyzing method action 730 (e.g., utilizing an onboard processor of the smartphone to analyze the signal in accordance with the teachings detailed above is a variation thereof).

[0110] Furthermore, in an exemplary embodiment, the captured sound may be a sound captured by a microphone of a portable smart device, for example, and thus the first electronic device may be a portable smart device. In any case, the method 700 also includes a method action 740, which includes manually providing information to one party of the conversation related to the captured sound based on the analysis during the conversation. By manually providing, it is meant that this is not performed in a natural manner by a person or the like. Instead, the information is provided manually, for example, via an indicator on the BTE device as detailed above, or a text-based message on a display screen of a smartphone or smartwatch or the like.

[0111] In an exemplary embodiment, the information provided to the conversation party may be any of the information detailed above and its variations.

[0112] In an exemplary embodiment of method 700, method action 740 includes providing a visual indicator to a party using a remote device that is remote from the party based on a specific result of the analysis. In an exemplary embodiment, the device can be, for example, a smart phone that is located on a table. In an exemplary embodiment, the device can be, for example, a smart watch that has been removed from the wrist of the recipient or another party and placed on a table. In another exemplary embodiment, the device can be a dedicated device for performing method 700 that does not perform other operations except performing the teachings and / or variations thereof detailed herein. In contrast, in an exemplary embodiment, method action 740 includes providing a visual indication to the party using a device worn and / or held by a party.

[0113] In the aforementioned exemplary embodiments, for example, this can be performed using an LED indicator on a smart watch or BTE. With respect to the latter, this can be performed using a smart phone or smart watch held in the recipient's hand. Therefore, in exemplary embodiments where the first electronic device can be a BTE device and the second electronic device can be a smart phone, in at least some exemplary embodiments, method action 740 includes manually providing information to the conversation party using the second electronic device (smart phone), and in some cases, method action 740 includes manually providing information related to the captured sound to one of the conversation parties based on analysis during the conversation to enhance aspects of the conversation at a subsequent point in time.

[0114] In an exemplary embodiment, the first electronic device is a hearing prosthesis worn by a hearing-impaired person, and method action 730 is performed using a second electronic device different from the first electronic device, such as, by way of example only and not limitation, a portable smartphone or remote processor that communicates signals with the prosthesis via a server, which can communicate signals with the prosthesis via a Bluetooth system. By way of example only and not limitation, in an exemplary embodiment of this exemplary embodiment, there is an additional action of providing a wireless signal to a third electronic device via the first electronic device (e.g., in this example, the hearing prosthesis) and / or via the second electronic device (e.g., a smartphone and / or server processor combination), commanding the third electronic device to perform an action of manually providing information to the interlocutor (performing method 740). By way of example only and not limitation, the third electronic device can be a smart watch or some other accessory different from the first and second electronic devices. Note that in the exemplary embodiment where the first electronic device provides the wireless signal, it may be the case in the following exemplary embodiments that the remote server communicates signals with the hearing prosthesis, and the hearing prosthesis receives the result of the analysis, and then provides a signal to the third electronic device, which causes the third electronic device to provide an indication. In contrast, in an exemplary embodiment where the processing is performed by a smartphone, the symbiotic relationship between the smartphone and the smartwatch may rely on having the smartwatch perform method act 740 .

[0115] In some embodiments, for example, where the first electronic device is a hearing prosthesis worn by a hearing-impaired person, the action of analyzing the captured sound is performed using the first electronic device. Similarly, in an exemplary embodiment, the action of manually providing information to a party to the conversation is performed by the first electronic device, all of which is consistent with the detailed embodiment above in which the hearing prosthesis includes an indicator. Therefore, in this exemplary embodiment, a party to the conversation can be a person other than the recipient of the hearing prosthesis. That is, in at least some exemplary embodiments, in accordance with the teachings detailed above, a party to the conversation can be a person with a hearing prosthesis. Further, action 740 is not mutually exclusive between a person with a hearing prosthesis and a person without a hearing prosthesis: a method action can be performed simultaneously with method action 740 so that the information can reach two people or two types of people, where, for example, there is more than one person without a hearing prosthesis in the conversation.

[0116] In at least some exemplary embodiments of method 700, the information of method act 740 is information specific to non-dialogue related sounds. By way of example only and not limitation, the information may be about background sounds or wind noise, etc. The information may indicate that there is considerable background noise and / or there is considerable wind noise, etc.

[0117] The embodiments detailed above generally focus on only one component providing information / providing instructions, and it is noted that in some exemplary embodiments, there may be a system including two or more such components. In exemplary embodiments, by way of example only and not limitation, both the prosthesis and the multi-purpose smart portable device may provide indicators / instructions / information. It is also noted that in some embodiments, there may also be multiple multi-purpose smart portable devices (more on this below). In some exemplary embodiments, there may be a prosthesis, one or more multi-purpose smart portable devices, and one or more non-multi-purpose smart portable devices, such as devices dedicated to performing one or more method actions here and not performing other devices. It is also noted that in at least some exemplary embodiments, there may be multiple hearing prostheses, one of which may communicate with other prostheses in the same or similar manner as the communication between the remote device and the prosthesis. In some cases, one or more or all of the devices in the system provide the same indicator simultaneously or in a time interval. In some cases, one or more or all of the devices in the system provide different indicators simultaneously or in a time interval. In some cases, these indicators provided by separate components are provided directly and / or only to a specific person who is a party to the conversation. By way of example only and not limitation, in an exemplary embodiment, prior to engaging in a conversation and / or during the conversation, one or more parties to the conversation download an application onto their multi-purpose smart portable device that is capable of implementing one or more or all of the method actions detailed herein. In an exemplary embodiment, for example, as an example, the devices may be configured with a brief information screen request that asks whether the holder or owner of the multi-purpose smart portable device is a person with a hearing prosthesis or a speaker to a hearing prosthesis, etc. In an exemplary embodiment, the application may cause a multi-purpose smart portable device owned or otherwise possessed by a party to the conversation (who does not have a hearing prosthesis) to be placed in signal communication with a hearing prosthesis and / or with a multi-purpose smart portable device being used by a person with a prosthesis, so that the multi-purpose smart portable device owned or otherwise possessed by the party without a hearing prosthesis may perform one or more of the method actions herein. By way of example only and not limitation, in an exemplary embodiment where, for example, an action of analyzing captured sound is performed using a device owned or otherwise owned by a person with a prosthesis, the first and / or second electronic device may provide a wireless signal to a third electronic device, causing the device to output the information, and the third electronic device is a portable device owned or otherwise owned by a person without a hearing prosthesis. That is, in an exemplary embodiment, the portable electronic device owned by a person without a hearing prosthesis may operate independently of the other components, as is the case with a multi-purpose smart portable device owned by the recipient or any other device owned by the recipient.Thus, in an exemplary embodiment, there may be a scenario where two or more separate smartphones and / or two or more separate smart watches independently provide information to their respective owners / process sources in accordance with the teachings detailed herein.

[0118] Figure 8 An exemplary flow chart of an exemplary method, method 800, is shown, which is a method of managing a conversation, which includes method actions 810 and 820. In method 800, method action 810 includes electronically analyzing, using a portable electronic device, sounds captured during a conversation. Method action 820 includes manually providing an indicator related to the speaking style of a participant to a recipient in the conversation. One requirement of method action 820 is that this action is based on the analysis of method action 810. Another requirement of method action 820 is that this action is performed to improve the conversation, consistent with the teachings herein.

[0119] That is, alternatively and / or in addition, in an exemplary embodiment, method action 810 may include utilizing a prosthesis to analyze the sounds captured during the conversation. In an exemplary embodiment, some embodiments where the prosthesis has additional "information" or capabilities beyond what may be associated with a remote device may have practical value. This may enable further refinement of the type of indicator, or even whether the indicator should be provided to the recipient in the conversation, relative to the case where the prosthesis is not utilized to perform method action 810. By way of example only and not limitation, the prosthesis may be equipped with a sound classifier or other type of device that can classify input and / or output. Further, in an exemplary embodiment, the prosthesis may be configured to obtain or otherwise receive input indicating context awareness. In an exemplary embodiment, the prosthesis may be configured to determine whether the recipient is utilizing the prosthesis in a manner that is paired with one or more remote devices. In addition, in an exemplary embodiment, some embodiments for reasons associated with the communication scheme between the prosthesis and other components may have practical value. By way of example only and not limitation, in an exemplary embodiment, the prosthesis may communicate signals with a remote microphone / miniature microphone. This is in contrast to, for example, a smartphone or smartwatch, at least in some usage scenarios. Figure 8 One or more or all of the associated method actions simultaneously directly adjust the remote microphone.In view of the above, in some embodiments, the prosthesis and others detailed herein are configured to automatically determine that it is paired with a remote device and, as a result of that determination, begin providing input to the remote device.

[0120] Also note that in at least some exemplary embodiments, the system is configured to analyze the state of the recipient, or otherwise infer the state of the recipient through potential variables, or otherwise simply receive information indicating the state of the recipient (such as, through direct input from the recipient) and take action based on the state of the recipient. As an example only and not as a limitation, the state of the recipient can be that the recipient is talking, or as an example, the recipient deliberately tries not to pay attention to a particular conversation. The action taken based on the state of the recipient can be an action to change the instruction or even cancel the instruction or otherwise activate the instruction relative to other situations. For example, if the system "recognizes" that the recipient is not really paying attention to the conversation (e.g., the recipient presses a button indicating that the recipient does not care what the people in the conversation are saying), the number or level of instructions to the recipient and / or to the parties and the conversation may be relatively reduced (if not eliminated) because it is not important to say what is said. On the contrary, if the system recognizes that the recipient is trying to pay relatively close attention to the conversation, the number and / or type and / or level of instructions may be increased relative to other situations. Thus, in an exemplary embodiment, the system is configured to drive the display of information / instructions to the parties in a manner that has practical value for the specific set of dynamics in the conversation, as opposed to just treating each conversation as a generic event to which the system should react in a standard manner.

[0121] It is also noted that in at least some exemplary embodiments, the systems detailed herein can be complex in that the system can actually identify specific features of the environment. By way of example only and not limitation, the system can determine that it is a specific television that should be adjusted in a certain way, for example, to enhance the dialogue. Furthermore, in exemplary embodiments, the system can be configured to provide a specific adjustment to that specific feature in the environment. For example, the instruction can be to turn off the television. That is, in exemplary embodiments, the instruction can be to adjust the volume of the television by a specific amount.

[0122] Furthermore, in at least some exemplary embodiments, the system may be configured to actually control certain features in the environment. For example, via the Internet of Things, a portion of the system may be able to communicate with a television, radio, etc. in the environment and automatically make adjustments thereto. Alternatively and / or in addition, in exemplary embodiments, the system may be configured to communicate with such other devices in the environment and prompt the device to display an indication to its user that it should adjust the device, or at least ask the user if they mind adjusting the device, all in an effort to enhance the conversation in accordance with the teachings detailed herein.

[0123] Fig. 9An exemplary flow chart of an exemplary method, method 900, is shown. Method 900 includes method action 910, which includes announcing to another participant of a conversation that an indicator may be provided by a participant using a hearing prosthesis of method 800, and explaining to another participant of the conversation by the participant using the hearing prosthesis what the indicator means. By way of example only and not limitation, in an exemplary embodiment, before engaging in the substance of the conversation, the recipient of the hearing prosthesis presents a multi-purpose smart portable device, or other device away from the hearing prosthesis that will be used to provide the indicator and / or point to or otherwise identify the indicator on the prosthesis (e.g., an LED on an ear hook facing the speaker). While doing this, or after doing this, or before doing this, the recipient explains what indicators will appear, what these indicators mean, and / or what the person speaking can do or should do or should not do when these indicators appear. Of course, method 900 includes method action 920, which includes performing method 800. Again, as with all methods detailed herein, the order of the method actions is not specific unless otherwise specified. In this regard, in an exemplary embodiment, method act 810 may be performed, at least in part, prior to method act 910. By way of example only and not limitation, two people may engage in a conversation, and then the recipient may feel fatigued, and then the teachings detailed herein may be implemented to reduce or otherwise mitigate the effects of such fatigue. Further, by way of example only and not limitation, two people may engage in a conversation, and then the recipient may realize that the person speaking to him or her is speaking in a manner that is subjectively and / or objectively undesirable to the recipient (e.g., in a statistically undesirable manner), and thus may present the concepts detailed herein by performing method act 910, and then continue the conversation.

[0124] In an exemplary embodiment, method act 820 includes providing a visual indicator by the portable device, and the visual indicator may be a first light indicating that the participant is speaking in an unsatisfactory manner. In an exemplary embodiment, the algorithm detailed above and variations thereof may be used to determine that the indication should be given.

[0125] In an exemplary embodiment, after the action of providing a first light, there is an action of capturing and analyzing the sound captured during the conversation and providing a visual indicator in the form of a second different type of light, which indicates that the participant is speaking in a satisfactory manner. Note that in some embodiments, the physical structure of this light is the same light corresponding to the first light, but this light can have different characteristics, such as a different color, this light can be steady, while the previous light can suddenly light up or suddenly go out, etc. That is, in some embodiments, this light can be a completely different physical structure, such as a completely different LED. In some embodiments, the indicator can indicate that the speaker is speaking in a practical and satisfactory manner, and / or can simply indicate that the speaker is speaking in a better manner and the recipient can speak better. By way of example only and not limitation, the first light can be red and the second light can be yellow. A third subsequent light can be provided that is green to indicate that the speaker is speaking better than he or she previously did.

[0126] Fig.10 An exemplary flow chart of another exemplary method, method 1000, is shown. Method 1000 includes method action 1010, which includes performing method 800. Method 1000 also includes method action 1020, which includes electronically analyzing the sound captured during the conversation for a second time using a portable electronic device. In this embodiment, method action 1020 is performed after a time progression from the first time the sound was captured in method 800. In an exemplary embodiment, method action 1020 can be triggered using a time trigger. By way of example only and not limitation, in an exemplary embodiment, the system can be configured to perform method action 1020 within one, two, three, four, five, six, seven, eight, nine, 10, 11, 12, 13, 14, 15, 20, 25, 30, 35, 40, 45, 50, 55, or 60 seconds from the previously analyzed captured sound. That is, the analysis can occur continuously. Still further, the analysis may be such that the analysis entails capturing sound for a portion or all of a time period from a last captured sound and performing the analysis on some or all of the captured sound. In some embodiments, the algorithm used to perform the analysis weights captured sound that is further away in time from a previously captured sound in order to account for the likelihood that the speaker has adjusted their speech since the last indication, wherein the speaker may have adjusted their speech better further away from the last indicator and / or their last captured sound.

[0127] Method 1000 also includes method action 1030, which includes manually providing a second subsequent indicator to the participant in the conversation. In an exemplary embodiment, the requirement of method action 1030 is that it is completed based on the second analysis of method action 1020. In addition, in an exemplary embodiment, the requirement of method action 1030 is that the indicator is an indicator that the participant is speaking differently. That is, in at least some exemplary embodiments, there is a method action between method actions 1020 and 1030, which may include performing a derivative action of method action 1030, except that the indicator is that the participant should improve his or her speaking based on the analysis of action 1020. In an exemplary embodiment, after this action, method 1000 may include performing a derivative action of method action 1020, which may require performing an analysis of the sound captured during the conversation for a third time using a portable electronic device, wherein, after this, if it is determined that the person is now speaking differently, the method proceeds to method action 1030.

[0128] In view of the above variations of method 1000, it is to be understood that, in at least some exemplary embodiments, variations of method 1000 may be performed repeatedly, and the results of those methods with respect to the indicators may be different each time based on the results of the analysis. In this regard, Fig.11 An exemplary flow chart of an exemplary method, method 1000, is presented, which includes method action 1110, which includes, for a value of N equal to 1, performing method 800. Method 1000 also includes method action 1020, which includes electronically analyzing, using a portable electronic device, sounds captured during a conversation N+1 times, where if N is the first time, the N+1 time may be related to the second time, if N is the second time, the N+1 time is the third time, and so on. Method 1100 also includes method action 1130, which includes manually providing, after the Nth indicator, based on the N+1 analysis, to a participant in the conversation, possibly an N+1th indicator that the participant is speaking in a particular manner. In this regard, note that the word "may" appears in this method action. In some embodiments, such as where the system only provides indicators when the participant is not speaking in a practical manner, method action 1130 may result in such an indication or instruction to speak in a different manner, whereas in such embodiments, if the person is speaking in a practical manner, the system will not provide an indicator. That is, in some embodiments, such as when the system provides indicators for all or most situations, for example, when the speaker is speaking in a pragmatic manner, such an indication may be provided. In any case, in method act 1130, there is an act of setting N equal to N+1, and thus incrementing N to the next value. The method then returns to method act 1120, wherein method act 1120 is performed for the new value of N, and then method act 1130 is performed again for this new value of N, and so on, until the system is turned off.

[0129] To be clear, consistent with the teachings detailed above, an indicator of the foregoing method actions can be an indicator that the participants can speak differently to improve the conversation. Consistent with the teachings detailed above, an indicator can be an indicator that one or more participants are speaking too softly or speaking too quickly. In exemplary embodiments, additional indicators and / or fewer indicators can be utilized, and in some embodiments, multiple indicators can be present simultaneously.

[0130] Fig.12 An exemplary flow chart of an exemplary method, method 1200, is presented, which includes method action 1210, which includes performing method 800. Method 1200 also includes method action 1220, which includes the following actions: based on the analysis (of method 800) and / or subsequent analysis of subsequently captured sound (e.g., a context associated with method 1000 or method 1100), artificially providing an indicator of the presence of a phenomenon detrimental to the conversation separated from the participant's voice to a participant in the conversation. By way of example only and not limitation, such method actions accompany the embodiments related to background noise detailed above. In at least some exemplary embodiments of this exemplary embodiment (embodiments of method 1200), the indicator of the presence of a phenomenon detrimental to the conversation to a participant in the conversation is an indicator of one or more ambient noise or defective microphone placement.

[0131] Note that at least some exemplary embodiments include algorithms based on statistical analysis of words that a recipient may have difficulty understanding when using the particular hearing prosthesis in question. By way of example only and not limitation, a statistical database may be created or otherwise obtained regarding words that are often difficult to understand for people who typically use cochlear implants, or more specifically for a particular recipient who participates in the method or otherwise uses the system detailed herein. By way of example only and not limitation, the analysis performed by the system of the speech of a conversation may include the identification of certain words that will trigger an indicator and / or a set of instructions, such as, for example, to speak more slowly, to enunciate more clearly, not to speak while eating, or not to speak when a cup is in front of one's face (this may be based on analysis of different parts of the speech, such as to indicate echoes of sound waves generated by speech hitting glass or fluids such as fluid in a cup) - also note that in at least some exemplary embodiments, the system is configured to use visual images that can assess the speaker's movements, which can be automatically analyzed to determine that the speaker is moving, or otherwise positioning itself in a less practical manner relative to the hearing prosthesis that captures the sound), etc.

[0132] It is noted that in some embodiments, the system may be configured to instruct someone to stop talking, whether the recipient or someone who is a party to the conversation. Further, in an exemplary embodiment, the system may be configured to instruct one or more parties to the conversation (regardless of who that may be) to notify others who are not a party to the conversation that they should stop talking.

[0133] Note also that in some exemplary embodiments, the indication provided by the system may be merely an indication to the recipient and / or the other party to the conversation taking place in the ambient sound environment. By way of example only and not limitation, the system may be configured to analyze the sound and indicate to the recipient certain characteristics of the sound, such as the presence of a moderate level of background noise, the person speaking to you is speaking clearly, the presence of wind noise, the presence of a low level of background machine noise (central fan), etc. Note also that the system may provide indications about the environment as well as instructions.

[0134] In at least some exemplary embodiments, any type of indication to one or more parties to the conversation may be utilized. In an exemplary embodiment, tactile feedback is provided. In an exemplary embodiment, an audio indicator is provided. While the embodiments detailed above have been directed to audio indicators provided only by the prosthesis and audible only by its recipient, in some alternative embodiments, the audio indicator may be audible to all parties, or the audio indicator may be audible only to one or more parties to the conversation without the prosthesis.

[0135] To be clear, in at least some exemplary embodiments, some or all of the methods, systems, and apparatuses herein are completely and thoroughly dialog-based. By way of example only and not limitation, any one or more or all method actions associated with the methods detailed herein may be completely dialog-based. That is, the evaluation / analysis and instructions are completely dialog-based and nothing else (in some embodiments).

[0136] Note that the disclosure herein includes analysis performed by certain devices and / or systems. It should be noted that any disclosure herein of analysis also corresponds to the disclosure of an embodiment of performing an action based on an analysis performed by another device. As an example only and not as a limitation, any disclosure herein of an apparatus that analyzes a certain feature and then reacts based on the analysis also corresponds to an apparatus that receives input from an apparatus that has performed the analysis, wherein the apparatus acts on the input. In addition, the opposite also applies. Any disclosure herein of an apparatus that acts based on input also corresponds to an apparatus that can analyze data and act according to the analysis.

[0137] It should be noted that any disclosure of instructions herein also corresponds to disclosure of embodiments where the term instruction is replaced with information, and vice versa.

[0138] It should be noted that any disclosure of alternative arrangements and / or alternative actions herein corresponds to the disclosure combining the original arrangement / original action with the alternative arrangement / alternative action.

[0139] It should be noted that any method actions detailed herein also correspond to the disclosure of an apparatus and / or system, which is configured to perform one or more or all of the method actions associated with the method actions detailed herein. In an exemplary embodiment, this apparatus and / or system is configured to perform one or more or all of the method actions in an automatic manner. That is, in an alternative embodiment, the apparatus and / or system is configured to perform one or more or all of the method actions after being prompted by a person. It should also be noted that any disclosure of an apparatus and / or system detailed herein corresponds to a method of making and / or using the apparatus and / or system, including a method of using the apparatus according to the functionality detailed herein.

[0140] It should be noted that the embodiments include a non-transitory computer-readable medium having a computer program recorded thereon, the computer program being used to perform one or more or any of the method actions detailed herein. In fact, in an exemplary embodiment, a non-transitory computer-readable medium has stored thereon a computer program for performing at least a portion of any method action detailed herein.

[0141] In an exemplary embodiment, a method includes: a hearing-impaired person participating in a conversation; using a first electronic device to capture at least a portion of the sound of the conversation at a certain point in time; using the first electronic device and / or a second electronic device to analyze the captured sound; and manually providing information related to the captured sound to a party to the conversation based on the analysis during the conversation to enhance aspects of the conversation at a subsequent point in time. In an exemplary embodiment, the method as described above and / or below, wherein the action of manually providing information includes providing a visual indication to the party using a device remote from the party based on a specific result of the analysis. In an exemplary embodiment, the method as described above and / or below, wherein the action of manually providing information includes providing a visual indication to the party using a device worn and / or held by the party. In an exemplary embodiment, the method as described above and / or below, wherein the first electronic device is a portable smart device, and the analyzing action and the providing action are performed by the first electronic device.

[0142] In an exemplary embodiment, the method as described above and / or below, wherein the information is information specific to non-conversation related sounds. In an exemplary embodiment, the method as described above and / or below, wherein the first electronic device is a hearing prosthesis worn by a hearing-impaired person, an action of analyzing the captured sounds is performed using the second electronic device, wherein the second electronic device is a portable smart device, the first electronic device provides a wireless signal to the second electronic device based on the captured sounds, and the second electronic device analyzes the signal in the analyzing action, and the action of manually providing information to a party to the conversation includes utilizing the second electronic device.

[0143] In an exemplary embodiment, the method as described above and / or below, wherein the first electronic device is a hearing prosthesis worn by a hearing-impaired person, the action of analyzing the captured sound is performed using a second electronic device, and the second electronic device and / or the first electronic device provides a wireless signal to a third electronic device, commanding the third electronic device to perform an action of manually providing information to a party to the conversation. In an exemplary embodiment, the method as described above and / or below, wherein the first electronic device is a hearing prosthesis worn by a hearing-impaired person, the action of analyzing the captured sound is performed using the first electronic device, and the action of manually providing information to a party to the conversation is performed by the first electronic device, and the party to the conversation is a person other than the recipient of the hearing prosthesis.

[0144] In an exemplary embodiment, a method of managing a conversation includes: utilizing a portable electronic device; electronically analyzing sounds captured during a conversation; and based on the analysis, manually providing indicators to participants of the conversation related to how the participants are speaking to improve the conversation, wherein at least one participant of the conversation is using a hearing prosthesis to hear the sounds. In an exemplary embodiment, the method as described above and / or below further includes the actions of: announcing, by the participant using the hearing prosthesis, to another participant of the conversation that the indicator is available; and explaining, by the participant using the hearing prosthesis, the meaning of the indicator to the other participant of the conversation. In an exemplary embodiment, the method as described above and / or below, wherein the action of manually providing the indicator includes providing, by the portable device, a visual indicator; the visual indicator being a first light indicating that the participant is speaking in an unsatisfactory manner.

[0145] In an exemplary embodiment, the method as described above and / or below further includes, after providing the first light, an action of capturing and analyzing sound captured during the conversation and providing a visual indicator in the form of a second different type of light, the second different type of light indicating that the participant is speaking in a satisfactory manner. In an exemplary embodiment, the method as described above and / or below further includes: utilizing a portable electronic device; electronically analyzing the sound captured during the conversation a second time; and based on the second analysis, manually providing a second subsequent indicator to a participant of the conversation that the participant is speaking differently. In an exemplary embodiment, the method as described above and / or below, wherein the indicator is an indicator that the participant can speak differently to improve the conversation. In an exemplary embodiment, the method as described above and / or below further includes, based on the analysis and / or subsequent analysis of subsequently captured sound, manually providing an indicator to a participant of the conversation that there is a phenomenon that is detrimental to the conversation that is separate from the participant's voice.

[0146] In an exemplary embodiment, a method as described above and / or below, wherein the indicator indicating to the participants of the conversation that there is a phenomenon detrimental to the conversation is an indicator of one or more ambient noise or defective microphone placement.

[0147] It is also noted that any disclosure of an apparatus and / or system detailed herein also corresponds to a disclosure that additionally provides the apparatus and / or system.

[0148] It should also be noted that any element of any embodiment detailed herein may be combined with any other element of any embodiment detailed herein, unless so stated, as long as the technology can achieve these. It should also be noted that in at least some exemplary embodiments, any one or more elements of the embodiments detailed herein may be explicitly excluded in the exemplary embodiments. That is, in at least some exemplary embodiments, there are embodiments that explicitly do not have one or more of the elements detailed herein.

[0149] Although various embodiments of the present invention have been described above, it should be understood that they are presented by way of example only and not limitation. It will be apparent to those skilled in the relevant art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

Claims

1. A system comprising: Signal input; processor; as well as Signal output, where The processor is configured to generate instructions related to data regarding a recipient of a sensory prosthesis based on input of the signal input, and The signal output is configured to output data indicative of the instruction.

2. The system of claim 1, wherein: The processor is configured to analyze data related to the recipient and determine whether the person speaking is providing sensory input in a manner that enhances the sensory perception of the recipient of the sensory prosthesis.

3. The system of claim 2, wherein: The system is configured to identify whether the person speaking is the person having the sensory prosthesis.

4. The system of claim 3, wherein: The system is configured to provide the instructions to a recipient of the sensory prosthesis.

5. The system of claim 3, wherein: The system is configured to provide the instructions to a person other than a recipient of the sensory prosthesis who is part of a group of people providing sensory input captured by the system.

6. The system of claim 1, wherein: The signal input is in signal communication with a microphone.

7. The system of claim 1, wherein: The system is configured to induce at least one of a visual, an audible, or a tactile indication as an output indicating the instruction.

8. The system of claim 1, wherein: The system includes a hearing prosthesis and a smart device, the smart device including an interactive display screen remote from the hearing prosthesis; and The system is configured to display one or more controls for the hearing prosthesis on the interactive display screen.

9. A method comprising: capturing sensory input during an interaction between two or more individuals, one of whom is using a sensory prosthesis to at least enhance sensory capabilities; processing the captured sensory input to identify an indication to one or more persons in the interaction such that the person using the sensory prosthesis is able to have at least one of an enhanced sensation or an adequate sensation of future sensory input; as well as The instructions to the one or more persons are outputted.

10. The method according to claim 9, wherein: The instructions are instructions to participants of the conversation other than the person using the hearing prosthesis.

Images