Multimodal sound perception auditory stimulation system and method

Abstract

A multi-modal auditory stimulation method for stimulating an auditory perception of a subject. The method includes generating two or more of a multi-channel electrical stimulation signal, a bone conduction stimulation signal, and an audio stimulation signal based on an audio signal representing a multi-channel sound. The electrical stimulation signal is configured to provide an electrical stimulation of the auditory perception including a sound image localization of the auditory perception. The bone conduction stimulation signal is configured to provide vibrations that cause a bone conduction stimulation of the auditory perception. The audio stimulation signal is configured to generate an air conduction pressure wave sound. Some embodiments include obtaining compensation information for use in connection with the multi-modal auditory stimulation system. The compensation information may be obtained by applying the electrical stimulation, the bone stimulation, or the pressure wave sound together to the subject and receiving and storing as the compensation information information representative of a direction in which the subject perceives the resulting audio perception.

Description

[Technical Field]

[0001] TECHNICAL FIELD The present disclosure relates generally to systems and methods for improving a subject's hearing or speech perception. [Background technology]

[0002] Auditory sounds can also be perceived through electrical stimulation of the cochlea (also called cochlear stimulation or galvanic vestibular stimulation (GVS)) or bone conduction stimulation (e.g., via vibration of the skull). This type of auditory perception is distinct from hearing produced by air conduction stimulation or pressure waves stimulating the ear, which may be enhanced by hearing aids. There is a continuing need for systems and methods for enhancing auditory perception in subjects, such as those with impaired or reduced hearing function. Summary of the Invention

[0003] In accordance with the embodiments described herein, a subject's auditory perception can be enhanced by combining two or more of electrical, bone, or air conduction stimulation. Examples of applications include human hearing testing and treatment, military intelligence (e.g., when air conduction communication is unavailable, unused, impaired, or limited), and entertainment. Functionality includes providing real-time, electrically transmitted audio to the head in the full auditory range from 100 Hz to 20,000 Hz for content such as speech, music, and movies. In particular, combining electrical stimulation with any or all of the other stimulation modes can improve lateralization for sounds above 0 dB. [Brief explanation of the drawings]

[0004] [Figure 1A] 1 is a schematic diagram of a headset including a multi-modal stimulator positioned relative to a user's head, according to an embodiment. [Figure 1B] 1 is a schematic diagram of a headset including a multi-modal stimulator positioned relative to a user's head, according to an embodiment. [Figure 2] 1 is a schematic diagram of functional components of a control system for a multi-modal auditory stimulation system, according to an embodiment. [Figure 3] Schematic diagram of test results showing the location and direction of sound perception for 11 different subjects when the same type of combination of mono-channel bone conduction stimulation and multi-channel electrical stimulation was applied to stimulation devices placed in the same position on the head of each subject. [Figure 4] 3 is a schematic diagram illustrating components of a computer system that can be used to implement the control system shown in FIG. 2, in accordance with some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0005] The multimodal auditory stimulation systems, methods, and components described herein utilize two or more stimulation modalities, such as electrical stimulation of the cochlea, bone conduction stimulation, or air conduction stimulation, to enhance a subject's or other user's speech or hearing perception. In particular, the multimodal stimulation systems, methods, and components enhance a subject's ability to localize and identify the source of perceived sounds (e.g., the location or direction from which the sound originates), thereby improving the subject's ability to understand, enjoy, and obtain information from sounds, such as music, speech, and sounds in the subject's environment.

[0006] 1A is a schematic diagram illustrating a headset 10 or other structure including a multi-modal stimulator and microphones positioned relative to a subject's head 12, according to an embodiment. The illustrated embodiment includes multiple electrodes 141-148 (eight shown for illustrative purposes), a bone conduction transducer 16, multiple air conduction, pressure wave, or audio transducers 181-182 (two shown for illustrative purposes), and multiple microphones 201-208 (eight shown for illustrative purposes), all of which are attached to the subject's head 12 or attached to a band 22 configured to be positioned relative to the head 12. The sound localization, or position or direction, of sound perceived due to stimulation generated by the electrodes 141-148, bone conduction transducer 16, and / or audio transducers 181-182 may depend on factors such as the number and / or position of the electrodes, bone conduction transducers, or audio transducers. Thus, in other embodiments, other numbers and / or combinations of electrodes, such as 141-148, bone conduction transducers, such as 16, or audio transducers, such as 181-182, are provided. Similarly, in other embodiments, other numbers and / or combinations of microphones, such as 201-208, are provided. For example, in some embodiments, only two of electrodes, such as 141-148, are provided. In such embodiments, the two electrodes are positioned near the cochlea of ​​the subject's head 12 (e.g., near the mastoid process).

[0007] In the embodiment shown in FIG. 1A , for example, electrodes 141-148 are positioned on band 22 so as to be located at proximal positions corresponding to 12:00, 1:30, 3:00, 4:30, 6:00, 7:30, 9:00, and 10:30 relative to the subject's head 12. Electrodes 143 and 147 are positioned proximal to the mastoid processes on the right and left sides of the subject's head 12, thereby optimizing or enhancing the ability of the electrical stimulation provided by the electrodes to stimulate the right and left cochlea of ​​the subject, respectively. Electrodes 141 and 145 are positioned proximal to the anterior and posterior portions of the subject's head 12 (e.g., at the ends of midline 24 through the subject's head). As will be described in more detail below, electrodes 141-148 generate electrical signals (schematically shown at 26 in FIG. 1A ) that stimulate the subject's cochlea to produce an auditory perception. Electrodes 141-148 may be conventional or well-known devices suitable for the functions and applications described herein. Non-limiting examples include surface electrodes commonly used for stimulating and recording electrical signals. While described in this disclosure as a device for providing transcutaneous electrical stimulation, electrodes such as 141-148 may also be used in other device embodiments.

[0008] The bone conduction transducer 16 is positioned near the front of the subject's head 12 (e.g., at the midline 24 of the subject's forehead). As will be described in more detail below, the bone conduction transducer 16 generates physical movement or other vibratory motion (schematically shown at 28 in FIG. 1A ) that vibrates the skull and / or other anatomical structures of the subject's head 12, thereby producing an auditory perception. The bone conduction transducer 16 may be any conventional or well-known device suitable for the functions and applications described herein. Non-limiting examples include devices that generate mechanical vibrations or oscillations.

[0009] Audio transducers 181-182 produce hearing or the perception of hearing by generating audible air-conducted pressure waves (schematically shown at 30 in FIG. 1A ) that vibrate the middle ear, tympanic membrane, and / or other anatomical structures of the ear or auditory system of the subject's head 12. Audio transducers 181-182 are positioned proximate to the subject's ear canal to optimize hearing through the ear canal. Audio transducers 181-182 may be conventional or well-known devices suitable for the functions and applications described herein. Non-limiting examples include audio speakers of the type used in hearing aids.

[0010] The microphones 201-208 are positioned on the band 22 so as to be located at proximate positions corresponding to 12:00, 1:30, 3:00, 4:30, 6:00, 7:30, 9:00, and 10:30 on the subject's head 12. The microphones 201-208 are positioned, oriented, or configured (e.g., structured to have directional characteristics) to receive sound from defined directions or positions relative to the subject's head 12. The microphones 201-208 may be conventional or well-known devices suitable for the functions and applications described herein.

[0011] FIG. 1B is a schematic diagram of an exemplary alternative embodiment of a headset 10′. As shown, the headset 10′ includes a plurality of electrodes 141′-148′, two bone conduction transducers 161′ and 162′, a plurality of audio transducers 181′-182′, and a plurality of microphones 201′-208′, all of which are attached to a band 22′ configured to be worn on or positioned relative to a subject's head 12. In the embodiment shown in FIG. 1B, the two bone conduction transducers 161′ and 162′ are positioned on the band 22′ such that they are positioned at adjacent positions corresponding to 3:00 and 2:00 on the subject's head 12, respectively. Other than the different number and locations of bone conduction transducers 161' and 162', headset 10', including electrodes 141'-148', audio transducers 181'-182', and microphones 201'-208', is the same as or similar to the configuration described with reference to FIG. 1A and is located in the same or nearby locations. Similar features are labeled with similar numbers in FIGS. 1A and 1B. In this disclosure, the term "stimulator" refers to one or more or all of stimulators 141-148, 141'-148', 16, 161', 162', 181-182, and 181'-182', for example.

[0012] FIG. 2 is a schematic diagram illustrating functional components of a control system 40, according to some embodiments. The control system 40 is used in conjunction with a stimulator, such as those described with reference to FIGS. 1A and 1B, to provide a subject with multimodal auditory sensory stimulation. The illustrated embodiment of the control system 40 includes a compensation training component 42, a compensation data component 44, an electrical stimulus generation component 46, a bone conduction stimulus generation component 48, an air conduction stimulus generation component 50, and an operator control device 52. Throughout this disclosure, the term "stimulus generation component" is used to refer to one or more or all of the stimulus signal generation components, such as the electrical stimulus generation component 46, the bone conduction stimulus generation component 48, and the audio stimulus generation component 50. Multichannel audio signals representing or defining multichannel audio, such as audio generated by microphones 201-208 or 201'-208', are coupled to inputs of the control system 40 and various components of the control system. In the embodiment of FIG. 2, for example, audio signals representing one or more channels of audio are coupled to each of compensation data component 44, electrical stimulus generation component 46, bone conduction stimulus generation component 48, and audio stimulus generation component 50. While multi-channel audio signals are described in this disclosure as being received from microphones 201-208 or 201'-208' (e.g., representing audio from the ambient or local environment of the subject using headset 10 or 10'), multi-channel audio signals may be received from other audio sources. Non-limiting examples of such other audio sources include audio sources remote from the subject but generated in real time by stimuli provided by control system 40 (e.g., audio from an event the subject is watching live or remotely via media such as a video screen) and / or recorded audio (e.g., audio provided in connection with a video or other media). In such other embodiments, for example, headsets 10 and 10' may not have microphones 201-208 or 201'-208'.

[0013] As will be described in more detail below, certain embodiments of control system 40 use compensation data in connection with the generation of multimodal stimuli to improve the quality and characteristics of the audio or hearing perception, e.g., sound image localization, produced by the system. Briefly, the compensation data controls or modifies how audio represented by the multichannel audio signal is effectively mapped or distributed to the multimodal stimulator. Examples include compensation to control the perceived level of hearing (e.g., when the subject's hearing level perception characteristics with respect to direction are asymmetric) or compensation to control the directional sound image localization (e.g., when the subject perceives audio as coming from a direction different from the direction of the corresponding audio channel). The compensation data stored in compensation data component 44 is coupled to one or more of electrical stimulus generation component 46, bone conduction stimulus generation component 48, and audio stimulus generation component 50. As will be described in more detail below, compensation training component 42 is used in connection with the compensation training process to generate the compensation data stored by compensation data component 44. Although shown as part of control system 40 in the embodiment of FIG. 2, in other embodiments, compensation training component 42 may be separate from control system 40, and in such embodiments, compensation data generated by a separate compensation training component 44 may be imported into the compensation data component.

[0014] The controller 52 includes conventional or well-known interface components that an operator uses to operate the control system 40 to perform the functions described herein. Non-limiting examples include graphical user interface (GUI) components such as knobs, buttons, switches, or displays that are used to select functions or information, control levels, provide information (e.g., visual information) to a clinician or other user, etc.

[0015] The electrical stimulation generation component 46 generates a multi-channel electrical stimulation signal based on the multi-channel audio signal. In some embodiments, the electrical stimulation signal generation component 46 generates the multi-channel electrical stimulation signal based on both the multi-channel audio signal and the compensation data. The multi-channel electrical stimulation signal is configured to be coupled to two or more electrodes, such as 141-148 or 141'-148'. The electrical stimulation generation component 46 generates the electrical stimulation signal having a form and characteristics configured such that the electrodes electrically stimulate the cochlea of ​​the subject's head 12, thereby causing the subject to perceive sound or hearing.

[0016] The multi-channel stimulation provided by the electrical stimulation generation component 46 can provide a user with a sound image localization auditory perception (e.g., a sense of direction). In particular, the multi-channel electrical stimulation signals, when applied to particular electrodes, have a form and characteristics (e.g., level, phase, etc.) that create an auditory perception of sound originating from a particular location or direction around the subject's head 12 (e.g., 12:00, 3:00, 6:00, and / or 9:00 positions, above, below, or other positions or directions relative to a sphere centered on the subject's head 12). For example, the electrical generation component 46 generates the multi-channel electrical stimulation signals based on multi-channel audio signals received from two or more microphones 201-208 or two or more microphones 201'-208'. The signals are applied to two or more electrodes, such as 141-148, or two or more electrodes 141'-148', such that the audio perceived by the subject based on the electrical stimulation is perceived to originate from a location corresponding to the location and / or level of the audio received by the microphones.

[0017] The electrical stimulus generation component 46 can be configured to generate electrical stimulation signals by conventional or well-known methods. Such electrical stimulation approaches are sometimes referred to as galvanic vestibular stimulation (GVS) or cochlear stimulation. For example, U.S. Patent No. 3,267,931 to PUHARICH, U.S. Patent No. 3,766,331 to ZINK, and the 1984 paper "HIGH FREQUENCY AUDIOMETRY" by Tondorf et al. describe approaches that modulate a carrier signal to generate an electrical stimulation signal that, when applied to the cochlea via electrodes, generates an electrical signal that stimulates auditory perception. The patents to PUHARICH and ZINK and the paper by Tondorf et al. are incorporated herein by reference. Such electrical stimulation methods typically stimulate the perception of sounds over a relatively wide range of frequencies, such as 100 Hz to 20,000 Hz, corresponding to most or all of the full range of typical hearing.

[0018] Embodiments of the bone conduction stimulus generation component 48 generate a mono-channel or single-channel bone conduction stimulus signal based on one or more channels of the multi-channel audio defined by the multi-channel audio signal. In some embodiments, the bone conduction stimulus generation component 48 generates the mono-channel bone conduction stimulus signal based on both the audio signal and the compensation data. For example, when used in combination with a headset such as 10, the mono-channel bone conduction stimulus signal is configured to be coupled to the bone conduction transducer 16. For example, when used in combination with a headset such as 10', the mono-channel bone conduction stimulus signal is configured to be coupled to one or both of the bone conduction transducers 161' and 162'. The bone conduction stimulus generation component 48 generates a bone conduction stimulus signal having a form and characteristics configured to cause the bone conduction transducer to vibrate the skull of the subject's head 12, thereby producing a hearing sensation in the subject. The bone conduction stimulation provided to the subject by the bone conduction stimulation signal generated by the bone conduction stimulation generating component 48, when provided together (e.g., simultaneously or in combination) with one or both of the multi-channel electrical stimulation generated by the electrical stimulation generating component 46 or the audio stimulation generated by the audio stimulation generating component 50, can improve the subject's overall sound perception, including the perceived sound image localization.

[0019] In some embodiments, the bone conduction stimulus generation component 48 generates a multi-channel bone conduction stimulus signal based on two or more channels of multi-channel audio defined by the multi-channel audio signal. In some embodiments, the bone conduction stimulus generation component 48 generates a multi-channel bone conduction stimulus signal based on both the multi-channel audio signal and the compensation data. For example, when used in combination with a headset such as 10′, the multi-channel bone conduction stimulus signal is configured to be coupled to bone conduction transducers 161′ and 162′. The bone conduction stimulus generation component 48 generates a bone conduction stimulus signal having a form and characteristics configured to cause the bone conduction transducers to vibrate the skull of the subject's head 12, thereby producing an auditory perception in the subject.

[0020] The multi-channel stimulation provided by the bone conduction stimulation generating component 48 can provide a subject with a sound image localization auditory perception. In particular, the multi-channel bone conduction stimulation signal has a form and characteristics that, when applied to a particular transducer, produces an auditory perception of the sound originating from a particular location or direction around the subject's head 12 (e.g., 12:00, 3:00, 6:00, and / or 9:00, above, below, or other location or direction relative to a sphere centered on the subject's head 12). For example, the bone conduction generating component 48 generates the multi-channel bone conduction stimulation signal based on multi-channel audio signals received from two or more microphones 201'-208'. The signals are applied to transducers such as 161' and 162' such that the audio perceived by the subject based on the bone stimulation is perceived to originate from a location, direction, or level corresponding to the location, direction, or level of the audio received by the microphones. It has been observed that, in at least some instances, the sound image localization capabilities (e.g., direction and level) provided by multi-channel bone conduction stimulation may be less than those provided by electrical or audio stimulation methods. However, the multi-channel bone conduction stimulation provided to the subject by the bone conduction stimulation signals generated by bone conduction stimulation generating component 48, when provided in conjunction with or in combination with either or both of the multi-channel electrical stimulation generated by electrical stimulation generating component 46 or the audio stimulation generated by audio stimulation generating component 50, can improve the subject's overall sound perception, including the perceived sound image localization.

[0021] The bone conduction stimulus generating component 48 can be configured to generate the bone conduction stimulus signal by conventional or known methods. For example, the bone conduction stimulus signal may be frequency-modulated to correspond to or represent a desired sound perception provided to the subject. In some embodiments, the bone conduction stimulus signal may be amplitude-modulated, frequency-modulated, or tuned. For example, the following documents describe various bone conduction stimulation approaches that can be used in connection with the bone conduction stimulus generating component 48. These documents are incorporated herein by reference in their entirety: SHIRAISHI, K., "SOUND LOCALIZATION AND LATERALIZATION BY BILATERAL BONE CONDUCTION DEVICES, MIDDLE EAR IMPLANTS AND CARTILAGE CONDUCTION HEARING AIDS," AUDIOLOGY RESEARCH 2021, 11, 508-523. STANLEY, R. et al., "Lateralization of Sounds Using Bone-Conducted Headsets," Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting 2006. DAGA, K. et al., "Bone-Conducted Sound Lateralization of Interaural Time Difference and Interaural Intensity Difference in Children and a Young Adult with Bilateral Microtia and Atresia of the Ears," Acta Otolaryngol, 2001, 121, 274-277. These types of bone conduction stimulation methods can produce auditory perception in a lower frequency range than the higher frequency range typically produced by electrical stimulation methods. For example, in a particular subject, the most effective frequency range for bone conduction stimulation is typically between about 250 Hz and about 6,000 Hz.

[0022] Some embodiments of the audio stimulus generation component 50 generate a mono-channel air-conducted stimulus signal based on one or more channels of multi-channel audio defined by the multi-channel audio signal. In some embodiments, the audio stimulus generation component 50 generates a mono-channel audio stimulus signal based on both the audio signal and the compensation data. In particular embodiments, the audio stimulus generation component 50 generates a multi-channel audio stimulus signal based on multiple channels of audio defined by the multi-channel audio signal. In some embodiments, the audio stimulus generation component 50 generates a multi-channel audio stimulus signal based on both the audio signal and the compensation data. For example, when used in conjunction with a headset such as 10 or 10′, the mono-channel or multi-channel audio stimulus signal is configured to be coupled to one or both of the audio transducers 181-182 or one or both of the audio transducers 181′-182′. The audio stimulus generation component 50 generates an audio stimulus signal in a form and with characteristics configured to provide hearing to the subject by having the audio transducers 181-182 or 181′-182′ generate audible sound via air-conducted pressure waves. The audio stimulation provided to the subject by the audio stimulation signal generated by the audio stimulation generation component 50 can improve the overall sound perception by the subject when provided together with or simultaneously with one or both of the multi-channel electrical stimulation generated by the electrical stimulation generation component 46 and the mono-channel or multi-channel bone conduction stimulation generated by the bone conduction stimulation generation component 48.

[0023] The multi-channel stimuli provided by the audio stimulus generator 50 can provide a user with a sound image localization auditory perception. In particular, the multi-channel audio stimulus signals have a form and characteristics that, when applied to particular audio transducers, create an auditory perception of sound originating from a particular location or direction around the subject's head 12 (e.g., 12:00, 3:00, 6:00, and / or 9:00 positions, above, below, or other positions or directions relative to a sphere centered on the subject's head 12). For example, the audio stimulus generation component 50 generates the multi-channel audio stimulus signals based on multi-channel audio signals received from two or more microphones 201-208 or 201'-208'. The signals are applied to transducers such as 181 and 182 or 181' and 182' such that the sound perceived by the subject based on the audio stimuli is perceived to originate from a location, direction, and / or level corresponding to the location, direction, and / or level of the sound received by the microphones.

[0024] The audio stimulus generation component 50 can be configured to generate the audio stimulus signal using conventional or known approaches. For example, the audio stimulus signal may be frequency-modulated, amplitude-modulated, or tuned to correspond to the frequency and / or level of the desired sound perception provided to the subject. Non-limiting examples include approaches used in conventional or known hearing aids. Such audio stimulus methods typically provide stimuli over a relatively wide range of frequencies, such as 100 Hz to 20,000 Hz, that correspond to most or all of the full range of typical hearing.

[0025] The embodiment of control system 40 shown in FIG. 2 includes drivers 56, 58, and 60 that couple electrical stimulation generating component 46, bone conduction stimulation generating component 48, and audio stimulation generating component 50 to corresponding stimulation devices, such as electrical stimulation electrodes 141-148 or 141′-148′, bone conduction stimulation transducers 16, 161′, or 162′, or audio stimulation transducers 181-182 or 181′-182′, respectively. Drivers 56, 58, or 60 convert signals generated by the respective signal generating components 46, 48, or 50 to levels suitable for application to the corresponding stimulation devices. For example, in some embodiments, drivers 56, 58, or 60 include transformers or other components for converting signals generated by each stimulation generating component 46, 48, or 50 to current and / or voltage and / or impedance levels suitable for application to the stimulation devices.

[0026] Compensation by signal generation components 46, 48, and / or 50 can correct (e.g., at least partially) for variations in the characteristics of hearing perceived by a subject. For example, a given level of stimulation applied to one or more stimulators on one side (e.g., the right side) of the subject may result in hearing being perceived at a different level (e.g., loudness) than the same level of stimulation applied to one or more stimulators on the other side (e.g., the left side) of the subject. Similarly, such asymmetric level perception will vary from subject to subject. Compensation data stored in compensation data component 44 can be used by signal generation components 46, 48, or 50 to account for these differences, such that the perceived level of hearing correlates with a “desired” level, e.g., the level represented by the audio signal input to control system 40. As another example, a particular type of stimulus applied to one or more specific stimulators of a first subject may be perceived as originating from a first location or direction, while the same type of stimulus applied to one or more identical or similar stimulators of a second subject may be perceived as originating from a second location or direction that differs from the first location or direction of the first subject. The compensation data stored by compensation data component 44 can be used by signal generation components 46, 48, or 50 to account for these differences, such that the perceived auditory location or direction correlates with a “desired” location or direction, e.g., a location or direction represented by an audio signal input to control system 40. For example, in embodiments, the compensation data of compensation data component 44 is used by signal generation components 46, 48, or 50 to map, transform, or correlate audio information (e.g., location, direction, or level) defined by the multi-channel audio signal to the stimuli generated by the signal generation components. The generated stimuli then provide an audio or auditory perception that corresponds to the audio information defined by the multi-channel audio signal.

[0027] As an example, Figure 3 shows the variability in the location and direction (e.g., angle) of sound perception for 11 different subjects when the same combination of mono-channel bone conduction stimulation and multi-channel electrical stimulation was applied to a stimulator placed at the same position on the head of each subject. The results shown in Figure 3 were obtained with mono-channel bone conduction stimulation applied to a bone conduction transducer located near the subject's forehead, anterior to the midline, and electrical stimulation applied to electrodes near the subject's left and right mastoid processes. For electrical stimulation, signals were applied to the right and left electrodes in both right-to-left current directions (anode to cathode) and left-to-right current directions. For the right-to-left current direction, sound was perceived on the left side of all subjects, and for the left-to-right current direction, sound was perceived on the right side of all subjects. As shown in Figure 3, for the same stimulation, different sound source directions or locations were perceived on the right and left sides of the midline, including both anterior and posterior directions of the coronal midline. Generally, when bone conduction stimulation was applied alone, the subjects perceived the resulting hearing to be located at various locations within a region roughly centered between the front, back, right, and left sides of the head. When electrical stimulation was applied alone from right to left, the subjects perceived the hearing to be located at various locations on the left side of the head. When electrical stimulation was applied alone from left to right, the subjects perceived the hearing to be located at various locations on the right side of the head.

[0028] The protocol used to generate the information shown in Figure 3 included stimulation at three different frequencies (2000 Hz, 4000 Hz, and 8000 Hz) configured to produce auditory perception. At each frequency, bone conduction stimulation alone (e.g., without electrical stimulation) was applied at various levels to determine the minimum or threshold level at which the subject could perceive the stimulation (e.g., the subject informed the clinician when auditory perception became possible). This determined the bone conduction stimulation threshold level at each frequency for each subject. At each frequency, electrical stimulation alone (e.g., without bone conduction stimulation) was applied at various levels with a right-to-left current direction to determine the minimum or threshold level at which the subject could perceive the stimulation (e.g., the subject informed the clinician when auditory perception became possible). Similarly, at each frequency, electrical stimulation alone was applied at various levels with a left-to-right current direction to determine the minimum or threshold level at which the subject could perceive the stimulation. This determined the electrical stimulation threshold level at each frequency and for both current directions for each subject. Next, each subject was subjected to simultaneous application of bone conduction and electrical stimulation with a right-to-left current direction, and simultaneous application of bone conduction and electrical stimulation with a left-to-right current direction, at a level a predetermined amount (e.g., +10 dB) above the determined threshold, and sound localization was assessed for each frequency. During the assessment, the subject reported the location and direction of the perceived hearing to the clinician.

[0029] In the method described above, compensation data for each subject can be obtained by using the compensation training component 42 of the control system 40. For example, a clinician can use the control device 52, including a display, to stimulate the subject with each stimulator at each of a plurality of characteristics, thereby causing the subject to perceive hearing at a plurality of different frequencies. The clinician then determines the threshold level for each stimulator at each frequency. The clinician can use the control device 52 to stimulate the subject with each stimulator at each of a plurality of characteristics (e.g., direction of current in the case of an electrical stimulator), thereby causing the subject to perceive hearing at a plurality of different frequencies. The clinician then determines the position or orientation of each stimulator at each frequency. The compensation data can be specific to each subject. Additionally or alternatively, the compensation data can correspond to an average of the characteristics for a group of subjects.

[0030] The stimulus generator generates a stimulus signal based on the multi-channel audio signal and the compensation data, such that when the stimulus signal is applied to the stimulus generator, audio having desired characteristics, such as position, direction, and / or level, is perceived. For example, in embodiments, the stimulus generator may generate a stimulus signal based on the multi-channel audio signal received by microphones 201-208 or 201'-208' such that the sound perceived by the subject corresponds to the position, direction, and level defined by the multi-channel audio signal. For example, in embodiments, the multi-channel audio signal may be effectively calculated and / or otherwise processed using the compensation data to generate the stimulus signal such that a desired auditory effect is perceived.

[0031] FIG. 4 is a schematic diagram of an exemplary computer system 138 that can be used to implement functional components of control system 40, including compensation training component 42, compensation data component 44, electrical stimulus generation component 46, bone conduction stimulus generation component 48, audio stimulus generation component 50, and operator control device 52, according to some embodiments. The illustrated embodiment of computer system 138 is comprised of a processing component 152, a storage component 154, a network interface component 156, and a user interface component 158, coupled by a system network or bus 159. Processing component 152, for example, includes a central processing unit (CPU) 160 and a graphics processing unit (GPU) 162, and provides processing functionality for the stimulus generation component. Storage component 154, for example, includes RAM memory 164 and hard disk / SSD memory 166, and provides storage functionality for compensation data component 44. For example, operating system software used by processing component 152 to implement methods described herein may be stored in storage component 154. In some embodiments, the network interface component includes one or more web servers 170 and one or more application programming interfaces (APIs) 172. Examples of user interface components 158 include a display 174, a keypad 176, a graphical user interface (GUI) 178, etc. Embodiments of computer system 138 may also include other conventional or well-known components for implementing methods in accordance with embodiments described herein.

[0032] A first example includes a system having a headset and a control system. Some embodiments include a multi-mode auditory stimulation system including an audio signal input configured to receive an audio signal representing multi-channel audio, at least two electrical stimulation electrodes (e.g., including an anode and cathode pair) configured to be positioned at spaced locations relative to a subject's head, at least one bone conduction stimulation transducer configured to be positioned relative to the subject's head, and a control system having at least one processor connected to the audio signal input, the at least two electrical stimulation electrodes, and the at least one bone conduction stimulation transducer, wherein the control system generates (1) a multi-channel electrical stimulation signal configured to generate, via the at least two electrical stimulation electrodes, an electrical signal that stimulates the subject's auditory perception, including sound image localization of the auditory perception, based on the audio signal, and (2) a bone conduction stimulation signal configured to generate, via the at least one bone conduction stimulation transducer, a vibration that stimulates the subject's auditory perception, based on the audio signal.

[0033] In some embodiments of the first example, the control system generates a multi-channel electrical stimulation signal including one or both of a level difference and a phase difference. In any or all of the above embodiments, the at least two electrical stimulation electrodes include first and second electrodes configured to be positioned near the subject's right and left mastoid processes (e.g., at 3:00 and 9:00 positions relative to the subject's head), respectively. Some embodiments further include first and second microphones connected to the audio signal input, the first and second microphones configured to be positioned near the subject's right and left mastoid processes (e.g., at 3:00 and 9:00 positions relative to the subject's head), respectively.

[0034] In any or all of the first example embodiments, the at least two electrical stimulation electrodes include two electrodes configured to be positioned at 12:00 and 6:00 positions relative to the subject's head, respectively. Some embodiments further include two microphones connected to the audio signal input and configured to be positioned at 12:00 and 6:00 positions relative to the subject's head, respectively.

[0035] In any or all of the first example embodiments, the at least two electrical stimulation electrodes include two electrodes configured to be positioned at 1:30 and 7:30 positions relative to the subject's head, respectively. Some embodiments further include two microphones connected to the audio signal input and configured to be positioned at 1:30 and 7:30 positions relative to the subject's head, respectively.

[0036] In any or all of the first example embodiments, the at least two electrical stimulation electrodes include two electrodes configured to be positioned at 4:30 and 10:30 positions relative to the subject's head, respectively. Some embodiments further include two microphones connected to the audio signal input and configured to be positioned at 4:30 and 10:30 positions relative to the subject's head, respectively.

[0037] In any or all of the first example embodiments, the at least one bone conduction transducer is configured to be positioned along the midline relative to the subject's head. In any or all of the first example embodiments, the at least one bone conduction stimulation transducer includes first and second bone conduction stimulation transducers configured to be positioned at spaced apart positions relative to the subject's head, and the control system generates, by the first and second bone conduction stimulation transducers, a multi-channel bone conduction stimulation signal configured to generate vibrations that stimulate the subject's auditory perception, including sound image localization of the auditory perception, based on an audio signal. In some embodiments, the first and second bone conduction stimulation transducers are configured to be positioned on opposite sides of the midline relative to the subject's head.

[0038] In any or all of the first example embodiments, the control system generates a multi-channel bone conduction signal that includes one or both of a level difference or a phase difference. Any or all of the first example embodiments further comprise a mounting structure, optionally a headband, configured to be worn on the subject's head, with at least two electrical stimulation electrodes and at least one bone conduction stimulation transducer coupled to the mounting structure.

[0039] In any or all of the first example embodiments, the control system (1) causes the multi-channel electrical stimulation signal to generate electrical signals that stimulate hearing at frequencies of 8 kHz or greater, optionally 15 kHz or greater, and (2) causes the bone conduction stimulation signal to generate vibrations that stimulate hearing at frequencies less than 8 kHz.

[0040] In any or all of the first example embodiments, the control system causes the multi-channel electrical stimulation signal to generate electrical signals that stimulate hearing in a frequency range that includes frequencies higher than the hearing frequency range caused by vibrations produced by the bone conduction stimulation signal.

[0041] In any or all of the first example embodiments, the control system further includes a memory that stores compensation information that characterizes one or both of the location and level of the subject's sound perception based on (1) characteristics of the multi-channel electrical stimulation signal, or (2) which of the at least two electrical stimulation electrodes the multi-channel electrical stimulation signal is coupled to, (3) characteristics of the bone conduction stimulation signal, or (4) which of the at least one bone conduction stimulation transducer the bone conduction stimulation signal is coupled to, and the control system generates one or both of (1) a multi-channel electrical stimulation signal based on the sound signal and the compensation information, or (2) a bone conduction stimulation signal based on the sound signal and the compensation information.

[0042] In any or all of the first example embodiments, the system further includes one or two audio transducers, each configured to be positioned relative to an ear of the user's head, and a control system coupled to the one or two audio transducers and configured to generate a multi-channel audio stimulus signal based on an audio signal received at the audio signal input, the control system being configured to generate air-conducted pressure wave sounds by the one or two audio transducers, including sound image localization of the audible sound.

[0043] A second example is a programmed control system that does not include a headset. Some embodiments of the second example include a control system programmed to provide the functionality of any or all of the control system embodiments of the first example (e.g., excluding other components such as one or more electrical stimulation electrodes, at least one bone conduction stimulation transducer, a microphone, one or two audio transducers, and a wearing structure).

[0044] A third example is software for a control system. Some embodiments of the third example include a non-transitory information storage medium containing programmed instructions for causing the control system to perform the functions of any embodiment of the control system of the first example.

[0045] A fourth example is a method for stimulating auditory perception. Some embodiments of the fourth example include a method for stimulating auditory perception in a subject. The method includes receiving an audio signal representing multi-channel audio; generating, based on the audio signal, a multi-channel electrical stimulation signal configured to generate an electrical signal when coupled to at least two electrical stimulation electrodes (e.g., including a cathode and an anode pair) positioned at spaced locations relative to the subject's head, causing a stimulation of the auditory perception in the subject, including sound image localization of the auditory perception; and generating, based on the audio signal, a bone conduction stimulation signal configured to produce a vibration when coupled to at least one bone conduction stimulation transducer positioned relative to the subject's head, causing a stimulation of the auditory perception in the subject.

[0046] In some embodiments of the fourth example, generating the multi-channel electrical stimulation signal includes generating the multi-channel electrical stimulation signal that includes one or both of a level difference or a phase difference.

[0047] In any or all of the fourth example embodiments, generating the multi-channel electrical stimulation signal includes generating the signal configured for first and second electrodes positioned near the subject's right and left mastoid processes (e.g., at 3:00 and 9:00 positions relative to the subject's head), respectively. In some embodiments, receiving the audio signal includes receiving a signal representing multi-channel audio from first and second microphones positioned near the subject's right and left mastoid processes (e.g., at 3:00 and 9:00 positions relative to the subject's head), respectively.

[0048] Any or all of the fourth example embodiments may further include generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at 12:00 and 6:00 positions relative to the subject's head, respectively. In some embodiments, receiving an audio signal includes receiving a signal representing multi-channel audio from two microphones positioned at 12:00 and 6:00 positions relative to the subject's head, respectively.

[0049] Any or all of the fourth example embodiments may further include generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at 1:30 and 7:30 positions relative to the subject's head, respectively. In some embodiments, receiving an audio signal includes receiving a signal representing multi-channel audio from two microphones positioned at 1:30 and 7:30 positions relative to the subject's head, respectively.

[0050] Any or all of the fourth example embodiments may further include generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at 4:30 and 10:30 positions relative to the subject's head, respectively. In some embodiments, receiving the audio signal includes receiving a signal representing multi-channel audio from two microphones positioned at 4:30 and 10:30 positions relative to the subject's head, respectively.

[0051] In any or all of the fourth example embodiments, generating the bone conduction stimulation signal includes generating the bone conduction stimulation signal configured for a bone conduction transducer positioned along the midline relative to the subject's head.

[0052] In any or all of the fourth example embodiments, generating the bone conduction stimulation signal includes generating, by a bone conduction stimulation transducer, a multi-channel bone conduction stimulation signal configured to generate vibrations that stimulate the subject's auditory perception, including sound image localization of the auditory perception, based on the audio signal. In some embodiments, generating the multi-channel bone conduction stimulation signal includes generating, by first and second bone conduction stimulation transducers positioned on opposite sides of the midline of the subject's head, multi-channel bone conduction stimulation signals configured to produce vibrations. In some embodiments, generating the multi-channel bone conduction signal includes generating a multi-channel bone conduction signal that includes one or both of a level difference or a phase difference.

[0053] In any or all of the fourth example embodiments, (1) generating an electrical stimulation signal includes generating a multi-channel electrical stimulation signal to generate an electrical signal that stimulates hearing at a frequency of 8 kHz or greater, and (2) generating a bone conduction stimulation signal includes generating a bone conduction stimulation signal to generate vibrations that stimulate hearing at a frequency of less than 8 kHz.

[0054] Any or all of the fourth example embodiments may include generating a multi-channel electrical stimulation signal to generate an electrical signal that stimulates hearing in a frequency range that includes frequencies higher than the hearing frequency range caused by vibrations produced by the bone conduction stimulation signal.

[0055] In any or all of the fourth example embodiments, (1) generating a multi-channel electrical stimulation signal includes generating a multi-channel electrical stimulation signal based on an audio signal and electrical stimulation sound image localization compensation information, and the electrical stimulation sound image localization compensation information includes information characterizing one or both of the location or level of the subject's sound perception based on one or more of (I) characteristics of the multi-channel electrical stimulation signal, or (II) to which of the at least two electrical stimulation electrodes the multi-channel electrical stimulation signal is coupled, or (2) generating a bone conduction stimulation signal includes generating a bone conduction stimulation signal based on the audio signal and bone conduction sound image localization compensation information, and the bone conduction sound image localization compensation information includes information characterizing one or both of the location or level of the subject's sound perception based on one or more of (III) characteristics of the bone conduction stimulation signal, or (IV) to which of the at least one bone conduction stimulation transducer the bone conduction stimulation signal is coupled.

[0056] A fifth example is a programmed control system. A fifth example embodiment includes a control system programmed to perform any or all of the methods of the fourth example embodiment.

[0057] A sixth example is software. A sixth example embodiment comprises a non-transitory information storage medium containing programmed instructions for causing one or more processors to perform any or all of the methods of the fourth example embodiment.

[0058] A seventh example is a compensation training method. An embodiment of the seventh example includes a method for generating compensation information for use in connection with a multi-modal auditory stimulation system having a bone conduction stimulation transducer positioned relative to a subject's head and at least two electrical stimulation electrodes (e.g., including a cathode and an anode pair) positioned at spaced locations relative to the user's head, the method including applying both bone conduction and electrical stimulation to the subject, and receiving and storing information representing a direction in which the subject perceived sound perceptions resulting from the bone conduction and electrical stimulation.

[0059] In a seventh example embodiment, applying the bone conduction stimuli and the electrical stimuli together includes providing bone conduction stimuli and electrical stimuli representative of sound perception at a first frequency. In some embodiments, applying the bone conduction stimuli and the electrical stimuli together includes providing bone conduction stimuli and electrical stimuli representative of sound perception at each of a plurality of frequencies, and receiving and storing as compensation information includes receiving and storing as compensation information information representative of a direction in which the subject perceived the sound perception provided by the bone conduction stimuli and electrical stimuli at each of the plurality of frequencies.

[0060] In any or all of the seventh example embodiments, applying the electrical stimulation includes applying the electrical stimulation in both a first direction and a second direction between at least two electrical stimulation electrodes at each frequency, and receiving and storing as compensation information includes receiving information representing the direction in which the subject perceives the sound perception brought about by the electrical stimulation in both the first direction and the second direction at each frequency and storing as compensation information.

[0061] Any or all of the seventh example embodiments may further include applying varying levels of bone conduction stimulation to the subject, receiving and storing as compensation information bone conduction threshold levels representing the bone conduction stimulation threshold levels at which the subject perceives sound through the bone conduction stimulation, and applying electrical stimulation to the subject at varying levels, receiving and storing as compensation information electrical stimulation threshold levels representing the electrical stimulation threshold levels at which the subject perceives sound through the electrical stimulation.

[0062] An eighth example is a method of generating compensation information for use in connection with a multimodal auditory stimulation system including a bone conduction stimulation transducer positioned relative to a subject's head and at least two electrical stimulation electrodes (e.g., including a cathode and an anode pair) positioned at spaced locations relative to the user's head. Some embodiments include applying bone conduction stimulation and electrical stimulation to the subject, and receiving and storing as compensation information information representing a direction in which the subject perceived sound perceptions resulting from the bone conduction stimulation and electrical stimulation.

[0063] A ninth example is a method of stimulating auditory perception in a subject. Some embodiments include receiving an audio signal representing multi-channel audio, generating, based on the audio signal, a multi-channel electrical stimulation signal configured to generate an electrical signal, when coupled to at least two electrical stimulation electrodes positioned at spaced locations relative to the subject's head, that generates an electrical signal that generates a stimulation of auditory perception in the subject, including sound image localization of the auditory perception, and generating, based on the audio signal, an audio stimulation signal configured to generate air-conducted pressure wave sounds in the subject, when coupled to audio transducers positioned relative to the ears of the subject's head.

[0064] Some embodiments of the ninth example further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, including applying both electrical stimulation and pressure wave sounds to a subject, and receiving and storing as compensation information information representing the direction in which the subject perceived the sound perception produced by the electrical stimulation and pressure wave sounds.

[0065] In any or all of the ninth example embodiments, generating the audio stimulus signal includes generating a multi-channel audio stimulus signal configured to generate air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned at spaced locations relative to the ears of the subject's head.

[0066] Any or all of the ninth example embodiments further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, including applying both electrical stimulation and pressure wave sounds to a subject, and receiving and storing as compensation information information representing the direction in which the subject perceived the sound perception produced by the electrical stimulation and pressure wave sounds.

[0067] A tenth example is a method of stimulating an auditory perception of a subject. Some embodiments include receiving an audio signal representing multi-channel audio, generating a bone conduction stimulation signal based on the audio signal, the bone conduction stimulation signal configured, when coupled to at least one bone conduction stimulation transducer positioned relative to a head of the subject, to produce vibrations that generate a stimulation of the auditory perception in the subject, and generating an audio stimulation signal based on the audio signal, the bone conduction stimulation signal configured, when coupled to an audio transducer positioned relative to an ear of the head of the subject, to generate an air-conducted pressure wave sound in the subject.

[0068] Some embodiments of the tenth example further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, including applying both bone conduction stimulation and pressure wave sound to a subject, and receiving and storing as compensation information information representing the direction in which the subject perceived the sound perception produced by the bone conduction stimulation and pressure wave sound.

[0069] In any or all of the tenth example embodiments, generating the audio stimulus signal includes generating a multi-channel audio stimulus signal configured to generate air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned at spaced locations relative to the ears of the subject's head.

[0070] Any or all of the tenth example embodiments further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, including applying both bone conduction stimulation and pressure wave sound to a subject, and receiving and storing as compensation information information representing the direction in which the subject perceived the sound perception produced by the bone conduction stimulation and pressure wave sound.

[0071] Some embodiments of the eleventh example include a method for stimulating an auditory perception in a subject, the method comprising: receiving an audio signal representing multi-channel audio; generating, based on the audio signal, a multi-channel electrical stimulation signal configured to generate an electrical signal when coupled to at least two electrical stimulation electrodes positioned at spaced locations relative to the subject's head, the electrical signal generating an electrical stimulation of the auditory perception in the subject, the electrical stimulation including sound image localization of the auditory perception; and generating, based on the audio signal, a bone conduction stimulation signal configured to produce a vibration when coupled to at least one bone conduction stimulation transducer positioned relative to the subject's head, the bone conduction stimulation producing a vibration when coupled to at least one bone conduction stimulation transducer positioned relative to the subject's head, the bone conduction stimulation of the auditory perception in the subject.

[0072] Some embodiments of the eleventh example further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, including applying both electrical stimulation and bone conduction stimulation to a subject, and receiving and storing as compensation information information representing the direction in which the subject perceived the sound perception produced by the electrical stimulation and bone conduction stimulation.

[0073] Any or all of the embodiments of the eleventh example further include generating, based on the audio signal, an audio stimulation signal configured to generate an air-conducted pressure wave sound in the subject when coupled to an audio transducer positioned relative to an ear of the subject's head. Some embodiments further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, comprising applying electrical stimulation, bone conduction stimulation, and pressure wave sound together to the subject, and receiving and storing as compensation information information representing a direction in which the subject perceived the sound sensations produced by the electrical stimulation, bone conduction stimulation, and pressure wave sound.

[0074] In any or all of the eleventh example embodiments, generating the audio stimulus signal includes generating a multi-channel audio stimulus signal configured to generate air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned on the subject's head at spaced-apart positions relative to the ears. Some embodiments further include obtaining compensation information for use in connection with a multi-modal auditory stimulation system, comprising applying electrical stimuli, bone conduction stimuli, and pressure wave sounds to the subject together, and receiving and storing information representing the direction in which the subject perceived the sound perceptions produced by the electrical stimuli, bone conduction stimuli, and pressure wave sounds as compensation information.

[0075] The above description is illustrative and not limiting. Other embodiments will be apparent to those skilled in the art from the above description. It is contemplated that features described in connection with one embodiment may optionally be used in addition to, or in place of, features described or associated with another embodiment. The scope of the invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. 1. A method of stimulating auditory perception in a subject, comprising: receiving an audio signal representing multi-channel audio; generating, based on the audio signal, a multi-channel electrical stimulation signal configured to generate an electrical signal that, when coupled to at least two electrical stimulation electrodes positioned at spaced locations relative to the subject's head, generates an electrical stimulation of an auditory perception in the subject, including sound image localization of the auditory perception; generating, based on the audio signal, a bone conduction stimulation signal configured to produce vibrations that, when coupled to at least one bone conduction stimulation transducer positioned relative to the subject's head, generate bone conduction stimulation of an auditory perception in the subject.

2. obtaining compensation information for use in connection with the multi-modal auditory stimulation system; applying both the electrical stimuli and the bone conduction stimuli to the subject; 2. The method of claim 1, further comprising receiving information representing a direction in which the subject perceived the sound perception provided by the electrical stimulation and the bone conduction stimulation, and storing the information as compensation information.

3. 10. The method of claim 1, further comprising generating an audio stimulus signal based on the audio signal, the audio stimulus signal being configured to produce air-conducted pressure wave sounds in the subject when coupled to an audio transducer positioned relative to an ear of the subject's head.

4. 4. The method of claim 3, wherein generating the audio stimulus signal comprises generating a multi-channel audio stimulus signal configured to produce air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned on the subject's head at spaced-apart positions relative to the ears.

5. obtaining compensation information for use in connection with the multi-modal auditory stimulation system; applying the electrical stimulation, the bone conduction stimulation, and the pressure wave sound together to the subject; The method of claim 3 or 4, further comprising receiving information representing the direction in which the subject perceived the sound sensations brought about by the electrical stimulation, the bone conduction stimulation, and the pressure wave sound, and storing the information as compensation information.

6. 1. A method of stimulating auditory perception in a subject, comprising: receiving an audio signal representing multi-channel audio; generating, based on the audio signal, a multi-channel electrical stimulation signal configured to generate an electrical signal that, when coupled to at least two electrical stimulation electrodes positioned at spaced locations relative to the subject's head, generates an auditory perceptual stimulation in the subject, including sound image localization of the auditory percept; generating, based on the audio signal, an audio stimulus signal configured to produce air-conducted pressure wave sounds in the subject when coupled to an audio transducer positioned relative to an ear on the subject's head.

7. 7. The method of claim 6, wherein generating the audio stimulus signal comprises generating a multi-channel audio stimulus signal configured to produce air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned on the subject's head at spaced-apart positions relative to the ears.

8. obtaining compensation information for use in connection with the multi-modal auditory stimulation system; applying the electrical stimulus and the pressure wave sound together to the subject; The method of claim 6 or 7, further comprising receiving information representing the direction in which the subject perceived the sound sensation brought about by the electrical stimulation and the pressure wave sound, and storing the information as compensation information.

9. 1. A method of stimulating auditory perception in a subject, comprising: receiving an audio signal representing multi-channel audio; generating, based on the audio signal, a bone conduction stimulation signal configured to produce vibrations in the subject that generate an auditory perceptual stimulus in the subject when coupled to at least one bone conduction stimulation transducer positioned relative to the subject's head; generating, based on the audio signal, an audio stimulus signal configured to produce air-conducted pressure wave sounds in the subject when coupled to an audio transducer positioned relative to an ear on the subject's head.

10. 10. The method of claim 9, wherein generating the audio stimulus signal comprises generating a multi-channel audio stimulus signal configured to produce air-conducted pressure wave sounds in the subject, including sound image localization, when coupled to two or more audio transducers positioned on the subject's head at spaced-apart positions relative to the ears.

11. obtaining compensation information for use in connection with the multi-modal auditory stimulation system; applying the bone conduction stimulation and the pressure wave sound together to the subject; The method of claim 9 or 10, further comprising receiving information representing the direction in which the subject perceived the sound sensation brought about by the bone conduction stimulation and the pressure wave sound, and storing the information as compensation information.

12. 10. The method of claim 1, wherein generating the multi-channel electrical stimulation signal comprises generating a multi-channel electrical stimulation signal that includes one or both of a level difference or a phase difference.

13. 13. The method of claim 12, comprising generating a multi-channel electrical stimulation signal configured for first and second electrodes positioned near the subject's right and left mastoid processes, respectively (e.g., at the 3:00 and 9:00 positions relative to the subject's head).

14. 14. The method of claim 13, wherein receiving the audio signals comprises receiving signals representing multi-channel audio from first and second microphones positioned near the subject's right and left mastoid processes, respectively (e.g., at 3:00 and 9:00 positions relative to the subject's head).

15. 10. The method of claim 1, 3, or 6, comprising generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at the 12:00 and 6:00 positions, respectively, relative to the subject's head.

16. 16. The method of claim 15, wherein receiving the audio signals comprises receiving signals representing multi-channel audio from two microphones positioned at 12:00 and 6:00 positions, respectively, relative to the subject's head.

17. 10. The method of claim 1, 3, or 6, comprising generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at 1:30 and 7:30 positions, respectively, relative to the subject's head.

18. 20. The method of claim 17, wherein receiving the audio signals comprises receiving signals representing multi-channel audio from two microphones positioned at 1:30 and 7:30 positions, respectively, relative to the subject's head.

19. 10. The method of claim 1, comprising generating a multi-channel electrical stimulation signal configured for at least two electrical stimulation electrodes positioned at 4:30 and 10:30 positions, respectively, relative to the subject's head.

20. 20. The method of claim 19, wherein receiving the audio signals comprises receiving signals representing multi-channel audio from two microphones positioned at 4:30 and 10:30 positions, respectively, relative to the subject's head.

21. 10. The method of claim 1, wherein generating the bone conduction stimulation signal comprises generating a bone conduction stimulation signal configured for a bone conduction transducer positioned along a midline relative to the subject's head.

22. 10. The method of claim 1, wherein generating the bone conduction stimulation signal comprises generating, based on the audio signal, a multi-channel bone conduction stimulation signal by the bone conduction stimulation transducer, the multi-channel bone conduction stimulation signal being configured to generate vibrations that stimulate auditory perception, including sound image localization of auditory perception, in the subject.

23. 23. The method of claim 22, wherein generating the multi-channel bone conduction stimulation signal comprises generating the multi-channel bone conduction stimulation signal configured to produce vibrations through first and second bone conduction stimulation transducers positioned on opposite sides of the midline of the subject's head.

24. 23. The method of claim 22, wherein generating the multi-channel bone conduction signal comprises generating a multi-channel bone conduction signal that includes one or both of a level difference or a phase difference.

25. 10. The method of claim 1, wherein (1) generating the electrical stimulation signal includes generating a multi-channel electrical stimulation signal to generate an electrical signal that stimulates hearing at a frequency of 8 kHz or more, and (2) generating the bone conduction stimulation signal includes generating a bone conduction stimulation signal to generate vibrations that stimulate hearing at a frequency of less than 8 kHz.

26. 10. The method of claim 1, 3, or 6, comprising generating the multi-channel electrical stimulation signal to generate an electrical signal that stimulates hearing in a frequency range that includes frequencies higher than the hearing frequency range caused by the vibrations brought about by the bone conduction stimulation signal.

27. 10. The method of claim 1, wherein: (1) generating the multi-channel electrical stimulation signal comprises generating the multi-channel electrical stimulation signal based on the audio signal and electrical stimulation sound image localization compensation information, and the electrical stimulation sound image localization compensation information includes information characterizing one or both of a location or a level of the subject's sound perception based on one or more of (I) characteristics of the multi-channel electrical stimulation signal, or (II) which of the at least two electrical stimulation electrodes the multi-channel electrical stimulation signal is coupled to; or (2) generating the bone conduction stimulation signal comprises generating the bone conduction stimulation signal based on the audio signal and bone conduction sound image localization compensation information, and the bone conduction sound image localization compensation information includes information characterizing one or both of a location or a level of the subject's sound perception based on one or more of (III) characteristics of the bone conduction stimulation signal, or (IV) which of the at least one bone conduction stimulation transducers the bone conduction stimulation signal is coupled to.

Images