Implantable brain-ear collaborative interface microsystem, implantable component and hearing enhancement method

Abstract

The invention provides an implantable brain-ear collaborative interface microsystem, an implantable component and a hearing enhancement method, the system comprises the implantable component and an in-vitro component, the implantable component collects a neural signal indicating a visual attention direction in a brain and receives an auditory stimulation instruction from the in-vitro component, and the auditory stimulation instruction is sent to the in-vitro component. The pulse electrical stimulation signal is converted into a pulse electrical stimulation signal to stimulate a cochlea; the in-vitro assembly collects a plurality of sound source signals in an environment where a user is located, receives a neural signal sent by the implantable assembly, determines a target visual direction according to the neural signal, enhances a sound source in a corresponding direction, encodes an enhanced audio into an auditory stimulation instruction, and returns the auditory stimulation instruction to the implantable assembly. According to the method and the device, a technical path for guiding auditory signal processing according to visual attention related neural signals is realized, so that the generation of auditory stimulation is associated with a spatial direction concerned by a user.

Description

Technical Field

[0001] This application relates to the field of implantable medical device technology, specifically to an implantable brain-ear co-processor interface microsystem, implantable components, and a hearing enhancement method. Background Technology

[0002] Hearing loss is a major health problem affecting hundreds of millions of people worldwide. For patients with severe to profound sensorineural hearing loss, cochlear implants (CI) are currently the most effective and mature medical treatment for restoring hearing function. By implanting a cochlear implant electrode array into the patient's cochlea, the cochlear implant directly electricalally stimulates the auditory nerve, bypassing damaged hair cells and successfully restoring sound perception for tens of thousands of deaf and disabled patients. In quiet environments, cochlear implant systems can help most users achieve good speech recognition rates.

[0003] However, current cochlear implant technology still faces significant challenges in dealing with complex acoustic environments in the real world. In "cocktail party" scenarios such as family gatherings, noisy restaurants, and public transportation, multiple sound sources (including target speech, competing speech, and background noise) coexist simultaneously, causing a sharp decline in speech comprehension for cochlear implant users. This phenomenon stems from the fact that people with normal hearing can utilize the brain's auditory attention mechanisms to freely "focus" and "extract" sounds of interest from a mixed sound stream, while cochlear implant users largely lose this ability, resulting in limited effectiveness in multi-source scenarios.

[0004] To address this widely recognized challenge, existing technologies primarily explore two paths. The first path involves improving microphone technology and acoustic front-end processing. For example, multi-microphone array beamforming techniques are used to algorithmically enhance sound from a specific direction (usually directly in front) while suppressing noise from other directions. However, these solutions assume the user is only interested in sound directly in front, but in reality, users frequently need to switch listening targets. The second path involves developing more complex back-end signal processing algorithms, such as noise reduction techniques based on computational auditory scene analysis. These algorithms attempt to separate speech from noise by analyzing the statistical characteristics of sound (such as pitch and rhythm). While achieving some success in specific scenarios, their fundamental flaw lies in their inherent indiscriminate nature. When multiple speakers are present simultaneously, the algorithm cannot distinguish which is the user's target speaker and which is the interfering speaker that needs to be suppressed. Therefore, it often incorrectly suppresses the target speech or fails to effectively separate it, making it difficult to improve the listening experience.

[0005] In summary, the fundamental limitation of existing technologies lies in their attempt to solve a cognitive problem that is inherently closely related to the user's subjective intent by analyzing only external acoustic physical signals. These systems lack a reliable, real-time input source to inform the processor what the user "wants to hear," i.e., they lack decoding of the user's "auditory attention focus." Therefore, there is an urgent need in this field for a novel technical solution that can overcome the limitations of pure acoustic processing, directly acquire the user's auditory intent, and use this intent information to intelligently guide the cochlear implant system, enabling it to focus on any target of interest in a complex sound field, just like a person with normal hearing, thereby truly solving the core pain point of the "cocktail party effect."

[0006] A search revealed Chinese patent application number 202010822286.7, which discloses a cochlear implant system based on closed-loop neurofeedback control. This system uses the neurofeedback efferent portion to regulate the stimulus afferent portion, forming a complete closed-loop control to protect and enhance auditory perception in noisy environments. However, it still neglects the user's visual focus and cannot achieve listening to targets of interest based on visual attention. Summary of the Invention

[0007] In view of one of the shortcomings of the prior art, the purpose of this application is to provide an implantable brain-ear co-interface microsystem, implantable components, and hearing enhancement method.

[0008] A first aspect of this application provides an implantable brain-ear co-channel interface microsystem, comprising an implantable component and an external component, wherein: The implantable component collects neural signals in the brain that indicate the direction of visual attention and receives auditory stimulation commands from external components, converting them into pulsed electrical stimulation signals to stimulate the cochlea. The external component collects multiple sound source signals from the user's environment, receives neural signals sent by the implanted component and determines the target visual direction accordingly, amplifies the sound source in the target visual direction, encodes the amplified audio into an auditory stimulation command, and sends it back to the implanted component.

[0009] Optionally, the implantable component includes: Implantable brain electrode arrays are used to collect neural signals related to the direction of visual attention; Cochlear implant electrodes are used to be inserted into the cochlea and, driven by the pulsed electrical stimulation signal, to apply electrical stimulation to the spiral ganglion in the cochlea; An implant processor is connected to the implantable brain electrode array and the cochlear implant electrodes; An in vivo communication medium, connected to the implant processor, is used to send the neural signals to the external component and receive auditory stimulation commands from the external component.

[0010] Optionally, the implantable brain electrode array is used to acquire neural signals originating from brain functional areas, including the frontal oculomotor area and the lateral parietal sulcus area; The implantable brain electrode array is configured as a planar electrode or a skull nail electrode. The planar electrode is used to attach to the cortex or the lateral dura mater of the brain functional area, or to be implanted within the brain functional area; The cranial nail electrode is used to fix the skull directly above the brain functional area at the corresponding location. The implantable brain electrode array and cochlear implant electrodes are respectively connected to the same implant processor via biocompatible insulated leads, for joint implantation into the human body during a single surgery.

[0011] Optionally, the implant processor includes: The EEG acquisition module, which is connected to the implanted EEG electrode array, is used to perform processing on the acquired neural signals, including amplification, filtering, analog-to-digital conversion and data compression, to generate digital EEG signals; A neurostimulation module is configured to receive auditory stimulation commands, decode them into pulsed electrical stimulation signals, and thereby drive the cochlear implant electrodes to apply electrical stimulation to the spiral ganglion. An in-vivo wireless communication module is used to enable bidirectional data communication between the implanted component and the external component.

[0012] Optionally, the external component includes: A multi-microphone array is used to collect sound source signals from multiple spatial locations in the environment; An external processor, communicating with the implanted component, is configured to: receive the neural signals and decode the user's target visual direction; enhance the sound source signal according to the target visual direction and extract the target audio stream; and encode the target audio stream into an auditory stimulation command. An external communication medium is connected to the external processor to receive auditory stimulation commands; it also communicates with the implantable component to receive its output neural signals and send auditory stimulation commands.

[0013] Optionally, the external processor includes: The EEG decoding module is used to receive the neural signals emitted by the implanted component and apply an algorithm to analyze them, decoding the target visual direction that represents the user's visual attention focus; The sound scene analysis module receives and processes audio signals from the multi-microphone array, and performs enhancement processing on the received audio signals based on the target visual direction obtained by the EEG decoding module to generate a target audio stream; An auditory encoding module, which is connected to the sound scene analysis module, encodes the target audio stream into a target direction auditory stimulus instruction; An external wireless communication module is connected to the auditory encoding module and the EEG decoding module to achieve bidirectional data communication with the implanted component, including receiving the neural signals and sending the auditory stimulation commands.

[0014] Optionally, the implantable component further includes an energy receiving coil; the external component further includes an energy transmitting coil. The energy receiving coil and the energy transmitting coil are aligned and fixed by magnetic force; The energy receiving coil has an implanted magnet at its geometric center, and the energy transmitting coil has an external magnet at its geometric center. The external magnet and the implanted magnet are attracted by transdermal magnetic force so that the two coils are automatically guided to a coaxial position when the external component is worn.

[0015] A second aspect of this application provides an implantable component comprising the features of any of the implantable components described in the previous application.

[0016] A third aspect of this application provides a brain-ear co-processing hearing enhancement method based on any of the implantable brain-ear co-processing interface microsystems described in any one of the claims, comprising: The implanted component collects neural signals related to visual attention and sends them to an external component; An external component decodes the neural signals to obtain the target visual direction; Based on the target visual direction, the external component enhances the acoustic signal in the corresponding direction of the multi-sound source signal it collects to obtain the target audio stream; The external component encodes the target audio stream into an auditory stimulation command and sends it back to the implanted component; The implanted component decodes the auditory stimulation command into a pulse electrical stimulation signal and stimulates the cochlea to achieve directional auditory enhancement.

[0017] Optionally, the decoding of the neural signal is implemented using a machine learning model, which includes a support vector machine, a convolutional neural network, or a recurrent neural network. Methods for enhancing acoustic signals include at least one of spatial filtering algorithms, blind source separation methods, or deep learning speech separation models.

[0018] The implantable brain-ear co-processing interface microsystem provided in this application determines the target visual direction by analyzing neural signals related to the direction of visual attention, and enhances the corresponding sound source based on the direction to generate auditory stimulation instructions and send them back to the implantable component to drive it to apply electrical stimulation to the cochlea. This realizes the technical path of guiding auditory signal processing with visual attention, so that the output of auditory stimulation matches the spatial direction of the user's attention.

[0019] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0020] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a framework diagram of an implantable brain-ear co-channel interface microsystem according to an exemplary embodiment; Figure 2 This is a schematic diagram of the implantation location and a topographical diagram of an implantable brain-ear co-channel interface microsystem according to an exemplary embodiment; Figure 3 This is a flowchart illustrating an application method of an implantable brain-ear co-channel interface microsystem according to an exemplary embodiment; Figure 4 This is a schematic diagram illustrating the layout of the tester, speaker, and screen for a verification test according to an exemplary embodiment.

[0021] In the diagram: 1-Implantable component, 101-Implantable brain electrode array, 102-Implantable processor, 103-In-vivo data antenna, 104-Cochlear implant electrode, 105-Energy receiving coil. 1021-EEG acquisition module, 1022-Implant power management module, 1023-Neurostimulation module, 1024-In-vivo wireless communication module; 2-External components, 201-Multi-microphone array, 202-External processor, 203-External data antenna, 204-Energy emission coil, 205-Rechargeable battery; 2021 - EEG decoding module, 2022 - External power management module, 2023 - Sound scene analysis module, 2024 - Auditory coding module, 2025 - External wireless communication module, 2026 - System control module; 3. Visual cortex of the brain; 4-Cochlear. Detailed Implementation

[0022] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0023] Terminology Explanation: Target visual orientation refers to the spatial orientation indicated by the user's gaze behavior, reflecting the focus of their visual attention, and is used as a target guidance signal for auditory enhancement in the brain-ear coordination interface.

[0024] When using existing cochlear implant devices, if multiple speakers are present simultaneously, the algorithm often cannot distinguish which speaker is the user's target speaker and which is an interfering speaker that needs to be suppressed. Therefore, it often incorrectly suppresses the target speech or fails to effectively separate them, making it difficult to improve the listening experience. To address these issues, this application provides an implantable brain-ear co-channel interface microsystem that uses visual attention to determine the target direction, thereby improving speech discrimination in complex sound source environments.

[0025] Reference Figure 1 As shown, in one embodiment of this application, an implantable brain-ear co-processing interface microsystem is proposed, comprising an implantable component 1 and an external component 2. The implantable component 1 collects neural signals from the brain that indicate the direction of visual attention and receives auditory stimulation commands from the external component 2, converting them into pulsed electrical stimulation signals to stimulate the cochlea. External component 2 collects multiple sound source signals from the user's environment, receives neural signals sent by implanted component 1 and determines the target visual direction accordingly, enhances the sound source in the target visual direction, encodes the enhanced audio into an auditory stimulation command, and sends it back to implanted component 1.

[0026] The above embodiments of this application use the user's intrinsic visual attention intention as a marker signal to directly guide the external components to perform acoustic processing, overcoming the fundamental defects of traditional beamforming technology, such as rigid direction and inability to quickly switch according to the user's intention, and enabling "hearing wherever you want".

[0027] Users can seamlessly switch the object of their listening without any manual operation, simply by moving their gaze to focus on the target. This "what you see is what you hear" closed-loop collaborative working mode provides an intuitive interactive experience superior to existing technologies and is more in line with natural human communication habits.

[0028] To achieve visual orientation perception, some specific embodiments of this application provide a preferred structure for the implanted component 1. Specifically, such as... Figure 1 As shown, the implantable component 1 is configured to: acquire neural signals indicating the direction of visual attention via an implantable brain electrode array 101; process and convert the neural signals via an implantable processor 102; transmit the neural signals to an external component via an in-body data antenna 103; receive auditory stimulation commands transmitted back to the implantable component 1 from the external component 2 via the in-body data antenna 103; decode the received auditory stimulation commands into pulsed electrical stimulation signals via the implantable processor 102; apply electrical stimulation to spiral ganglion neurons via a cochlear implant electrode 104; and receive wirelessly transmitted energy via an energy receiving coil 105.

[0029] The embodiments described above in this application integrate EEG acquisition and cochlear stimulation functions to achieve closed-loop regulation of neural signals based on visual attention, thereby improving the accuracy and responsiveness of auditory focusing.

[0030] Of course, the in-vivo data antenna 103 in this embodiment is a specific implementation of a radio electromagnetic communication medium. In other embodiments, the communication function can also be accomplished through other suitable communication media, including but not limited to near-field optical communication, ultrasonic communication or capacitive / inductive coupling, as long as it realizes the bidirectional data transmission function between the implanted component and the external component.

[0031] To enable further processing of the acquired neural signals, some specific embodiments of this application provide preferred structures for the external component 2. Specifically, such as... Figure 1 As shown, the external component 2 is configured to: capture multiple spatially separated sound sources via the multi-microphone array 201; The following operations are performed via the external processor 202: a) Receiving neural signals: Receiving neural signals indicating the direction of visual attention transmitted from the data antenna 103 in the implanted component 1 via the external data antenna 203; b) Decoding visual attention intention: Decode neural signals to determine the user's visual attention direction and use it as the target auditory direction; c) Perform auditory focusing: Based on the target auditory direction, apply a spatial filtering algorithm to multiple sound source signals to generate a target audio stream, where sounds from the target auditory direction are enhanced, while sounds from other directions are suppressed. Of course, the enhancement method is not limited to a single approach; blind source separation methods or deep learning speech separation models can also be used.

[0032] d) Generate and send instructions: Encode the target audio stream into auditory stimulation instructions and send them to the implantable component 1 via the external data antenna 203.

[0033] The above embodiments determine the target auditory direction by decoding neural signals and combine spatial filtering to achieve selective enhancement of sound sources, so that auditory perception and user attention direction are coordinated, thereby improving speech intelligibility in complex environments.

[0034] In order to achieve high-precision acquisition of visual attention-related neural signals and establish a basis for collaborative work with cochlear implants, in some specific embodiments of this application, the layout and configuration of the implantable brain electrode array 101 and the cochlear implant electrode 104 can adopt a combination of synchronous implantation, individualized positioning and multimodal electrode design to improve signal quality and system integration.

[0035] Specifically, the implantable EEG array 101 is configured to acquire neural signals from at least the frontal eyefields (FEF) and the lateral intraparietal area (LIP), which are highly correlated with eye movement control and spatial attention allocation and are core brain functional areas for decoding the user's direction of attention.

[0036] It should be understood that this application is not limited to collecting signals from the above-mentioned brain regions. In other embodiments, brain regions involved in audiovisual spatial integration, such as the anterior parietal cortex, the auxiliary ocular area, or the superior temporal sulcus, can also be used as signal sources. As long as their neural activity can reflect the user's visual attention intention, they all fall within the technical scope of this application.

[0037] It should be noted that, in order to ensure that the electrodes accurately cover the core brain functional areas, individualized implantation planning is required by combining preoperative functional imaging and intraoperative navigation technology. For example, during the surgical preparation phase, the patient undergoes a functional magnetic resonance imaging (fMRI) scan, which activates the target brain region and maps its functional hotspots by performing tasks such as a "delayed saccade task" and a "spatial cue cue task". This map is then fused with high-resolution CT and MRI images and imported into the intraoperative neuronavigation system to generate a three-dimensional positioning guidance model, which is used to guide the precise placement and coverage of the implanted EEG array 101.

[0038] Simultaneously, the cochlear implant electrode 104 is implanted via a standard otological surgical approach: an incision is made behind the ear, and after mastoidectomy and cochlear fenestration, it is stably placed within the scala tympani of the cochlea, ensuring effective contact with the spiral ganglion. Subsequently, the leads of both are led out through the same minimally invasive through-hole on the skull and connected together to the corresponding interface of the implant processor 102 fixed to the surface of the skull. Finally, the component is encapsulated and the scalp is sutured, forming an integrated, co-located implantable component 1.

[0039] Specifically, based on different clinical needs regarding signal resolution, long-term stability, and surgical trauma, the implantable brain electrode array 101 can be configured in two main forms: planar electrodes (such as...) Figure 2 (as shown) or cranial nail electrodes.

[0040] For example, when planar electrodes are used, they are preferably flexible high-density microcortical electroencephalography (ECoG) arrays or flexible high-density epidural electrode arrays. The substrate material is polyimide or Parylene-C, etc., with a thickness of only 10–20 micrometers, which has excellent mechanical flexibility and can closely conform to the curved surface of the cerebral cortex or epidural space, improving the conformal signal acquisition capability. The array integrates 8 to 256 platinum-iridium alloy or gold electrode points with a diameter of 25–800 μm and a spacing of 50–2000 μm, which can be precisely fabricated by laser cutting or photolithography. During surgery, they are fixed on the cortex or slid into the epidural space through a micro-bone window to record local field potential (LFP) signals, balancing signal quality and tissue safety.

[0041] For example, the planar electrode can be further designed as a flexible puncture-type microelectrode array with a guide hole of about 5-50 μm in diameter at its tip, which can be directly implanted into the target brain cortex to record the action potentials (Spikes) of single or multiple neurons, thereby obtaining neural activity information with higher spatiotemporal resolution, which is suitable for high-precision decoding scenarios of subtle attentional changes.

[0042] For example, the implantable brain electrode array 101 can also be composed of multiple cranial screw electrodes. These electrodes are biocompatible titanium alloy screw structures with built-in conductive contacts. They are screwed in after drilling holes in the skull, ensuring stable contact between the conductive end and the dura mater corresponding to the target brain region. This fixation method is minimally invasive, easy to operate, and offers good long-term stability, making it suitable for applications requiring long-term wear and with low sampling density requirements.

[0043] The implantable brain electrode array 101 and the cochlear implant electrode 104 are connected to the same implant processor 102 via biocompatible insulated leads and are configured to be implanted together in a single surgical procedure. This integrated design reduces the number of surgeries and anesthesia risks, and achieves a compact layout and stable connection through a shared fixing structure, unified lead path, and shared percutaneous interface, which is beneficial for power supply, communication, and overall system coordination.

[0044] The embodiments described above in this application, through the simultaneous implantation of brain-ear dual-mode electrodes, combined with fMRI-guided individualized positioning and various electrode morphologies adapted to different clinical needs, achieve high-fidelity acquisition and system-level integration of visual attention-related neural signals, significantly improving the accuracy of neural decoding and the overall synergistic performance of the device while ensuring surgical safety.

[0045] To achieve efficient processing and low-power transmission of the acquired neural signals and ensure accurate execution of electrical stimulation commands, in some specific embodiments of this application, the signal processing and energy management functions in the implantable component 1 can be implemented using an integrated and modular implantable processor 102. This processor integrates four collaboratively working functional modules: an EEG acquisition module 1021, an implantable power management module 1022, a neural stimulation module 1023, and an in vivo wireless communication module 1024.

[0046] EEG acquisition module 1021, which is connected to implantable EEG electrode array 101.

[0047] Specifically, this module is an application-specific integrated circuit (ASIC). It is configured to receive microvolt-level analog neural signals from the electrodes and perform preprocessing.

[0048] Preprocessing steps include: First, the signal is amplified by a low-noise amplifier (LNA) array; Secondly, noise and interference are filtered out using a programmable bandpass filter and a notch filter. The processed analog signal is converted into a multi-channel EEG digital signal using a high-precision analog-to-digital converter (ADC, e.g., 16-bit, 1 k-30 kHz sampling rate).

[0049] Finally, optionally, algorithms such as neural impulse detection can be used to compress sparse events, reducing the amount of data transmitted and meeting the bandwidth requirements of wireless transmission.

[0050] The implant power management module 1022 is connected to the energy receiving coil 105 and is configured to modulate and rectify the received wireless energy to provide a stable operating power to all electronic components of the implant component 1.

[0051] Specifically, the power management module is configured to perform the following functions: First, it includes a rectifier circuit for converting the high-frequency alternating current induced by the energy receiving coil 105 into direct current.

[0052] Secondly, a voltage regulation and charging management unit precisely regulates the DC power, generating multiple stable DC voltages (e.g., 1.2V, 1.8V, 3.3V, and 5.0V) through one or more low-dropout linear regulators (LDOs) or high-efficiency switching regulators to adapt to and drive different modules of the implant processor 102 respectively.

[0053] The neurostimulation module 1023 is configured to receive auditory stimulation commands, decode them into pulsed electrical stimulation signals, and thereby drive the cochlear implant electrode 104 to apply electrical stimulation to the spiral ganglion.

[0054] Specifically, this module is the final execution unit for auditory sensory reconstruction. It is configured to receive auditory stimulus commands generated by external components. These commands typically contain key parameter information such as channel selection, pulse amplitude, pulse width, and pulse rate, encoded using a sound processing strategy.

[0055] Upon receiving the instruction, the module's internal instruction decoder first parses it. Subsequently, a high-precision programmable current source array generates a series of precisely timed and amplitude-controllable electrical pulses based on the decoded parameters. These pulse signals are ultimately routed precisely to designated electrode contacts on the cochlear implant electrode 104 via a multiplexer matrix. In this way, the module drives the cochlear implant electrode 104 to apply a customized electrical stimulation pattern to the spiral ganglion cells at the target location within the cochlea 4, thereby simulating the neural discharge activity triggered by natural sounds in the auditory nerve, ultimately forming a perceptible hearing sensation in the brain.

[0056] The in-vivo wireless communication module 1024 is a critical data link connecting the implanted component 1 and the external component. This module establishes and manages a highly reliable bidirectional wireless communication channel via a dedicated in-vivo data antenna 103.

[0057] Specifically, its core functions include: (i) In vivo-external transmission: This module continuously collects processed binary event pulse signals from the EEG acquisition module, packages them into data frames and adds check codes according to a preset communication protocol (e.g., Bluetooth Low Energy BLE or proprietary protocol), then modulates them onto a radio frequency carrier and transmits them efficiently to external components through an antenna for further decoding and analysis.

[0058] (ii) In vitro-in vivo transmission: Simultaneously, this module is responsible for receiving control signals from external components, particularly auditory stimulation commands containing channel, amplitude, and timing information. Upon receipt, the module demodulates and verifies the commands to ensure their integrity, and then precisely routes them to the neurostimulation module.

[0059] The embodiments described above in this application integrate four core modules—signal acquisition, power management, neurostimulation, and wireless communication—into the implant processor, achieving miniaturized, low-power, and high-precision closed-loop brain-ear coordinated control. This improves the efficiency of neural signal processing and the accuracy of electrical stimulation response, while reducing the overall system energy consumption and wireless transmission burden, thus enhancing the clinical applicability and long-term stability of the device.

[0060] In order to achieve efficient data interaction and continuous energy supply between implanted components and external components, in some specific embodiments of this application, a combination of highly reliable wireless links and contactless energy transmission can be adopted for the communication and power supply functions in the external components to ensure long-term stable operation of the system.

[0061] Specifically, the external component 2 also includes an external data antenna 203, an energy transmission coil 204, and a rechargeable battery 205, which work together to achieve bidirectional communication and power supply.

[0062] It should be noted that the above components are integrated into a portable external device, supporting daily wear and use, and meeting the clinical needs for low latency, anti-interference, and long-term operation.

[0063] For example, the external data antenna 203 is paired with the data antenna 103 in the implantable component 1, operating in a dedicated frequency band (such as 400–900 MHz or the 2.4 GHz ISM band) to establish a high-fidelity, low-latency bidirectional wireless data link. This antenna is responsible for receiving binary event pulse signals from the implantable component 1—signals compressed and generated by the EEG acquisition module, containing neural event information related to the user's visual attention; simultaneously, it also precisely transmits auditory stimulation commands (including channel selection, pulse parameters, etc.) generated by the external processor to the implant, ensuring the real-time performance and accuracy of closed-loop control.

[0064] Furthermore, the energy transmitting coil 204, based on the principle of electromagnetic induction, forms a transdermal coupling with the energy receiving coil 105 implanted in the body, realizing non-contact wireless energy transmission. Its operating frequency is typically set in the kHz to MHz range, using an alternating magnetic field to transmit electrical energy through the skin into the body, avoiding the risk of infection caused by protruding wires.

[0065] The entire external system is powered by a replaceable, high-capacity rechargeable battery 205, preferably a lithium polymer battery, which offers advantages such as high specific energy, lightweight design, and long cycle life. Typical capacities range from 100 to 1500 mAh, supporting continuous operation for 8 to 12 hours, meeting all-day usage requirements. Battery level can be displayed in real-time via external indicator lights or an app interface.

[0066] The embodiments described above in this application achieve highly reliable two-way communication by configuring a dedicated data antenna. Combined with wireless power transmission and a replaceable power supply design, the system's battery life, wearing comfort, and clinical applicability are significantly improved while ensuring accurate interaction between nerve signals and stimulation commands.

[0067] In order to realize real-time auditory direction decoding and multi-module collaborative processing based on neural signals, and to build a low-latency, high-precision brain-ear closed-loop control system, in some specific embodiments of this application, an external processor 202 integrating multiple functional modules can be used for the information processing architecture in the external component to complete the entire process control from neural signal parsing to auditory command generation.

[0068] Specifically, the external processor is constructed to include six core functional modules that work together: EEG decoding module 2021, external power management module 2022, sound scene analysis module 2023, auditory coding module 2024, external wireless communication module 2025, and system control module 2026.

[0069] It should be noted that the above modules can be distributed in digital signal processors (DSPs), microcontrollers (MCUs), or system-on-a-chip (SoCs), and can achieve data sharing and task synchronization through high-speed interconnection.

[0070] For example, the EEG decoding module 2021 is implemented by a high-performance DSP or MCU to receive multi-channel EEG digital signal streams or binary event pulse signals transmitted via the external data antenna 203. This module first extracts key features such as power variations in the high gamma band (70–150 Hz) from the neural signals in the frontal oculomotor area and the lateral parietal sulcus; then, it calls a pre-trained machine learning classifier model (such as a support vector machine (SVM) or a lightweight deep neural network) to perform pattern recognition on the multi-dimensional feature vectors, and outputs in real time a target auditory direction signal representing the user's current visual attention focus (e.g., "left side," "directly in front," or a specific angle value), with a latency typically less than 100 milliseconds.

[0071] The external power management module 2022 is electrically connected to the energy transmitting coil 204 and is responsible for converting the DC power from the rechargeable battery 205 into a high-frequency AC signal (such as kHz to MHz), driving the transmitting coil to generate an alternating magnetic field, and transmitting power to the energy receiving coil 105 implanted in the body non-invasively through electromagnetic induction to achieve continuous and stable power supply.

[0072] The Sound Scene Analysis Module 2023 is driven by a DSP and is operatively coupled to a multi-microphone array. Its functions include: (i) real-time acquisition of ambient sound, combined with sound source localization algorithms (such as SRP-PHAT) to construct a dynamic acoustic scene map and identify the location of each sound source; (ii) after receiving the target auditory direction output by the EEG decoding module, it initiates an adaptive spatial filtering algorithm (such as Minimum Variance Distortionless Response MVDR or Beamforming) to dynamically adjust the filtering weights, enhance the speech signal in the target direction and suppress background noise, and output a clear and focused target audio stream.

[0073] The auditory coding module 2024 receives the target audio stream, decomposes it into frequency sub-bands according to advanced cochlear implant coding strategies (such as ACE, CIS or FS4 strategies), extracts the envelope information, and maps it into digital auditory stimulation instructions containing channel number, pulse amplitude and timing parameters for subsequent wireless transmission.

[0074] The external wireless communication module 2025 establishes a high-bandwidth bidirectional communication link with the implantable component 1 through the external data antenna 203: on the one hand, it modulates and sends the auditory stimulation command to the body; on the other hand, it receives and demodulates the decoded EEG event stream to ensure the data integrity and real-time performance of the closed-loop pathway.

[0075] The system control module 2026 is typically implemented by the main MCU or SoC, responsible for coordinating the parallel operation of the six modules. Through unified clock management, task scheduling, and data stream orchestration, it ensures that the entire process from neural signal input to stimulation command output is executed with precise timing at the microsecond to millisecond level, maintaining the low latency and high stability of the entire system.

[0076] The embodiments described above in this application integrate six functional modules, including EEG decoding, sound field analysis, instruction encoding, and system control, to achieve real-time auditory focusing processing based on visual attention intent. This significantly improves the system's response speed, anti-interference capability, and clinical usability while ensuring efficient fusion of multi-source signals.

[0077] In order to achieve automatic alignment between the energy receiving coil and the energy transmitting coil during the wearing process and improve energy transmission efficiency, in some specific embodiments of this application, the positioning structure of the energy receiving coil 105 and the energy transmitting coil 204 can be fixed and guided by magnetic adsorption.

[0078] Specifically, the energy receiving coil 105 and the energy transmitting coil 204 are aligned and fixed by magnetic force; wherein, an implantable magnet is integrated at the geometric center of the energy receiving coil 105, and an external magnet is integrated at the geometric center of the energy transmitting coil. The external magnet and the implantable magnet are attracted by percutaneous magnetic force so that the two coils are automatically guided to a coaxial position with optimal energy transmission efficiency when the external component is worn.

[0079] It should be noted that the magnet is integrated into the center of the coil, which ensures that the axes of the two coils coincide and the spacing is constant when aligned, which is conducive to forming efficient electromagnetic coupling.

[0080] For example, during wear, when the external component approaches the implantation area, a transdermal attraction is generated between the external magnet and the implanted magnet, automatically adjusting their relative positions so that the energy transmitting coil and the receiving coil are coaxially aligned without the need for manual calibration.

[0081] In the embodiments described above, an implanted magnet and an external magnet are integrated at the geometric center of the energy receiving coil and the energy transmitting coil, respectively, and automatic alignment is achieved by transdermal magnetic adsorption, thus ensuring the high efficiency and stability of wireless energy transmission.

[0082] In order to achieve the coexistence of high-speed data communication and efficient energy transmission and avoid mutual interference, in some specific embodiments of this application, different frequency bands can be used for the operating frequency band configuration of the internal data antenna 103 and the external data antenna 203, as well as the energy receiving coil 105 and the energy transmitting coil 204 to achieve channel separation.

[0083] Specifically, the internal data antenna 103 and the external data antenna 203 operate in the gigahertz (GHz) band to support high-speed data communication, while the energy receiving coil 105 and the energy transmitting coil 204 operate in the kilohertz (kHz) or megahertz (MHz) band to achieve efficient inductive energy transmission.

[0084] It should be noted that data communication and energy transmission operate in different frequency ranges, which can avoid signal crosstalk both physically and in the spectrum.

[0085] The embodiments described above in this application achieve complete physical and spectral separation between the data channel and the energy channel by setting data communication in the GHz band and energy transmission in the kHz / MHz band, thus ensuring the reliability of system communication and the stability of power supply.

[0086] Based on the same technical concept, other embodiments of this application also provide an implantable component, including any of the features of the implantable component mentioned in the above embodiments. This implantable component acquires neural signals related to the user's attentional intentions through an implantable brain electrode array and integrates neural signal acquisition with cochlear electrical stimulation, achieving high integration and perception-feedback interaction capabilities, providing a key in vivo hardware foundation for constructing a "brain-ear synergy" intelligent hearing system.

[0087] Based on the same technical concept, some specific embodiments of this application also provide a brain-ear co-hearing hearing enhancement method based on an implantable brain-ear co-interface microsystem, including the following steps: S100, an implantable component, collects neural signals related to visual attention and sends them to an external component; S200: External components decode neural signals to obtain the target visual direction; S300, the external component enhances the acoustic signal in the corresponding direction of the multi-sound source signal it collects based on the target visual direction, to obtain the target audio stream.

[0088] S400, the external component encodes the target audio stream into an auditory stimulation command and sends it back to the implanted component; The S500 implantable component decodes auditory stimulation commands into pulsed electrical stimulation signals and stimulates the cochlea to achieve directional auditory enhancement.

[0089] The above embodiments, through a closed-loop collaborative mechanism, convert visual attention neural signals into directional auditory enhancement output, realize user intent-driven dynamic auditory focusing, and improve speech intelligibility and perceptual naturalness in complex environments.

[0090] In some specific implementations, the decoding of neural signals in S200 is achieved using a machine learning model, including a support vector machine (SVM), a convolutional neural network (CNN), or a recurrent neural network (RNN).

[0091] Specifically, the S200 can adopt the following steps: S201, the external component receives the neural signals sent by the implanted component; S202, the external component extracts temporal and / or spectral features from neural signals, wherein the spectral features include high gamma-band power and the temporal features include event-related potentials; S203, input the features into the trained machine learning model, and the model outputs a spatial angle or spatial vector representing the auditory direction of the target.

[0092] The above embodiments, based on real-time classification of neural signals using machine learning models, achieve low-latency, high-precision decoding of the gaze direction, ensuring that the auditory focus direction aligns with the user's attention. Figure 1 To.

[0093] In some specific implementations, spatial filtering algorithms can be used to enhance the acoustic signal in S300.

[0094] Specifically, spatial filtering algorithms include beamforming or minimum variance distortionless response algorithms. These algorithms dynamically form an acoustic main lobe to enhance the target sound source based on the target's auditory direction, and construct one or more acoustic nulls in the direction of interfering sound sources to actively suppress background noise and interference signals.

[0095] The above embodiments employ beamforming or minimum variance distortionless response algorithms to enhance the target sound source through the main lobe and suppress interference through nulls, thereby significantly improving the signal-to-noise ratio and achieving highly selective auditory scene separation.

[0096] Of course, this application does not limit the specific methods or number of sound source enhancements; any technical solution or combination thereof capable of enhancing the target sound source is applicable. For example, in some other embodiments, independent component analysis (ICA) can be used for blind source separation, or deep neural network models such as Conv-TasNet and SepFormer can be used to achieve directional extraction of the speech stream. The above enhancement process can utilize the target visual direction as prior information to guide the model to focus on the sound signal in the user's gaze direction, thereby achieving personalized auditory enhancement.

[0097] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0098] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0099] Application Example 1 like Figure 1 As shown, an implantable brain-ear co-channel interface microsystem includes the following components and their connections: An implantable component 1 is placed inside the body; wherein, an implantable brain electrode array 101 is used to collect neural signals from the visual area 3 of the brain and is electrically connected to an implant processor 102; the implant processor 102 integrates an EEG acquisition module 1021, an implant power management module 1022, a neurostimulation module 1023, and an in vivo wireless communication module 1024; the implant processor 102 is connected to an in vivo data antenna 103, an energy receiving coil 105, and a cochlear implant electrode 104; the cochlear implant electrode 104 is implanted in the cochlea 4 and is used to apply electrical stimulation to the spiral ganglion; the energy receiving coil 105 is connected to the implant power management module 1022 and is used to receive wirelessly transmitted energy.

[0100] External component 2 is located outside the body; a multi-microphone array 201 is used to collect ambient sound sources and is connected to an external processor 202; the external processor 202 integrates an EEG decoding module 2021, an external power management module 2022, a sound scene analysis module 2023, an auditory coding module 2024, an external wireless communication module 2025, and a system control module 2026; the external processor 202 is connected to an external data antenna 203, an energy transmitting coil 204, and a rechargeable battery 205; the rechargeable battery 205 powers each module in external component 2; the external data antenna 203 establishes a bidirectional wireless communication link with the internal data antenna 103; the energy transmitting coil 204 and the energy receiving coil 105 achieve transdermal energy transmission through electromagnetic induction.

[0101] The components work together through electrical signal connection, wireless communication or magnetic coupling to form a complete brain-ear coordinated closed-loop system.

[0102] refer to Figure 3 The brain-ear co-processing hearing enhancement method using the above-mentioned implantable brain-ear co-processing interface microsystem includes the following steps: S301: In complex auditory scenarios, users can actively indicate the direction of target hearing by consciously directing their gaze toward a sound source of interest (e.g., a specific speaker); S302: Implantable EEG array: Real-time acquisition of neural activity in brain regions such as the frontal lobe oculomotor area and the lateral parietal sulcus. These signals, especially the local field potentials in the high gamma band, are directly related to the user's eye movement intentions and spatial attention allocation. S303: Implantable processor: Performs preprocessing such as amplification, filtering and analog-to-digital conversion on the acquired weak analog neural signals, converts them into digital signal streams, and then transmits them to the external processor through its wireless communication module after packaging. S304: External processor: Through its external data antenna, it stably receives the EEG digital signal stream or binary event pulse signal transmitted from the implant, demodulates it and performs preliminary data integrity verification to ensure the accuracy of subsequent decoding; S305: External processor: Utilizing its EEG decoding module, it runs a pre-trained machine learning model to analyze the features of received neural signals in real time, thereby accurately decoding the auditory direction of the target representing the user's visual focus. S306: External processor: Based on the determined target direction, immediately start the sound scene analysis module to perform adaptive spatial filtering algorithms, such as beamforming, on the mixed sound collected by the multi-microphone array to enhance the target sound source and suppress interference noise; S307: External processor: Converts the enhanced target audio stream into digital auditory stimulation commands containing information such as channel, amplitude and pulse width through a speech coding strategy, and then sends them to the implant through a wireless communication module; S308: Implantable processor: It receives and parses auditory stimulation instructions from outside the body in its neurostimulation module, and precisely controls its internal current source array accordingly to generate pulsed electrical stimulation signals with highly controllable timing and amplitude. S309: Cochlear implant electrode: It precisely applies pulsed electrical stimulation signals to the corresponding spiral ganglion neurons through its multi-channel electrode array located in the cochlea to simulate the frequency-position coding pattern of sound. S310: The user ultimately perceives clear, focused hearing in the brain, with low latency between the hearing and the user's visual attention focus, achieving effective enhancement of target sound and the "hear what you see" experience in noisy environments.

[0103] This application example demonstrates the closed-loop control of the entire process of this application, from neural signal acquisition, intent decoding, sound source focusing to electrical stimulation output. It verifies the feasibility and superiority of the system in achieving "what you see is what you hear" in a real and complex acoustic environment, and provides a new technical path for the social rehabilitation of patients with severe hearing impairment.

[0104] The above system and methods were used for verification testing.

[0105] Specifically: such as Figure 4 As shown, in an anechoic chamber, five speaker-screen units were deployed at 45° intervals (0°, 45°, 90°, 135°, and 180°) along the horizontal plane, centered on the subject. During the test, a prompt image (such as a checkmark appearing at the 45° angle) was randomly selected on one screen to guide the subject's visual attention. Simultaneously, the speaker corresponding to the screen displaying the prompt image played a sentence from a target speech phrase library (such as an internationally or domestically recognized standardized phrase library). The remaining seven speakers simultaneously played different competing conversation noises.

[0106] The test was conducted in two phases, with an experimental group and a control group. The experimental group received an implantable brain-ear interface microsystem based on this application, while the control group received a traditional cochlear implant system with a fixed forward beam for sound resolution. Subjects were instructed to keep their head in a fixed position, only moving their eyes to observe the screen displaying the prompts, and then repeat the sentences they heard. Their speech recognition rate (SRR) was recorded.

[0107] Under a signal-to-noise ratio of 0dB, the control group achieved a speech recognition rate (SRR) of 85% only when the target speech came from directly in front (90°). However, when the target speech came from any other lateral direction (0°, 45°, 135°, 180°), the SRR dropped below 30%, exhibiting significant "directional hearing loss." In contrast, the experimental group, regardless of whether the visual cues appeared on the screen at any angle (0°, 45°, 90°, 135°, 180°), was able to quickly (with an average response delay of <50 ms) switch their auditory focus to that direction. Across all test directions, the average SRR remained consistently above 85%.

[0108] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.

[0109] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. An implantable brain-ear co-channel interface microsystem, characterized in that, Includes implantable components and external components, among which: The implantable component collects neural signals in the brain that indicate the direction of visual attention and receives auditory stimulation commands from external components, converting them into pulsed electrical stimulation signals to stimulate the cochlea. The external component collects multiple sound source signals from the user's environment, receives neural signals sent by the implanted component and determines the target visual direction accordingly, amplifies the sound source in the target visual direction, encodes the amplified audio into an auditory stimulation command, and sends it back to the implanted component.

2. The implantable brain-ear co-channel interface microsystem according to claim 1, characterized in that, The implantable component includes: Implantable brain electrode arrays are used to collect neural signals related to the direction of visual attention; Cochlear implant electrodes are used to be inserted into the cochlea and, driven by the pulsed electrical stimulation signal, to apply electrical stimulation to the spiral ganglion in the cochlea; An implant processor is connected to the implantable brain electrode array and the cochlear implant electrodes; An in vivo communication medium, connected to the implant processor, is used to send the neural signals to the external component and receive auditory stimulation commands from the external component.

3. The implantable brain-ear co-channel interface microsystem according to claim 2, characterized in that, The implantable EEG array is used to collect neural signals from functional areas of the brain, including the frontal ophthalmos area and the lateral parietal sulcus area. The implantable brain electrode array is configured as a planar electrode or a skull nail electrode. The planar electrode is used to attach to the cortex or the lateral dura mater of the brain functional area, or to be implanted within the brain functional area; The cranial nail electrode is used to fix the skull directly above the brain functional area at the corresponding location. The implantable brain electrode array and cochlear implant electrodes are respectively connected to the same implant processor via biocompatible insulated leads, for joint implantation into the human body during a single surgery.

4. The implantable brain-ear co-channel interface microsystem according to claim 2, characterized in that, The implant processor includes: The EEG acquisition module, which is connected to the implanted EEG electrode array, is used to perform processing on the acquired neural signals, including amplification, filtering, analog-to-digital conversion and data compression, to generate digital EEG signals; A neurostimulation module is configured to receive auditory stimulation commands, decode them into pulsed electrical stimulation signals, and thereby drive the cochlear implant electrodes to apply electrical stimulation to the spiral ganglion. An in-vivo wireless communication module is used to enable bidirectional data communication between the implanted component and the external component.

5. The implantable brain-ear co-channel interface microsystem according to claim 1, characterized in that, The external component includes: A multi-microphone array is used to collect sound source signals from multiple spatial locations in the environment; An external processor, communicating with the implanted component, is configured to: receive the neural signals and decode the user's target visual direction; enhance the sound source signal according to the target visual direction and extract the target audio stream; and encode the target audio stream into an auditory stimulation command. An external communication medium is connected to the external processor to receive auditory stimulation commands; it also communicates with the implantable component to receive its output neural signals and send auditory stimulation commands.

6. The implantable brain-ear co-channel interface microsystem according to claim 5, characterized in that, The external processor includes: The EEG decoding module is used to receive the neural signals emitted by the implanted component and apply an algorithm to analyze them, decoding the target visual direction that represents the user's visual attention focus; The sound scene analysis module receives and processes audio signals from the multi-microphone array, and performs enhancement processing on the received audio signals based on the target visual direction obtained by the EEG decoding module to generate a target audio stream; An auditory encoding module, which is connected to the sound scene analysis module, encodes the target audio stream into a target direction auditory stimulus instruction; An external wireless communication module is connected to the auditory encoding module and the EEG decoding module to achieve bidirectional data communication with the implanted component, including receiving the neural signals and sending the auditory stimulation commands.

7. The implantable brain-ear co-channel interface microsystem according to claim 1, characterized in that, The implantable component further includes an energy receiving coil; the external component further includes an energy transmitting coil. The energy receiving coil and the energy transmitting coil are aligned and fixed by magnetic force; The energy receiving coil has an implanted magnet at its geometric center, and the energy transmitting coil has an external magnet at its geometric center. The external magnet and the implanted magnet are attracted by transdermal magnetic force so that the two coils are automatically guided to a coaxial position when the external component is worn.

8. An implantable component, characterized in that, Includes the features of the implantable component as described in any one of claims 2-4.

9. A brain-ear co-processing hearing enhancement method based on the implantable brain-ear co-processing interface microsystem according to any one of claims 1-7, characterized in that, include: The implanted component collects neural signals related to visual attention and sends them to an external component; An external component decodes the neural signals to obtain the target visual direction; Based on the target visual direction, the external component enhances the acoustic signal in the corresponding direction of the multi-sound source signal it collects to obtain the target audio stream; The external component encodes the target audio stream into an auditory stimulation command and sends it back to the implanted component; The implanted component decodes the auditory stimulation command into a pulse electrical stimulation signal and stimulates the cochlea to achieve directional auditory enhancement.

10. The brain-ear synergistic hearing enhancement method according to claim 9, characterized in that, The decoding of the neural signals is achieved using a machine learning model, which includes a support vector machine, a convolutional neural network, or a recurrent neural network. Methods for enhancing acoustic signals include at least one of spatial filtering algorithms, blind source separation methods, or deep learning speech separation models.

Images