Cochlear implant system with optimized frame coding

Abstract

A cochlear implant system with optimized frame coding is disclosed, comprising: a receiving unit; a processor unit configured to receive an acoustic signal and divide the audio signal into a plurality of band limited audio signals and generate a plurality of stimulation pulses based on samples of the plurality of band limited audio signals, wherein each of the plurality of band limited audio signals is sampled with a time onset, determine one or more audio components for each of the plurality of stimulation pulses sampled; an electrode array; the processor unit comprising a switching module configured to switch between using a fixed stimulation frame onset and using a variable stimulation frame onset for frame coding the plurality of stimulation pulses into one or more stimulation frames, wherein the switching module is configured to switch between the fixed stimulation frame onset and the variable stimulation frame onset based on the one or more audio components, wherein the electrode array is configured to stimulate an auditory nerve of a user of the cochlear implant system based on the frame coded plurality of stimulation pulses.

Description

Technical Field

[0001] This application relates to cochlear implantation systems. More specifically, this application relates to a processor unit including a switching module configured to switch between using a fixed stimulation frame start and using a variable stimulation frame start for encoding a plurality of stimulation pulse frames into one or more stimulation frames. Background Technology

[0002] Known multichannel cochlear implants (CIs) directly encode ambient sound into multiple stimulation pulses, which are then frame-encoded into one or more stimulation frames using a specific waveform strategy such as Continuous Interleaved Sampling (CIS). The frame-encoded stimulation pulses are then transmitted to the inner ear via electrical stimulation. This electrical stimulation is provided through an electrode array comprising multiple electrodes. The goal of the Continuous Interleaved Sampling (CIS) strategy is to sequentially stimulate the cochlea by decomposing ambient sound into spectral bands according to the placement of each electrode in the electrode array. Each electrode generates a charge-modulated current at a fixed rate based on its corresponding bandpass-filtered envelope. This temporal modulation is proposed to reproduce the neural stimulation pattern (temporal encoding).

[0003] In CIS-type implementations, temporal encoding may not be ideally transmitted. A perceptual example illustrating this is pitch, even though substantial evidence suggests that temporal encoding plays a significant role in pitch perception, pitch is considered related to positional encoding. For example, in temporal encoding, it is known that using variable stimulus frame initiations instead of fixed stimulus frame initiations during frame encoding results in stronger pitch discrimination. The goal of frame encoding is to generate one or more stimulus frames that transmit the most probable and intrinsic pre-selected features or audio components characterizing the ambient sound as electrical stimulation pulses with either fixed or variable stimulus frame initiations.

[0004] The CIS waveform strategy is based on frame-encoded stimulation pulses delivered to the corresponding electrodes in a time-interleaved manner. These pulses are delivered sequentially from the bottom electrode to the top electrode (frame). Therefore, this strategy requires a fixed time period for restimulating the same electrode. The CIS waveform strategy relies on a fixed stimulation frame start.

[0005] In known cochlear implant systems, information and energy are transdermal transmitted between the external unit and the implantable unit on the same modulated radio frequency (RF) carrier. The energy supplied to the implantable unit is proportional to the power level and active time of the RF transmission. To provide stable control, the external unit provides a constant active time for each RF transmission frame. An RF frame is established by a series of pulses transmitting information or control stimuli, or pulses transmitting energy.

[0006] With only a constant RF length, the optimal frame start strategy does not provide stable energy control (due to the variable nature of frame length).

[0007] Therefore, a solution is needed that addresses at least some of the problems mentioned above. Summary of the Invention

[0008] One aspect of the present invention is to provide an improved frame coding for obtaining improved timing information provided by electrical stimulation of auditory nerve fibers.

[0009] Another aspect of the present invention is to provide stable energy control for the implantable unit.

[0010] One aspect of the invention is implemented by a cochlear implant system, which may include a receiving unit configured to receive acoustic signals and transmit audio signals based on the acoustic signals. The receiving unit may be disposed in an external unit and / or an implantable unit. The external unit may be disposed on the user's skin, such as on the user's head or ear, and the implantable unit may be disposed beneath the user's skin, such as between the skin and skull of the head. The receiving unit may include one or more microphones and / or a radio frequency interface configured to wirelessly communicate with an assistive device. The wireless communication may be based on Bluetooth, Bluetooth Low Energy, or any short / long-range communication protocol.

[0011] The cochlear implant system may also include a processor unit configured to receive an acoustic signal, divide the audio signal into multiple frequency band-limited audio signals, and generate multiple stimulation pulses based on sampling of the multiple frequency band-limited audio signals, wherein each of the multiple frequency band-limited audio signals is sampled with a time start. For each of the sampled multiple stimulation pulses, one or more audio components are determined.

[0012] The processor unit can be configured to generate an event sequence for each of a plurality of frequency-limited audio signals, wherein each event in the event sequence is detected when the phase of the frequency-limited audio signal exceeds a phase threshold, wherein the phase threshold can be any value between 0 and 2π. The timing start can be equal to the event rate of the event sequence. At each event, a stimulus pulse is generated based on one or more audio components extracted from the audio signal and / or from each of the sampled plurality of frequency-limited audio signals, wherein the sampling is based on a waveform strategy, a feature extraction strategy, or a mixture of waveform and feature extraction strategies.

[0013] The processor unit may include a filter bank comprising multiple bandpass filters, each configured to generate a band-limited audio signal comprising a frequency band of the audio signal, wherein the frequency band includes a frequency range of the audio signal. The filter bank may include more than 20 bandpass filters covering frequencies from 80 Hz to 10000 Hz. Each filter may be assigned to an electrode of an electrode array, and each electrode may be assigned to a specific frequency range.

[0014] Sampling of multiple frequency band-limited audio signals can be based on waveform strategies such as Continuous Interleaved Sampling (CIS) or Spectral Peak (SPEAK), or on deterministic feature extraction strategies that include the temporal fine structure (TFS) of the audio signal, or on a hybrid of CIS and feature extraction strategies.

[0015] The feature extraction strategy can be configured to extract features across the entire bandwidth of the audio signal, enabling feature identification without limiting analysis to individual channel bands. In this example, the processor unit can be configured to receive the audio signal and generate one or more band-limited audio signals based on sampling of one or more band-limited audio signals, wherein one band encompasses the full bandwidth of the audio signal.

[0016] Therefore, compared to first dividing the audio signal into multiple audio signals with limited frequency bands through a filter group and then extracting features from each audio signal with limited frequency bands of interest, the extracted features are determined more accurately.

[0017] One or more audio components may include extracted features that ideally characterize the acoustic signal received by the receiving unit. The extracted features may be the fundamental frequency, harmonics relative to the fundamental frequency of the audio signal, the fundamental frequency and harmonic timing of the audio signal and their energy, and / or the fundamental frequency and harmonic timing of the audio signal and their phase.

[0018] A waveform strategy can extract the amplitude, phase, fundamental frequency, or harmonic frequency of one bandwidth-limited audio signal from multiple bandwidth-limited audio signals. In this example, the processor unit can be configured to receive audio signals and generate one or more bandwidth-limited audio signals or multiple bandwidth-limited audio signals. Each of the one or more bandwidth-limited audio signals or multiple bandwidth-limited audio signals may include an oscillation envelope. The amplitude, phase, fundamental frequency, or harmonic frequency can be derived from the oscillation envelope through rectification and low-pass filtering. The extracted amplitude, phase, fundamental frequency, and / or harmonic frequency can be a portion of one or more audio components that ideally characterize the acoustic signal received by the microphone unit.

[0019] One or more audio components may include multiple amplitudes, phases, fundamental frequencies, and / or harmonic frequencies that ideally characterize the oscillating envelope of the acoustic signal, and / or one or more audio components may include extracted features characterizing the acoustic signal.

[0020] Audio components may include the fundamental frequency, harmonics relative to the fundamental frequency of the audio signal, the fundamental frequency and harmonic timing of the audio signal and the energy to the left and right, and / or the fundamental frequency and harmonic timing of the audio signal and the phase to the left and right.

[0021] The processor unit can be configured to generate an event sequence for each of a plurality of frequency-limited audio signals, wherein each event in the event sequence is determined when the phase of the bandpass audio signal exceeds a phase threshold. The phase threshold can be any value between 0 and 2π. The timing start can be equal to the event rate of the event sequence.

[0022] The cochlear implant system may also include an electrode array containing multiple electrodes. The electrode array may be wired to the implantable unit and implanted into the cochlea of ​​the user of the cochlear implant system. The electrode array is configured to transmit stimulation pulses as electrical stimulation to the auditory nerve fibers of the cochlea.

[0023] Acoustic signals can be sound waves that include acoustic information about the user's surroundings.

[0024] To obtain optimal timing information based on the acoustic signal provided by stimulation pulses, the processor unit may include a switching module configured to encode multiple stimulation pulse frames into one or more stimulation frames and switch between using a fixed stimulation frame start and using a variable stimulation frame start. The switching module is configured to switch between fixed and variable stimulation frame start based on one or more audio components, wherein the electrode array is configured to stimulate the auditory nerve of the user of the cochlear implant system based on the frame-encoded multiple stimulation pulses.

[0025] The purpose of frame coding is to time the delivery of multiple frame-coded stimulation pulses to the implantable unit, so that the user's auditory nerve fibers receive as much temporal information as possible from the stimulation pulses. Frame coding generates an electrical stimulation spectrogram, which includes information on which electrodes of the electrode array will be activated and when they will be activated to stimulate the auditory nerve fibers with stimulation pulses.

[0026] The timing information may include one or more audio components, such as features extracted from the acoustic signal or from the extracted envelope information characterizing the acoustic signal.

[0027] Within one or more stimulation frames, one electrode of the electrode array may be activated only once, meaning that the stimulation frame may include only one stimulation pulse for each electrode.

[0028] To deliver multiple stimulation pulses to the implantable unit of the cochlear implant system and then to the electrode array, the stimulation pulses need to be organized into one or more stimulation frames provided by frame coding. The stimulation pulses can then be delivered to the implantable unit sequentially according to a predetermined order within the stimulation frame. For example, the stimulation pulses can be organized in ascending or descending order based on the frequency content of each pulse. For instance, the bottom or top electrode of the electrodes to be activated within a stimulation frame will be activated first.

[0029] The switching module can be configured to switch between a fixed stimulus frame rate and a variable stimulus frame rate based on one or more audio components, whereby the one or more audio components may include envelope information and / or features extracted from the audio signal. For example, the switching module can be configured to switch to a fixed stimulus frame start when the fundamental frequency of the stimulus pulse is higher than a frequency threshold and the fundamental frequency energy is higher than an energy threshold, and the switching module can be configured to switch to a variable stimulus frame start when the fundamental frequency of the stimulus pulse is lower than a frequency threshold and the fundamental frequency energy is higher than an energy threshold.

[0030] The switching module can be configured to switch between a fixed stimulus frame rate and a variable stimulus frame rate based on one or more audio components, wherein the one or more audio components may include envelope information and / or features extracted from the audio signal. For example, the switching module can be configured to switch to a fixed stimulus frame start when the fundamental frequency of the stimulus pulse is higher than a frequency threshold, and the switching module can be configured to switch to a variable stimulus frame start when the fundamental frequency of the stimulus pulse is lower than a frequency threshold.

[0031] The switching module can be configured to switch to a fixed stimulation frame start when the energy in dB is higher than an energy threshold, and the switching module can be configured to switch to a variable stimulation frame start when the energy in dB is lower than an energy threshold.

[0032] If at least one of the multiple stimulus pulses has a fundamental frequency energy higher than the energy threshold and a fundamental frequency lower than the frequency threshold, all multiple stimulus pulses are frame-coded using a variable stimulus rate.

[0033] If at least one stimulus pulse in a group of stimulus pulses has a fundamental frequency energy higher than the energy threshold and the fundamental frequency is lower than the frequency threshold, then all of the stimulus pulses in that group are frame-coded using a variable stimulation rate.

[0034] The frequency threshold can be between 100 Hz and 4000 Hz, 300 Hz and 3500 Hz, or 30 Hz and 3000 Hz.

[0035] The energy threshold can be between -40 dB and -20 dB, -20 dB and -30 dB, or -25 dB and -30 dB.

[0036] The switching module can be configured to switch to a fixed stimulus frame start when the audio component is related to speechless speech, or to switch to a variable stimulus frame start when the audio component is related to speech. The processor unit is configured to determine whether the acoustic signal includes speech or speechless speech based on analysis of one or more audio components extracted from samples of multiple frequency-limited audio signals and their spectra. For example, if the audio signal includes speech, and the multiple stimulus pulses of one or more stimulus frames include a fundamental frequency below a frequency threshold, the switching module is configured to switch to a variable stimulus frame start; however, if the audio signal is related to speechless speech, and the multiple stimulus pulses of one or more stimulus frames include a fundamental frequency above a frequency threshold, the switching module is configured to switch to a fixed stimulus frame start. Thus, the cochlear implant system can optimize the amount of temporal information provided to the auditory nerve fibers based on the content of the audio signal.

[0037] The processor unit can be configured to provide a first frame encoding scheme and a second frame encoding scheme for encoding multiple stimulus pulse frames into one or more stimulus frames, respectively, for a fixed stimulus frame start and a variable stimulus frame start. Having different frame encoding schemes for the fixed stimulus frame start and the variable stimulus frame start further improves the temporal information provided to the auditory nerve fibers.

[0038] The first frame encoding scheme provides frame encoding of a stimulus frame from a set of stimulus pulses to one or more stimulus frames when the time start of a set of stimulus pulses is equal to or nearly equal to the time start of a fixed stimulus frame of one or more stimulus frames. In this example, the processor unit is configured to provide frame encoding of a stimulus frame from a set of stimulus pulses to one or more stimulus frames at time t. fr The stimulation frame begins at a fixed rate, and the stimulation pulse occurs at time t. sp Upon arrival, the stimulus pulse is frame-encoded into t fr and t sp The stimulus frame with the smallest time difference between them.

[0039] For example, a group of stimulus pulses in a plurality of stimulus pulses may include subsequent stimulus pulses and preceding stimulus pulses, with the subsequent stimulus pulses occurring temporally after the preceding stimulus pulses. A second-frame encoding scheme may include encoding the preceding stimulus pulse into a first stimulus frame of one or more stimulus frames, wherein the frame-encoded preceding stimulus pulse ends at an offset time, and, at a second frame time, when the second frame time is after the analysis window, encoding the subsequent stimulus pulse into a second stimulus frame of one or more stimulus frames, the analysis window starting at the offset time and having the maximum possible duration of the stimulus frame.

[0040] The second-frame encoding scheme results in the initiation of variable stimulus frames, as each stimulus frame is generated based on the arrival time of the stimulus pulse.

[0041] By enabling frame coding with variable stimulation frame starts, the cochlear implant system can arrange stimulation frames to more explicitly encode temporal information of acoustic signals including speech sounds without requiring a speech detection algorithm. By avoiding speech detection algorithms, the signal processing executed by the processor unit becomes simpler, whereby the power consumption of the cochlear implant system is reduced.

[0042] The processor unit is configured to switch between using a waveform sampling strategy, a feature extraction strategy, and / or a mixture of the waveform sampling strategy and the feature extraction strategy.

[0043] The switching module may be configured to switch to a variable stimulation frame start when one or more audio components comprise temporal fine structure (TFS) information, and / or the switching module may be configured to switch to a fixed stimulation frame start when one or more audio components comprise envelope information.

[0044] A frame time period can be defined between the starts of two consecutive stimulation frames of one or more stimulation frames. When the frame coding has a variable stimulation frame start, the frame time period varies between at least two consecutive stimulation frames of the one or more stimulation frames. When the frame coding has a fixed stimulation frame start, the frame time period is fixed among the one or more stimulation frames.

[0045] One stimulation frame among the plurality of stimulation frames and each of the one or more stimulation frames may satisfy the following conditions: - the one or more stimulation frames may be organized in ascending or descending order such that the stimulation frame allocated to the most basal electrode is always transmitted first to the implantable unit; - stimulation frames allocated to EAFx (Electrode Apical First) will always be placed before those allocated to EAFy (another specific electrode of the electrode array) if x<y. When the stimulation frame allocated to the most apical electrode is reached, the next frame can be generated; - for inter-frame interval t f (T fmin <t f <T fmax ), there are an upper limit and a lower limit. t f is defined as the time between the end of the last stimulation pulse of one stimulation frame and the start of the first stimulation pulse of the next frame; - for intra-frame interval t p (T pmin <t p <T pmax ), there are an upper limit and a lower limit. t p is defined as the time between the end of one stimulation pulse and the start of a subsequent stimulation pulse in the same stimulation frame.

[0046] In one stimulation frame, if EAFy has a stimulation pulse and EAFx does not have a stimulation pulse (x<y), a notification pulse with a duration of T s shall be inserted before the corresponding pulse of EAFy (the bottommost electrode assigned with a stimulation pulse), to notify the implant that EAFx is skipped. If there are multiple electrodes not assigned with stimulation pulses before EAFy, the same rule shall be applied to each of these electrodes.

[0047] The start of a variable stimulation frame may be equal to the time start of the bottommost electrode assigned with a stimulation pulse. For example, a first stimulation frame may start with the stimulation pulse assigned to the topmost electrode, that is, the electrode assigned to the lowest frequency range. The time when the stimulation pulse reaches the bottommost electrode in one stimulation frame is recorded as t0. In addition, if a stimulation pulse occurs in EAFx while the previous stimulation pulse is in EAFy (x<y) and EAFx is assigned to a lower frequency range than EAFy, the first stimulation frame ends at the end of the previous stimulation pulse, and a second stimulation frame is inserted to encode the stimulation pulse in EAFx. This process is repeated until all of the plurality of stimulation pulses have been frame-encoded. The start of a variable stimulation frame may be determined between the end of a preceding stimulation frame and the start of a subsequent stimulation frame, and the start of the subsequent frame is determined by the offset time t of the stimulation pulse of the bottommost electrode in the preceding stimulation frame f .

[0048] An analysis window may be defined as starting from t f and having a time length of t A , wherein t A may be the maximum possible time length of a stimulation frame, and t f is the offset time of the stimulation pulse in the preceding stimulation frame, that is, the time when the stimulation pulse is turned off.

[0049] Among the plurality of stimulation pulses, those stimulation pulses that arrive at time t sp are subjected to frame encoding sequentially in ascending order starting from the topmost electrode and transmitted to the electrode array in descending order starting from the bottommost electrode within the stimulation frame, where t sp is within the first analysis window.

[0050] A low-frequency group may include several electrodes of an electrode array, each of these electrodes is assigned to a frequency range within a main low-frequency range. For example, electrodes EAF0 to EAF7 may be part of the low-frequency group, since they are all assigned to frequencies within the main low-frequency range.

[0051] A high-frequency group may include several electrodes from an electrode array, each of which is assigned to a frequency range within the main high-frequency range. For example, electrodes EAF8 through EAF19 may be part of a high-frequency group, as they are all assigned to frequencies within the main high-frequency range. First, all electrodes in the low-frequency group assigned to stimulation pulses are encoded. Once all assigned electrodes in the low-frequency group have been encoded within the analysis window, all assigned electrodes in the high-frequency group will be encoded within that analysis window. However, if an assigned electrode in the high-frequency group cannot be frame-coded within that analysis window, it is either deleted or frame-coded in the next stimulation frame within the next analysis window.

[0052] In situations where multiple stimulus pulses may be frame-encoded simultaneously, the processor unit can be configured to prioritize the frame encoding of the stimulus pulses to avoid conflicts between them. The processor unit can be configured to prioritize the stimulus pulses to be frame-encoded from among the multiple stimulus pulses, and can select those stimulus pulses with the highest priority for frame encoding. The processor unit can be configured to prioritize the stimulus pulses based on the frequency content or energy level of each stimulus pulse among the multiple stimulus pulses. By establishing prioritization based on frequency content and / or energy level, the temporal information transmitted to the auditory nerve will be best suited to achieve the best possible improvement in the user's hearing ability.

[0053] The frequency content of the first stimulus pulse in a plurality of stimulus pulses includes a first frequency range, and the frequency content of the second stimulus pulse in a plurality of stimulus pulses includes a second frequency range. The first frequency range includes frequencies lower than those in the second frequency range. The processor unit is configured to prioritize the first stimulus pulses before the second stimulus pulses. By prioritizing lower frequencies, the processor unit's ability to obtain optimal pitch timing information is not degraded when conflicts may occur between stimulus pulses.

[0054] The first frequency range and / or the second frequency range may include the fundamental frequency.

[0055] The processor unit can be configured to prioritize the first stimulus pulse over the second stimulus pulse when the energy level of the first stimulus pulse in a plurality of stimulus pulses is higher than that of the second stimulus pulse in a plurality of stimulus pulses. Thus, the possibility of prioritizing speech over noise in the acoustic signal is improved through the aforementioned prioritization.

[0056] The processor unit can be configured to time-shift the preceding stimulus pulse of a plurality of stimulus pulses when the sample time interval between the start of the preceding stimulus pulse and the end of the subsequent stimulus pulse is equal to or less than a sample time threshold. Thus, no conflict will occur between two stimulus pulses; however, time shifting is only permitted when the time shift amount is less than a predetermined threshold.

[0057] Cochlear implantation systems may include dynamic energy controllers that provide a stable energy supply from the external unit to the implantable unit when frame coding begins based on variable stimulation frames.

[0058] One or more stimulation frames may include a precharge pulse, which may be modified by a dynamic energy controller to maintain a charge level within the implantable cell between a minimum and a maximum charge level. And / or, the dynamic energy controller is configured to insert a precharge frame between two stimulation frames if the modification of the precharge pulse is insufficient to achieve the minimum charge level. A precharge frame includes one or more precharge pulses.

[0059] On the other hand, the processor unit is configured to generate multiple stimulation pulses based on sampling the full frequency range of an audio signal with a time start, and to determine one or more audio components for sampling the full frequency range of the audio signal. The one or more audio components are more precisely identified as the acoustic interaction occurring between different fundamental frequencies in the audio signal, since the separation of different fundamental frequencies is not currently occurring. By including the acoustic interaction, the timing information provided by the one or more audio components becomes closer to the acoustic signal received by the receiving unit.

[0060] It is well known that cochlear implant patients report different pitch levels when a given electrode is stimulated at different rates. If the stimulation rate of a particular electrode increases, the pitch level will also increase linearly up to a certain frequency, such as approximately 1000 Hz. By changing the position of the electrode, the same trend is observed again; however, the absolute values ​​of the pitch levels will differ. This means that position encoding (cochlear depth) and time encoding (stimulation rate) determine the pitch level ultimately assessed in cochlear implant patients.

[0061] Pitch scales are more sensitive to low stimulus rates than to high stimulus rates. For example, if the frequency increases from 100 Hz to 200 Hz, the pitch scale increases from 0 to 30. However, when the frequency increases from 200 Hz to 300 Hz, the pitch scale only increases by 10.

[0062] Now, suppose the center frequencies of the two lowest channels in the TFS coding strategy are approximately 60 Hz and 120 Hz, respectively. Each of these channels transmits a very precise impulse event (e.g., instantaneous frequencies of 60.2 Hz and 120.5 Hz) corresponding to the phase and frequency of the channel-filtered audio signal. However, when both channels are activated, the cochlear region corresponding to the first channel (i.e., center frequency 60 Hz) will not actually observe a precise impulse event (i.e., 62.2 Hz), but rather the sum of the two channels.

[0063] The consequences of this summation of pitch perception are very significant because pitch scales are highly sensitive to gradient changes in low-stimulus-rate pulses.

[0064] As mentioned earlier, transmitting pitches of approximately 60 Hz to cochlear implant patients may have certain technical limitations: for example, high sensitivity to pitch changes and interactions between channels. If electrodes inserted at different depths in the cochlea are stimulated at different rates, it might be possible to obtain the same pitch scale using them. Of all these different configurations, only one will conform to the physiology of a normal ear, but due to technical limitations, we may have to choose another configuration and a slightly different way of transmitting pitch to cochlear implant patients than is perceived by people with normal hearing.

[0065] In practice, to enable pitch perception in cochlear implant patients, it is not necessary to use pulse events at the fundamental frequency and multiple frequency-encoded electrodes as in normally hearing individuals. Instead, pitch perception can be provided to cochlear implant patients by modifying the pulse rate of all (or some) channels of the CI device in a similar manner as a function of pitch. This can be achieved through a modified CIS coding strategy that includes variable stimulation rates.

[0066] In CIS-based strategies, all channels are stimulated at the same rate with perfect accuracy. Therefore, in such encoding strategies, interference between adjacent channels does not modify the overall perceived pitch. Because channels have the same stimulation rate in CIS-based strategies, the overall perceived pitch constitutes robust pitch perception, insensitive to the content of the input signal. However, the problem with classic CIS strategies is that, since channels are always stimulated at a fixed rate, the perceived pitch remains almost constant under different conditions.

[0067] One approach to achieving robust yet variable pitch perception is to modify the classic CIS (Chronic Intensive Stimulation) to include a modified CIS coding strategy that incorporates variable stimulation rates. In this modified strategy, all channels or electrodes are stimulated at the same rate, however, this rate is modulated temporally as a function of pitch. This method effectively transmits both positional and temporal encoded information to the patient.

[0068] The processor unit may include a pitch estimator configured to receive an input signal including an audio signal and estimate the pitch value of the input signal. Based on the estimated pitch value, a target stimulation rate function is determined, which may be an exponential function that can be adjusted for the patient for optimal performance. Since the stimulation rate of the first frame coding scheme (e.g., CIS) varies with pitch, there may be some loudness effect on the pulses generated by the first frame coding scheme. The processor unit may then include a loudness compensator configured to compensate for the loudness effect based on the stimulation rate.

[0069] In one example, the stimulation rate decreases / increases as the estimated pitch decreases / increases. Both channels can have the same stimulation rate, so the current flowing from one electrode to the other does not affect the perceived pitch due to the pitch estimator and loudness compensator.

[0070] The processor unit is configured to separate the electrodes of a channel or electrode array into two main groups. The first group includes channels or electrodes assigned to frequencies below or equal to 200 Hz, and the second group includes channels or electrodes assigned to frequencies above 200 Hz. The two groups of electrodes can operate at two different stimulation rates. For example, the stimulation rate of the second group may always be twice that of the first group. In this case, the stimulation of the first group will be closer to the physiological stimulation rate. For example, when the second group of electrodes increases the stimulation rate from 400 pps to 500 pps, the first group of electrodes increases its frequency from 200 pps to 250 pps.

[0071] If the stimulation rate of the second group is not a multiple of that of the first group, a physical distance can be created between the two groups on the electrode array. This reduces the current interaction between the two groups of electrodes. Attached Figure Description

[0072] Various aspects of the invention will be best understood from the following detailed description taken in conjunction with the accompanying drawings. For clarity, these drawings are schematic and simplified, showing only the details necessary for understanding the invention while omitting other details. Throughout the specification, the same reference numerals are used for the same or corresponding parts. Features of each aspect may be combined with any or all features of other aspects. These and other aspects, features, and / or technical effects will be apparent from and illustrated in the following figures, wherein: Figure 1 The cochlear implant system is shown; Figure 2A and 2B The cochlear implant system is shown; Figures 3A-3D Different examples of frame coding are shown; Figure 4A and 4B Examples of the spectra of spoken and non-spoken speech are shown; Figure 5 An example of a cochlear implant system is shown. Detailed Implementation

[0073] The detailed description below, taken in conjunction with the accompanying drawings, serves as a description of various different configurations. This detailed description includes specific details to provide a thorough understanding of several different concepts. However, it will be apparent to those skilled in the art that these concepts can be implemented without these specific details. Several aspects of the apparatus and method are described by various different blocks, functional units, modules, elements, circuits, steps, processes, algorithms, etc. (collectively, “elements”). Depending on the specific application, design constraints, or other reasons, these elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0074] Hearing aids can be or include devices adapted to improve or enhance a user's hearing ability by receiving sound signals from the user's environment, generating corresponding audio signals, possibly modifying the audio signals, and providing the possibly modified audio signals as audible signals to at least one ear of the user. Improving or enhancing a user's hearing ability may include compensating for the specific hearing loss of an individual user. "Hearing aid" may also mean a device adapted to electronically receive audio signals, such as a wearable device, headset, or earphone, which may modify the audio signals and provide the possibly modified audio signals as audible signals to at least one ear of the user. The audible signals may be provided as acoustic signals radiated into the user's outer ear, or as acoustic signals transmitted as mechanical vibrations through the bone structures of the user's head and / or through the user's middle ear to the user's inner ear, or as electrical signals transmitted directly or indirectly to the user's cochlear nerve and / or auditory cortex.

[0075] Hearing aids are suitable for wearing in any known manner. This may include: i) placing the hearing aid unit behind the ear (having a tube to guide airborne sound signals into the ear canal or having a receiver / speaker positioned close to or within the ear canal and connected via a wire (or wirelessly) to the behind-the-ear unit), such as behind-the-ear hearing aids; and / or ii) placing the hearing aid wholly or partially within the user's auricle and / or ear canal, such as in-the-ear (ITE) or in-the-canal (ITC) / deep-in-the-canal (DIC) hearing aids; or iii) positioning the hearing aid unit to be connected to a fixation device implanted into the skull, such as a bone-anchored hearing aid or a cochlear implant; or iv) positioning the hearing aid unit as a wholly or partially implanted unit, such as a bone-anchored hearing aid or a cochlear implant system. Hearing aids may be implemented in a single unit (shell) or in multiple units individually connected to each other.

[0076] A “hearing system” refers to a system comprising one or two hearing aids, and a “binaural hearing system” refers to a system comprising two hearing aids, wherein the hearing aids are adapted to provide audio signals to both ears of a user in a cooperative manner. A hearing system or a binaural hearing system may also include one or more assistive devices that communicate with at least one hearing aid, which affect the operation of the hearing aid and / or benefit from its functionality. A wired or wireless communication link is established between at least one hearing aid and the assistive device to allow the exchange of information (such as control and status signals, possibly audio signals). The assistive device may include at least one of the following: a remote control, a remote microphone, an audio gateway device, a wireless communication device such as a mobile phone (e.g., a smartphone) or a tablet computer or another device (e.g., including a graphical interface), a broadcasting system, a car audio system, a music player, or a combination thereof. The audio gateway device may be adapted to receive multiple audio signals, such as from an entertainment device such as a TV or music player, from a telephone device such as a mobile phone, or from a computer such as a PC. The assistive device may also be adapted (e.g., enabling the user) to select and / or combine appropriate signals from the received audio signals (or combinations of signals) to transmit to at least one hearing aid. The remote control is suitable for controlling the functions and operation of at least one hearing aid. The functions of the remote control can be implemented in a smartphone or other (e.g., portable) electronic device, which may run an application (APP) to control the functions of at least one hearing aid.

[0077] Generally, a hearing aid includes i) a receiving unit, such as a microphone, for receiving acoustic signals from the user's surroundings and providing a corresponding input audio signal, and / or ii) a receiving unit for electronically receiving the input audio signal. The hearing aid also includes a signal processing unit for processing the input audio signal and an output unit for providing an audible signal to the user based on the processed audio signal.

[0078] The receiving unit may include multiple input microphones, for example, for providing direction-dependent audio signal processing. Such directional microphone systems are adapted to (relatively) amplify a target acoustic source among a large number of acoustic sources in a user's environment and / or attenuate other sound sources (such as noise). In one aspect, the directional system is adapted to detect (e.g., adaptively detect) the direction from which a specific portion of the microphone signal originates. This can be achieved using methods conventionally known. The signal processing unit may include an amplifier adapted to apply a frequency-dependent gain to the input audio signal. The signal processing unit may also be adapted to provide other related functions such as compression, noise reduction, etc. The output unit may include an output converter, such as a speaker / receiver for providing airborne acoustic signals transdermally or percutaneously to the skull, or a vibrator for providing structure-borne or fluid-borne acoustic signals. In some hearing aids, the output unit may include one or more output electrodes, such as those in a cochlear implant, for providing electrical signals.

[0079] Cochlear implants typically include: i) an external portion for picking up and processing sound from the environment and determining a pulse sequence for electrode stimulation based on the current input sound; ii) a (typically wireless, such as inductive) communication link for simultaneously transmitting information about the stimulation sequence and transmitting energy to the implant portion; and iii) an implant portion that enables stimulation to be generated and applied to multiple electrodes, which may be implanted at different locations in the cochlea, thereby enabling stimulation of different frequencies within the auditory range. Such systems are described, for example, in US 4,207,441 and US 4,532,930.

[0080] On the one hand, hearing aids include multi-electrode arrays, for example, in the form of a carrier containing multiple electrodes adapted to be located in the cochlea and close to the user's auditory nerve. This carrier is preferably made of a flexible material to enable proper positioning of the electrodes in the cochlea, allowing the electrodes to be inserted into the recipient's cochlea. Preferably, the individual electrodes are spatially distributed along the length of the carrier, thereby providing a corresponding spatial distribution along the cochlear nerve in the cochlea when the carrier is inserted into the cochlea.

[0081] Now for reference Figure 1 The illustration shows a cochlear implant system 1, which includes an external unit 2 capable of percutaneous communication with the user's skin and an implantable unit 4. The implantable unit 4 is connected to an electrode array 5, which is configured to be inserted into the user's cochlea 6. The electrode array may include a plurality of electrodes 7.

[0082] Figure 2A and 2B Different examples of a cochlear implant system 1 are shown, which includes a receiving unit 10 configured to receive acoustic signals and transmit audio signals based on the acoustic signals. In this specific example, the receiving unit includes a microphone. Figure 2AIn this system, system 1 also includes a processor unit 12 configured to receive an audio signal and divide the audio signal into multiple frequency band-limited audio signals (FB1, FB2, FB3, and FBN) via a filter bank 14, and generate multiple stimulation pulses based on sampling 16 of the multiple frequency band-limited audio signals (FB1, FB2, FB3, and FBN). Each of the multiple frequency band-limited audio signals (FB1, FB2, FB3, and FBN) is sampled with a time onset, and one or more audio components 24 are determined for each of the multiple frequency band-limited audio signals after sampling. The processor unit 12 is connected to an electrode array 5 including multiple electrodes 7, wherein the processor unit 12 includes a switching module 18 configured to switch between using a fixed stimulation frame onset 20 and using a variable stimulation frame onset 22 for encoding multiple stimulation pulse frames into one or more stimulation frames, wherein the switching module 18 is configured to switch between the fixed stimulation frame onset 20 and the variable stimulation frame onset 22 based on one or more audio components 24, wherein the electrode array 5 is configured to stimulate the auditory nerve of the user of the cochlear implant system 1 based on the frame-encoded multiple stimulation pulses. One or more audio components are determined by sampling process 16, wherein the one or more audio components may include at least one of the following: fundamental frequency, harmonic frequency (harmonic frequency) relative to the fundamental frequency of the audio signal, the fundamental frequency and harmonic timing of the audio signal and the energy to the left and right, and / or the fundamental frequency and harmonic timing of the audio signal and the phase to the left and right.

[0083] Alternatively, the switching module 18 is configured to switch to the start of a variable stimulus frame when one or more audio components include temporal fine structure (TFS) information of the acoustic signal, and / or the switching module is configured to switch to the start of a fixed stimulus frame when one or more audio components include envelope information of the acoustic signal.

[0084] exist Figure 2B In the filter bank, there is a filter bank 14 including a fast Fourier transform unit 34, which is configured to perform a Fourier transform of the audio signal across the full frequency range, resulting in sampling of the audio signal across the full frequency range, wherein one or more audio components are determined based on the sampling of the audio signal across the full frequency range.

[0085] Figures 3A-3D Different examples of frame encoding performed by processor unit 12 are shown. Figure 3AIn this configuration, the switching module 18 receives multiple stimulation pulses 32 from multiple frequency band-limited audio signals (FB1-FBN) and encodes these pulses 32 frames into a stimulation frame F0. In this specific example, the top electrode EAF1, as well as electrodes EAF2, EAF3, and EAFN, are activated by receiving stimulation pulses (P1 – P4) from the multiple stimulation pulses 32. EAF1 is stimulated first, and EAFN is stimulated at the end of stimulation frame F0. The stimulation pulses (P1 – P4) are transmitted to the electrode array 5 such that the bottom electrode EAFN is always transmitted to the implantable unit first, with the same timing between stimulation pulses, as shown in the electrical stimulation spectrum diagram 19.

[0086] Figure 3B An example is shown where the switching module 18 uses a variable stimulus frame start 22 to frame encode multiple stimulus pulses 32. The processor unit 12 frames a first group of stimulus pulses (P1-P4) as a first stimulus frame F0. A second stimulus frame F1 is generated when the frequency of the next stimulus pulse P5 to be framed is higher than the previously framed stimulus pulse P4. Within the second stimulus frame F1, the processor unit 12 frames a second group of stimulus pulses including P5 and P6. A third stimulus frame F2 is generated when the frequency of the next stimulus pulse P7 is higher than the previously framed stimulus pulse P6. Within the third stimulus frame F2, the processor unit 12 frames a third group of stimulus pulses including P7-P9. However, conflicts are observed between all three stimulus pulses in the third stimulus frame F2. The processor unit is configured to shift the subsequent stimulus P8 to P8A such that the pulse time difference between the start of the preceding stimulus pulse P8 and the end of the subsequent stimulus pulse P7 is higher than the maximum time difference or lower than the minimum time difference. In this specific example, P8 is time-shifted to P8A, and P9 is time-shifted to P9A. The frame encoding result of the multiple stimulus pulses 32 is a frame time interval (33A and 33B) that varies between one or more stimulus frames (F0, F1, F2).

[0087] Figure 3CAn example is shown where the switching module 18 uses a fixed stimulation frame start 20 to frame-encode multiple stimulation pulses 32. The processor unit 12 frames-encodes a first group of stimulation pulses (P1-P4) as a first stimulation frame F0. A second stimulation frame F1 is generated when the frequency of the next stimulation pulse P5 to be frame-encoded is higher than the previously frame-encoded stimulation pulse P4. Within the second stimulation frame F1, the processor unit 12 frames-encodes a second group of stimulation pulses including P5 and P6. A third stimulation frame F2 is generated when the frequency of the next stimulation pulse P7 is higher than the previously frame-encoded stimulation pulse P6. Since one or more audio components have instructed the switching module to switch to the fixed stimulation frame start 20, the processor unit needs to time-shift stimulation pulse P7 to keep the frame time intervals (33A, 33B) between stimulation frames (F0, F1, F2) fixed (33B). Furthermore, the processor unit must time-shift P8 and P9 to maintain correct timing between other stimulation pulses P8 and P9 so that the bottom electrode is always delivered to the implantable unit first.

[0088] In the variable and fixed stimulus frame start examples, analysis window 49 determines which stimulus pulses 32 will be frame-encoded in the next stimulus frame F1. Figure 3D In the analysis, the time offset t of the bottom stimulus pulse P3 in the previous stimulus frame F0 is used to determine the window's position. f The stimulus frame begins at a specific time and has the maximum possible duration. In this example, stimulus pulses P1-P3 are frame-encoded within the first stimulus frame F0, stimulus pulses within the analysis window 40 are frame-encoded within the second stimulus frame F1, and stimulus pulse P7 outside the analysis window 40 will be frame-encoded in the next stimulus frame (not shown).

[0089] Figure 4A and 4B Examples of the spectra of spoken and non-spoken speech are shown, including a frequency threshold F for determining whether the audio signal and one or more audio components 24 are related to spoken or non-spoken speech. th and energy threshold E th In both examples, the energy threshold E th Set to -30 dB of the normalized speech energy signal, the frequency threshold is F th Set to 2000 Hz, Figure 4A From this, it can be seen that for speech containing voice, the value is higher than E. th The energy peak occurs at frequencies below the frequency threshold. Figure 4B From this, it can be seen that for speech without voice, the value is higher than E. thThe energy peak occurs at frequencies above a frequency threshold. During the sampling period of each of multiple frequency-band-limited audio signals, one or more audio components are determined for each of the multiple stimulus pulses. In this example, at least one of the multiple stimulus pulses has multiple audio components, including a fundamental frequency and an energy at the fundamental frequency. The energy at the fundamental frequency is higher than E. th And the fundamental frequency is lower than F th At that time, multiple audio components are associated with speech containing voice, thus the switching module ensures that the frame encoding provided by the processor unit starts with a variable stimulus frame.

[0090] In another example, if at least one of the multiple stimulation pulses has a frequency higher than E at the fundamental frequency. th Energy and below F th If the fundamental frequency is such that multiple stimulation pulses are all frame-coded using a variable stimulation rate, then all stimulation pulses will be frame-coded using a variable stimulation rate.

[0091] In yet another example, if at least one of the stimulation pulses in a group of stimulation pulses has a frequency higher than E at the fundamental frequency. th Energy and below F th If the fundamental frequency is such that all stimulation pulses in this group are frame-coded using a variable stimulation rate.

[0092] Figure 5 An example of a cochlear implant system is shown. Because the stimulation rate, along with the content of one or more audio components 24 such as pitch, varies, it will have a certain loudness effect on the resulting stimulation pulse. In this example, the processor unit includes a loudness compensator 30 configured to compensate for the loudness effect based on a measurement of one or more audio components 24. For example, the processor unit 12 is configured to measure one or more audio components, and based on this measurement, the processor unit 12 is configured to determine compensation for the loudness effect (i.e., the loudness change due to the change in stimulation rate) and determine the stimulation rate of the variable stimulation frame initiation 22. Figure 5 In this example, the loudness compensator receives one or more audio components 24 and provides compensation for the loudness effect to the frame encoding 31 of multiple stimulus pulses. In this example, the variable stimulus frame initiation 22 determines the stimulus rate based on one or more audio components 24.

[0093] Unless explicitly stated otherwise, the singular forms “a” and “the” used herein include the plural forms (i.e., meaning “at least one”). It should be further understood that the terms “having,” “comprising,” and / or “including” as used in the specification indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. It should be understood that, unless explicitly stated otherwise, when an element is referred to as “connected” or “coupled” to another element, it may be a direct connection or coupling to the other element, or there may be intermediate inserting elements. The term “and / or” as used herein includes any and all combinations of one or more of the listed related items. Unless explicitly stated otherwise, the steps of any method disclosed herein do not necessarily have to be performed in the exact order disclosed.

[0094] It should be understood that references to "an embodiment," "an embodiment," "an aspect," or "may" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Furthermore, particular features, structures, or characteristics may be suitably combined in one or more embodiments of the invention. The foregoing description is provided to enable those skilled in the art to implement the various aspects described herein. Various modifications will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Unless expressly stated, elements referred to in the singular do not mean "one and only one," but rather "one or more." Unless expressly stated, the term "some" means one or more.

[0095] Therefore, the scope of this invention should be determined based on the claims.

Claims

1. A cochlear implantation system, comprising: The receiving unit is configured to receive acoustic signals and transmit audio signals based on the acoustic signals; The processor unit is configured to receive an acoustic signal and divide the audio signal into multiple frequency band-limited audio signals and generate multiple stimulation pulses based on sampling of the multiple frequency band-limited audio signals, wherein each of the multiple frequency band-limited audio signals is sampled with time start, and one or more audio components are determined for each of the sampled multiple stimulation pulses; An electrode array comprising multiple electrodes; The processor unit includes a switching module configured to encode multiple stimulation pulse frames into one or more stimulation frames and switch between using a fixed stimulation frame start and using a variable stimulation frame start. The switching module is configured to switch between the fixed stimulation frame start and the variable stimulation frame start based on one or more audio components. The electrode array is configured to stimulate the auditory nerve of the user of the cochlear implant system based on the frame-encoded multiple stimulation pulses.

2. The cochlear implant system according to claim 1, wherein, When using a fixed stimulus frame start, the stimulus frame start is the same for each of one or more stimulus frames; when using a variable stimulus frame start, the stimulus frame start varies between each of one or more stimulus frames.

3. The cochlear implant system according to any of the preceding claims, wherein, The processor unit is configured to generate an event sequence for each of a plurality of frequency band-limited audio signals, wherein each event in the event sequence is detected when the phase of the frequency band-limited audio signal exceeds a phase threshold, wherein the phase threshold is any value between 0 and 2π, and the time start is equal to the event rate of the event sequence.

4. The cochlear implant system according to claim 1 or 2, wherein, One or more audio components are determined by the processor unit, wherein the one or more audio components include at least one of the following: fundamental frequency, harmonic frequency relative to the fundamental frequency of the audio signal, energy of the audio signal at and around the fundamental frequency and harmonic frequency, and / or phase of the audio signal at and around the fundamental frequency and harmonic frequency.

5. The cochlear implant system according to claim 4, wherein, The switching module is configured to switch to the start of a fixed stimulation frame when the fundamental frequency is higher than the frequency threshold, and the switching module is configured to switch to the start of a variable stimulation frame when the fundamental frequency is lower than the frequency threshold; or, the switching module is configured to switch to the start of a fixed stimulation frame when the energy in dB is higher than the energy threshold, and the switching module is configured to switch to the start of a variable stimulation frame when the energy in dB is lower than the energy threshold.

6. The cochlear implant system according to claim 4, wherein, The switching module is configured to switch to a fixed stimulus frame start when the audio component is related to speech without speech, or to switch to a variable stimulus frame start when the audio component is related to speech with speech.

7. The cochlear implant system according to any one of claims 1, 2, 5, and 6, wherein, The processor unit is configured to provide a first frame encoding scheme and a second frame encoding scheme for encoding multiple stimulus pulse frames into one or more stimulus frames for a fixed stimulus frame start and a variable stimulus frame start, respectively, wherein the first frame encoding scheme is different from the second frame encoding scheme.

8. The cochlear implant system according to claim 7, wherein, The first frame coding scheme provides frame coding from a set of stimulus pulses to a stimulus frame when the time start of a set of stimulus pulses is equal to or nearly equal to the start of a fixed stimulus frame of one or more stimulus frames.

9. The cochlear implant system according to claim 7, wherein, A group of stimulation pulses in a plurality of stimulation pulses includes a subsequent stimulation pulse and a preceding stimulation pulse, wherein the subsequent stimulation pulse follows the preceding stimulation pulse in time to be transmitted to the electrode array, wherein the second frame encoding scheme includes: Provides a first frame encoding of the pre-stimulus pulse into one or more stimulation frames within the first stimulation frame, wherein the frame-encoded pre-stimulus pulse is at offset time t. f End; and At the second frame time, when the second frame time is after the analysis window, the second frame encoding of the subsequent stimulus pulse is provided into the second stimulus frame of one or more stimulus frames, the analysis window starts at the offset time and has the maximum possible duration of the stimulus frame.

10. The cochlear implant system according to any one of claims 1, 2, 5, 6, 8, and 9, wherein, The switching module is configured to switch to the start of a variable stimulus frame when one or more audio components include temporal fine structure information of the acoustic signal, and / or the switching module is configured to switch to the start of a fixed stimulus frame when one or more audio components include envelope information of the acoustic signal.

11. The cochlear implant system according to any one of claims 1, 2, 5, 6, 8, and 9, wherein, The processor unit is configured to distinguish the priority of stimulus pulses to be frame-coded among multiple stimulus pulses, wherein the processor unit is configured to select those stimulus pulses with the highest priority for frame coding.

12. The cochlear implant system according to claim 11, wherein, The processor unit is configured to prioritize stimulation pulses based on the frequency content or energy level of each stimulation pulse among a plurality of stimulation pulses.

13. The cochlear implant system according to claim 12, wherein, The first frequency content of the first stimulation pulse among a plurality of stimulation pulses includes a first frequency range, the second frequency content of the second stimulation pulse among a plurality of stimulation pulses includes a second frequency range, the first frequency range includes frequencies lower than the frequencies of the second frequency range, and the processor unit is configured to prioritize the first stimulation pulses before the second stimulation pulses.

14. The cochlear implant system according to claim 12, wherein, The processor unit is configured to prioritize the first stimulation pulse over the second stimulation pulse when the energy level of the first stimulation pulse of the plurality of stimulation pulses is higher than that of the second stimulation pulse of the plurality of stimulation pulses.

15. The cochlear implant system according to claim 9, wherein, Within one or more stimulus frames, the processor unit is configured to time-shift the preceding stimulus pulse when the pulse time difference between the start of the preceding stimulus pulse and the end of the subsequent stimulus pulse is greater than the maximum time difference or less than the minimum time difference.

Images