Self-sound restoring method of artificial cochlea and sound processor

Abstract

The invention discloses an artificial cochlea self-sound restoration method and an artificial cochlea sound processor. The artificial cochlea self-sound restoration method comprises the following steps: dividing digital signals acquired by a bone conduction microphone and an air conduction microphone into signal frames; converting the signal frame from an initial time domain signal to an initial frequency domain signal; judging whether the microphone signal is self sound or not; when the signals collected by the bone conduction microphone and the air conduction microphone are self-sound, after the frequency domain signal frame of the bone conduction microphone and the frequency domain signal frame of the air conduction microphone are superposed, a new frequency domain signal is converted into a new time domain signal, and a signal frame after self-sound restoration is obtained; the sound of an implanter is collected from two channels of air conduction and bone conduction and is mixed and output, so that the accuracy of sound collection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to cochlear implant sound processing. Background Art

[0002] The sound that people hear comes mainly from air conduction and bone conduction. For air conduction, the sound vibrates the eardrum through the external auditory canal, and then is transmitted to the cochlea through the eardrum. Finally, the sound wave signal is converted into an electrical signal on the auditory nerve and transmitted to the cerebral cortex.

[0003] For bone conduction, the sound is transmitted to the skull, and then from the skull to the cochlea, and then perceived in the cerebral cortex through the auditory nerve. Therefore, the sound that people hear is a combination of bone conduction and air conduction.

[0004] For people with cochlear implants, the only sound they hear is the part that is transmitted through the air, so the accuracy of the sound is poor. Summary of the invention

[0005] In view of the above problems, the present invention proposes a method for restoring the self-sound of a cochlear implant and a sound processor, which can collect the implantee's own voice through both air conduction and bone conduction pathways and output the mixed voice.

[0006] To achieve the above object, the present invention provides a method for restoring the self-sound of a cochlear implant, comprising the steps of:

[0007] Dividing the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames;

[0008] Converting a signal frame from an initial time domain signal to an initial frequency domain signal;

[0009] Determine whether the microphone signal is self-sound;

[0010] When the signal collected by the bone conduction microphone and the air conduction microphone is self-sound,

[0011] After the frequency domain signal frame of the bone conduction microphone is superimposed with the frequency domain signal frame of the air conduction microphone, the new frequency domain signal is transformed into a new time domain signal to obtain a signal frame after the sound is restored.

[0012] Optionally, the step of determining whether the bone conduction microphone and the air conduction microphone signals are self-sounds includes:

[0013] Calculate the frequency domain energy of the bone conduction microphone frequency domain signal frame, and determine whether the signal frame is a non-silent frame according to the magnitude of the frequency domain energy;

[0014] If it is a non-silent frame, the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal is calculated, and based on the resonance peak characteristics, it is determined whether the sound collected by the microphone is the self-sound.

[0015] Optionally, in the step of collecting digital signals with a bone conduction microphone, the bone conduction microphone is arranged at a position in contact with the skin on the surface of the skull.

[0016] Optionally, the step of judging whether the signal frame is a non-silent frame by the magnitude of the frequency domain energy includes setting a threshold, comparing the frequency domain energy with the threshold, and if it is less than the threshold, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

[0017] Optionally, the step of determining whether the sound collected by the microphone is self-sound includes comparing whether the first peaks of the resonance peak structures of the two microphone signals are at the same frequency. If so, it is determined that self-sound occurs.

[0018] The present invention also discloses a cochlear implant sound processor, comprising:

[0019] A framing module, which divides the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames;

[0020] The WOLA analysis module converts the signal frame from the initial time domain signal to the initial frequency domain signal; and transforms the new frequency domain signal to the new time domain signal to obtain the signal frame after the sound is restored;

[0021] Self-sound detection module, to determine whether the bone conduction microphone and air conduction microphone signals are self-sound;

[0022] The self-sound restoration module, if it is self-sound, superimposes the frequency domain signal frame of the bone conduction microphone and the frequency domain signal frame of the air conduction microphone to obtain a new frequency domain signal; uses the WOLA analysis module to transform the new frequency domain signal into a new time domain signal to obtain a signal frame after restoring the self-sound.

[0023] Optional, self-acoustic detection module includes:

[0024] The frequency domain short-time energy calculation module calculates the frequency domain energy of the bone conduction microphone frequency domain signal frame, and determines whether the signal frame is a non-silent frame according to the magnitude of the frequency domain energy;

[0025] The resonance peak calculation module, if it is a non-silent frame, the frequency domain short-time energy calculation module receives the frequency domain signal and calculates the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal;

[0026] The self-sound frame judgment module determines whether the sound collected by the microphone is self-sound based on the resonance peak characteristics.

[0027] Optionally, the frequency domain short-time energy calculation module includes an energy comparison unit, which is provided with a threshold value and compares the frequency domain energy with the threshold value. If the frequency domain energy is less than the threshold value, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

[0028] Optionally, the self-sound frame judgment module includes a frequency comparison unit, which compares whether the first peaks of the resonance peak structures of the two microphone signals are at the same frequency. If so, it is determined that self-sound occurs.

[0029] The present invention collects the voice of the implantee through two pathways, air conduction and bone conduction, and outputs the mixed voice, thereby improving the accuracy of voice collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, in which like reference numerals generally represent like components.

[0031] Figure 1 A flowchart of restoring self-sound according to an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the steps of a method for restoring self-sound according to an embodiment of the present invention;

[0033] Figure 3 FIG. 1 is a schematic diagram of the structure of a sound processor according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art. In order to facilitate the understanding of the present invention, components that are not essential to the present invention are also explained in the schematic diagram, but the non-essential parts do not limit the present invention.

[0035] The present invention provides a method for restoring the self-sound of a cochlear implant, comprising the steps of:

[0036] Dividing the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames;

[0037] Converting a signal frame from an initial time domain signal to an initial frequency domain signal;

[0038] Determine whether the microphone signal is self-sound;

[0039] When the signal collected by the bone conduction microphone and the air conduction microphone is self-sound,

[0040] After the frequency domain signal frame of the bone conduction microphone is superimposed with the frequency domain signal frame of the air conduction microphone, the new frequency domain signal is transformed into a new time domain signal to obtain a signal frame after the sound is restored.

[0041] Accordingly, the present invention also provides a cochlear implant sound processor, comprising:

[0042] A framing module, which divides the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames;

[0043] The WOLA analysis module converts the signal frame from the initial time domain signal to the initial frequency domain signal; and transforms the new frequency domain signal to the new time domain signal to obtain the signal frame after the sound is restored;

[0044] Self-sound detection module, to determine whether the bone conduction microphone and air conduction microphone signals are self-sound;

[0045] The self-sound restoration module, if it is self-sound, superimposes the frequency domain signal frame of the bone conduction microphone and the frequency domain signal frame of the air conduction microphone to obtain a new frequency domain signal; uses the WOLA analysis module to transform the new frequency domain signal into a new time domain signal to obtain a signal frame after restoring the self-sound.

[0046] Optionally, the self-sound detection module includes: a frequency domain short-time energy calculation module, which calculates the frequency domain energy of the bone conduction microphone frequency domain signal frame, and determines whether the signal frame is a non-silent frame according to the magnitude of the frequency domain energy; a resonance peak calculation module, if it is a non-silent frame, the frequency domain short-time energy calculation module receives the frequency domain signal, calculates the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal; and a self-sound frame judgment module, which determines whether the sound collected by the microphone is self-sound based on the resonance peak characteristics.

[0047] Optionally, the frequency domain short-time energy calculation module includes an energy comparison unit, which is provided with a threshold value and compares the frequency domain energy with the threshold value. If the frequency domain energy is less than the threshold value, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

[0048] Optionally, the self-sound frame judgment module includes a frequency comparison unit, which compares whether the first peaks of the resonance peak structures of the two microphone signals are at the same frequency. If so, it is determined that self-sound occurs.

[0049] Optionally, it also includes a bone conduction microphone, an air conduction microphone, a housing, a battery, and a transmitting coil.

[0050] The method for restoring the self-sound of a cochlear implant and the cochlear implant sound processor of the present invention are described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the present invention provides a method for restoring self-sound applicable to a cochlear implant, which includes three stages: framing, self-sound detection, self-sound judgment, and self-sound restoration. Figure 2 Includes steps:

[0052] S10: dividing the digital signal collected by the bone conduction microphone and the air conduction microphone into signal frames;

[0053] In this embodiment, in the step of collecting digital signals with the bone conduction microphone, the bone conduction microphone is arranged at a position in contact with the skin on the surface of the skull.

[0054] Framing: Divide the input digital signal into signal frames to facilitate subsequent analysis and processing.

[0055] like Figure 3 As shown, specifically in this embodiment, the frame segmentation module 10 divides the digital signal collected by the bone conduction microphone and the air conduction microphone into signal frames.

[0056] S20: converting the signal frame from an initial time domain signal to an initial frequency domain signal;

[0057] like Figure 3 As shown, the WOLA (Weighted Overlap and Add) analysis module 121 includes a filter bank, an efficient implementation of a DFT (Discrete Fourier Transform) filter bank, which can decompose the sound signal into narrowband signals of different frequencies from 0 to frequency Fs / 2, obtain data of the signal frame spectrum, and is suitable for fast processing of the audio signal. The filter bank of the WOLA analysis module 121 takes the frequency domain signal of the bone conduction microphone as X and the frequency domain signal of the air conduction microphone as A to find the base frequency.

[0058] S30: Determine whether the microphone signal is self-sound.

[0059] like Figure 3 As shown, in this embodiment, the self-sound detection module 20 includes: a frequency domain short-time energy calculation module 122 and a base frequency common peak vibration calculation module 123. The filter bank of the WOLA analysis module 121 converts the signal frame from the initial time domain signal to the initial frequency domain signal. The short-time energy reflects the strength of the signal at different times.

[0060] In this embodiment, specifically, the step of determining whether the bone conduction microphone and the air conduction microphone signals are self-sounds includes:

[0061] First, the frequency domain energy of the bone conduction microphone signal frame is calculated, and whether the signal frame is a non-silent frame is determined by the magnitude of the frequency domain energy.

[0062] Specifically, in this embodiment, the self-sound judgment includes two steps: silent frame judgment and self-sound frame judgment. The premise of restoring the self-sound is to accurately judge the self-sound. Only when the implantee makes a sound himself, it is necessary to start the self-sound restoration algorithm. Frequency domain energy calculation: It can be used to determine whether a frame signal is a silent frame.

[0063] In this embodiment, in the step of judging whether the signal frame is a non-silent frame by the magnitude of the frequency domain energy, in the frequency domain short-time energy calculation module 122, a threshold is first set, and the frequency domain energy is compared with the threshold, if it is less than the threshold, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

[0064] like Figure 3 As shown, after the filter group of the WOLA analysis module 121 converts the air conduction microphone signal frame and the bone conduction microphone signal frame from the initial time domain signal to the initial frequency domain signal, the frequency domain short-time energy calculation module 122 calculates the frequency domain energy of the frame signal. Since the bone conduction microphone has a natural shielding effect on the sound transmitted through the air, the frequency domain energy of the bone conduction microphone is used to judge the silent frame to exclude the interference of background air noise. The threshold value can be set using the empirical value determination method. After the cochlear implant sound processor is worn, the implantee wears the sound processor for several seconds in a quiet and silent state. The frequency domain energy of all signal frames of the bone conduction microphone during this period is summed and then averaged. This value can be set as the threshold value T1 for silent frame judgment. En represents the short-time energy of the speech signal in the nth frame of the signal.

[0065] When En≤T1, it is determined that this frame has no speech, and when En>T1, it is determined that this frame may have speech, that is, it is a non-silent frame.

[0066] Next, if it is a non-silent frame, the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal is calculated, and based on the resonance peak characteristics, it is determined whether the sound collected by the microphone is the self-sound.

[0067] Specifically, the silent frame judgment can determine whether the signal frame is silent, but it cannot accurately determine whether the implantee is speaking. It may also be a signal from the friction between the bone conduction microphone and the scalp, or strong noise in the environment. Therefore, the silent frame judgment alone is not accurate enough. The premise of restoring the self-sound is to accurately judge the self-sound. The self-sound restoration algorithm only needs to be started when the implantee makes a sound. After determining that there may be voice, determine whether the fundamental frequency of the bone conduction microphone frequency domain signal X and the air conduction microphone frequency domain signal A are equal. If they are equal, it is determined that there is self-sound.

[0068] Formant is an important feature that reflects the resonance characteristics of the vocal tract, and it represents the most direct source of pronunciation information. The formant estimation module 123 calculates the short-time power spectrum, and then uses the linear prediction coefficient (LPC) to calculate the LPC spectrum to complete the formant estimation.

[0069] Specifically, the resonance peak estimation module 123 is based on the resonance peak structure of the bone conduction microphone and the air conduction microphone signal frame. Since the bone conduction microphone has high sensitivity in medium and low frequencies and a narrow response frequency band, only the resonance peak with the largest amplitude is selected through resonance peak estimation. The self-sound frame judgment module 125 compares the frequencies of the resonance peaks with the largest amplitudes of the bone conduction microphone and the air conduction microphone signal frames. If they are equal, self-sound occurs, and if they are not equal, self-sound does not occur.

[0070] In addition, in other embodiments, other methods may be used to determine the self-sound, all of which are within the protection scope of the present invention.

[0071] S40: When the signals collected by the bone conduction microphone and the air conduction microphone are self-sound, the frequency domain signal frame of the bone conduction microphone is superimposed with the frequency domain signal frame of the air conduction microphone, and the new frequency domain signal is transformed into a new time domain signal to obtain a signal frame after restoring the self-sound.

[0072] When the implant recipient makes a sound himself, it is determined to be his own sound, and the self-sound restoration algorithm is started. The self-sound restoration module 30 adds the frequency domain signals of the bone conduction microphone and the air conduction microphone.

[0073] The frequency domain signal of the self-sound is C=X+A.

[0074] Next, the new frequency domain signal is transformed into a new time domain signal through the inverse operation of the filter bank of the WOLA analysis module 121, and a signal frame c after restoration of the original sound is obtained.

[0075] The present invention can collect the voice of the implantee through the two pathways of air conduction and bone conduction and output the mixed voice, thus greatly improving the accuracy of the voice.

[0076] The following takes the sound signal sampling rate Fs=16kHz as an example to further illustrate the method of restoring the self-sound.

[0077] Framing: The signal is divided into frames with a frame length of 64, that is, one frame is 4ms.

[0078] The WOLA analysis filter bank converts the initial time domain signal into narrowband signals with different frequencies from 0 to Fs / 2 = 8000 Hz, and obtains 128 initial frequency domain signals, each with a bandwidth of 8000 / 128 = 62.5 Hz:

[0079]

[0080] Among them, X k (m) is the frequency domain signal after WOLA analysis,

[0081] W K =e j2π / K, K is the number of frequency bands (i.e., the number of FFT points, set to 256, but the number of effective frequency bands is the number of FFT points / 2=128), M is the downsampling factor, set to be the same as the signal frame length, which is 64, n is the signal frame, h is the FIR low-pass filter, and each signal frame outputs a set of initial frequency domain signals X(k), k=1,…,128.

[0082] Energy spectrum: E(k)=〖|X(k)|〗^2, k=1,...128.

[0083] The energy spectrum is converted into dB values.

[0084] Silent frame judgment:

[0085] The frequency domain energy of the bone conduction microphone signal frame is calculated as follows:

[0086]

[0087] Where K is the number of effective frequency bands, 128, and the frequency domain energy threshold is set to 15dB. When the frequency domain energy is lower than 15dB, the current signal frame is judged to be a silent frame, otherwise it is a non-silent frame.

[0088] Next, formant estimation: Using the envelope spectrum of the 16 linear prediction coefficients, formants are extracted from the 16 frequency bands representing the first 1000 Hz, and the formant with the largest peak value is selected.

[0089] Self-sound frame judgment: compare the frequencies of the maximum resonance peaks of the bone conduction microphone signal frame and the air conduction microphone signal frame. If they are equal, it is judged that the self-sound occurs. If they are not equal, it is judged that the self-sound does not occur.

[0090] Restoring the self-sound: After determining that the self-sound occurs, the frequency domain signal of the bone conduction microphone signal frame is B(k), k = 1, ..., 128;

[0091] The frequency domain signal of the air conduction microphone signal frame is A(k), k=1,…,128; then the restored self-acoustic signal frame and signal are:

[0092] O(k)=B(k)+A(k),k=1,…,128

[0093] WOLA synthesis: transform the new frequency domain signal O(k) into a new time domain signal to obtain the signal frame after restoring the original sound.

[0094] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for restoring the self-sound of a cochlear implant, characterized in that: Includes steps: Dividing the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames; Converting a signal frame from an initial time domain signal to an initial frequency domain signal; Determine whether the microphone signal is self-sound; When the signal collected by the bone conduction microphone and the air conduction microphone is self-sound, After the frequency domain signal frame of the bone conduction microphone is superimposed with the frequency domain signal frame of the air conduction microphone, the new frequency domain signal is transformed into a new time domain signal to obtain a signal frame after the sound is restored.

2. The method for restoring the natural sound of a cochlear implant according to claim 1, characterized in that: The steps of determining whether the bone conduction microphone and the air conduction microphone signals are self-sounds include: Calculate the frequency domain energy of the bone conduction microphone frequency domain signal frame, and determine whether the signal frame is a non-silent frame according to the magnitude of the frequency domain energy; If it is a non-silent frame, the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal is calculated, and based on the resonance peak characteristics, it is determined whether the sound collected by the microphone is the self-sound.

3. The method for restoring the self-sound of a cochlear implant according to claim 1, characterized in that: In the step of collecting digital signals with a bone conduction microphone, the bone conduction microphone is arranged at a position in contact with the skin on the surface of the skull.

4. The method for restoring the natural sound of a cochlear implant according to claim 2, characterized in that: The step of judging whether the signal frame is a non-silent frame by the magnitude of the frequency domain energy includes setting a threshold, comparing the frequency domain energy with the threshold, and if it is less than the threshold, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

5. The method for restoring the self-sound of a cochlear implant according to claim 4, characterized in that: The step of judging whether the sound collected by the microphone is self-sound includes comparing whether the first peaks of the resonance peak structures of the two microphone signals are at the same frequency. If so, it is judged that self-sound occurs.

6. A cochlear implant sound processor, characterized in that: include: A framing module, which divides the digital signals collected by the bone conduction microphone and the air conduction microphone into signal frames; WOLA analysis module, converts the signal frame from the initial time domain signal to the initial frequency domain signal; and transforming the new frequency domain signal into a new time domain signal to obtain a signal frame after the sound is restored; Self-sound detection module, to determine whether the bone conduction microphone and air conduction microphone signals are self-sound; The self-sound restoration module, if it is self-sound, superimposes the frequency domain signal frame of the bone conduction microphone and the frequency domain signal frame of the air conduction microphone to obtain a new frequency domain signal.

7. The cochlear implant sound processor according to claim 6, characterized in that: The self-sound detection module comprises: The frequency domain short-time energy calculation module calculates the frequency domain energy of the bone conduction microphone frequency domain signal frame, and determines whether the signal frame is a non-silent frame according to the magnitude of the frequency domain energy; The resonance peak calculation module, if it is a non-silent frame, the frequency domain short-time energy calculation module receives the frequency domain signal and calculates the resonance peak structure of the bone conduction microphone signal and the air conduction microphone signal; The self-sound frame judgment module determines whether the sound collected by the microphone is self-sound based on the resonance peak characteristics.

8. The cochlear implant sound processor according to claim 7, characterized in that: The frequency domain short-time energy calculation module includes an energy comparison unit. The energy comparison unit is set with a threshold and compares the frequency domain energy with the threshold. If it is less than the threshold, the frame is marked as a silent frame, otherwise it is marked as a non-silent frame.

9. The cochlear implant sound processor according to claim 8, characterized in that: The self-sound frame judgment module includes a frequency comparison unit, which compares whether the first peaks of the resonance peak structures of the two microphone signals are both at the same frequency. If so, it is determined that the self-sound occurs.

10. The cochlear implant sound processor according to claim 9, characterized in that: It also includes a bone conduction microphone, an air conduction microphone, a shell, a battery, and a transmitting coil.

Images