Cochlea pulse nerve coding method

Abstract

The invention relates to a cochlea pulse neural coding method, which comprises the following steps of: obtaining intrinsic mode functions with narrowband and approximate octave characteristics of nonlinear and unsteady voice signals through an adaptive method of empirical mode decomposition, simulating a frequency selection function of a cochlea, performing Hilbert transform on each intrinsic mode function, extracting an instantaneous frequency, and finally obtaining a cochlea pulse neural coding result. After bandwidth limitation and center frequency modulation, zero crossing point detection is carried out, and a pulse sequence is generated to serve as the input of the pulse neural network. According to the coding method, the cochlea frequency selection function can be effectively simulated, and meanwhile, the calculation complexity is reduced.

Description

Technical Field

[0001] The present application relates to cochlear technology, and more particularly to a cochlear pulse neural coding method. Background Art

[0002] The process of hearing in normal people can be summarized as follows: the mechanical vibration of external speech passes through the external ear and reaches the eardrum, which transmits the corresponding mechanical vibration to the ossicular system of the middle ear. The stapes near the inner side of the ossicular bone is connected to the oval window of the inner ear, and the vibration is transmitted to the cochlea through this connection. The sound waves move in the cochlea filled with lymph fluid in the form of periodic pressure changes of the liquid medium. The vibration of the liquid causes the up and down movement of the basilar membrane. According to the frequency of the speech, the traveling wave reaches the maximum amplitude at different parts of the basilar membrane. The maximum amplitude of high frequency appears at the base of the cochlea, and the maximum amplitude of low frequency appears at the top of the cochlea. In other words, the base of the cochlea encodes high-frequency speech, and the top of the cochlea encodes low-frequency speech. Different speech frequencies cause nerve excitement at different positions along the basilar membrane. Therefore, the cochlea can be considered to be composed of a set of spatially distributed bandpass filters, and the quality factor Q (ratio of center frequency to bandwidth) of the filter is approximately constant. Finally, the speech receptors attached to the basilar membrane also move accordingly. Under the action of inertia, the difference in direction and speed of movement of different structures in the speech receptors produces a force that causes the cilia to bend. The impedance of the bioelectrical circuit that often exists between the cochlear duct and the ciliary cells is changed, thereby modulating the current passing through. The change in current causes the synapses between the speech receptor cells and the auditory nerve fibers to release chemical transmitters, which excite the auditory nerve and emit impulses. After receiving various voices with different characteristics, the impulses emitted by the auditory nerve have different configurations in time and space. They carry relevant voice information and are transmitted to the auditory centers at all levels in sequence. After processing and analysis, hearing is generated.

[0003] The cochlea has a frequency-selective function, and different positions of the cochlea respond to speech of a specific frequency, thereby generating negative and noisy neural signals. In order to simulate this perception process of the cochlea, scientists have proposed computational models such as filter banks and transmission lines. The filter bank model uses multiple bandpass filters covering a certain frequency range to process speech signals, simulating the frequency selection function of the cochlea. At present, electronic cochlear implants used for hearing restoration of severe sensorineural hearing loss are mainly designed based on this model for speech coding strategies, mainly including simulation, amplitude modulation and amplitude-frequency modulation schemes. The simulation scheme mainly sends the collected speech signal directly to the electrode to stimulate the auditory nerve after simple bandpass filtering and appropriate dynamic range compression. This scheme is mainly compression simulation. The principles of amplitude modulation and amplitude-frequency modulation schemes are similar. Both pre-process the speech signal and pass it through multiple bandpass filters for a series of processing. Finally, the output of each bandpass filter is feature-processed and sent to the electrode to stimulate the auditory nerve. The difference is that the amplitude modulation scheme mainly transmits the amplitude information of the original speech, and uses the acquired amplitude information to modulate the fixed-frequency biphasic pulse and then send it to the auditory nerve. The main strategies include: simultaneous simulation stimulation, maximum spectrum peak processor, spectrum peak method, continuous interval sampling, etc. The amplitude-frequency coding strategy extracts the frequency (phase or fine structure) information of the original speech signal while extracting the amplitude information. The amplitude and frequency (phase or fine structure) of the original speech are jointly encoded to achieve the purpose of improving speech recognition. This type of method is mainly amplitude-frequency joint coding. From the perspective of signal processing, whether it is bandpass filtering or wavelet transform, its essence is the traditional Fourier transform. It is OK to process stationary signals, but there may be problems for non-stationary signals such as speech. At the same time, it is limited to varying degrees by window functions and uncertainty principles, so it cannot accurately express the time-frequency characteristics of the signal. In addition, when using bandpass filters and wavelet transforms for signal processing, the filter parameters and mother functions cannot be modified once they are determined, and the adaptability is very poor. The transmission line model divides the cochlea into several segments, each of which is described as a difference equation. This model can accurately describe the physiological characteristics of the cochlea, but unfortunately the amount of calculation is too large. This computational complexity poses a huge challenge to the design of electronic cochlear implants, hearing aids, and machine hearing devices that use this model and have high real-time requirements, and it is difficult to use it in actual systems. Summary of the invention

[0004] In view of the above problems, the present application aims to propose a cochlear pulse neural coding method, which reduces the computational complexity, does not require huge computational effort, and has good adaptability.

[0005] The cochlear pulse neural coding method of the present application comprises: The original speech signal is pre-processed by noise reduction, windowing, and pre-emphasis to obtain a pre-processed signal; The preprocessed signal is subjected to empirical mode decomposition to obtain the intrinsic mode functions of multiple channels arranged in order from low to high frequency; For the intrinsic mode function of each channel, a Hilbert transform is performed on the intrinsic mode function of the channel; based on the Hilbert transform result, the instantaneous frequency information corresponding to the intrinsic mode function of the channel is accurately extracted; according to the frequency perception range of the cochlea, the instantaneous frequency corresponding to the channel is bandwidth-limited to obtain a bandwidth-limited processed signal; the bandwidth and quality factor of the bandpass filter used in the bandwidth-limited processing are the same as those of the intrinsic mode function of the channel; the bandwidth-limited processed signal is frequency-modulated to obtain a frequency-shifted bandwidth-limited processed signal; a sinusoidal signal of the center frequency of the intrinsic mode function of the channel is frequency-modulated to obtain a frequency-modulated signal to achieve frequency shift; the frequency-modulated signal is output to a low-pass filter to achieve mixing, and a mixed slow-varying signal is obtained; the mixed slow-varying signal is subjected to zero-point detection to generate a pulse at the zero-crossing position, and the pulse is the output pulse corresponding to the intrinsic mode function of the channel.

[0006] Preferably, frequency modulation of the bandwidth-limited processing signal is performed by frequency modulating a sinusoidal signal of the center frequency of the intrinsic mode function of the channel to achieve frequency shift.

[0007] Preferably, the output pulses corresponding to the intrinsic mode functions of each channel are sent to the spiking neural network.

[0008] The cochlear pulse neural coding method of the present application adopts the Hilbert-Huang transform method combining empirical mode decomposition and Hilbert transform in the encoding process, which reduces the computational complexity and has the characteristics of adaptive, multi-scale, and multi-resolution extraction of speech features. It can effectively simulate the biological cochlea and accurately express the time-varying characteristics of the original signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of the cochlear pulse neural coding method of the present application; Figure 2 For voice signals Figure 1 Schematic diagram of the method encoding and output to the pulse neural network; Figure 3 is the original speech signal; Figure 4 is the intrinsic mode function IMF obtained after empirical mode decomposition; Figure 5 is the instantaneous frequency corresponding to the intrinsic mode function IMF; Figure 6 The pulse sequence is obtained after the instantaneous frequency is band-limited, center-frequency modulated, and low-pass filtered; Figure 7 This is the situation after local amplification of the pulse sequence. DETAILED DESCRIPTION

[0010] Huang first proposed the new signal analysis method of empirical mode decomposition, which was later improved to form the Hilbert Transform (HT), a time-frequency analysis tool for nonlinear and non-stationary signals called HHT by the NASA Center of the United States. It has been widely used in physiological signals, earthquakes, satellites, geophysics and other fields.

[0011] When using HHT, the empirical mode decomposition (EMD) is first completed through a screening process to decompose the original signal into a series of intrinsic mode functions (IMF) that meet conditions a and b, where: Condition a is that in the entire data sequence, the number of extreme points and the number of zero-crossing points must be equal, or differ by no more than one.

[0012] Condition b is that at any point in time, the envelope average value defined by the local maximum and local minimum of the signal is zero.

[0013] Condition a is similar to the definition of narrowband in the traditional stationary Gaussian process, and condition b changes the traditional global limitation into a local limitation. This not only removes the instantaneous frequency fluctuations caused by waveform asymmetry, but also ensures that the instantaneous frequency obtained by IMF has a clear physical meaning.

[0014] The EMD decomposition process is as follows: (1) First, the original signal is detected by the algorithm All maximum points And all the minimum points , and use the cubic spline function to fit the upper and lower envelopes of the original data sequence; (2) The upper and lower envelopes obtained in step (1) are used to calculate the instantaneous mean , the original signal Subtract the instantaneous time to get the new sequence ; (3) Judgment Whether conditions (a) and (b) are met. If not, the target signal is used as the target signal and steps (1) and (2) are repeated k times until the termination condition is met. , ,at this time, ,in is the envelope mean of k iterations, is the difference between the signal and the (k-1)th envelope mean.

[0015] definition is the first IMF, which is separated from the original signal ; Will As a new initial signal, extract it through the above screening process IMFs; (Formula 1) when When the termination condition is met, it is prompted that it is impossible to get the original signal Separate the IMF from the It can be expressed as a combination of IMFs and residuals: (Formula 2) According to the above analysis, EMD decomposition is an adaptive decomposition based on the signal itself, that is, its decomposition process depends on the change information contained in the signal itself, and can sensitively reflect the change of the signal. At the same time, its screening idea reflects the filtering process of multi-resolution analysis. Each IMF component usually has a certain physical meaning and contains a certain range of characteristic scales, so this feature can be used to filter the target signal. The IMFs of different scales obtained by the screening process have the characteristics of narrowband signals and are local narrowband signals. They can be directly used in HT to obtain the instantaneous amplitude and instantaneous frequency of the original signal. These instantaneous parameters have clear physical meanings and can accurately express the time-varying characteristics of the original signal.

[0016] Then, each IMF component is subjected to Hilbert transform to obtain the energy distribution on the time-frequency plane. The specific implementation of Hilbert transform is as follows:

[0017] For any real signal , its analytical form It can be expressed as: (Formula 3) in and To form a Hilbert transform pair, then It can also be expressed as: (Formula 4) in , , thus, the Hilbert transform provides a unique function that defines the instantaneous amplitude and phase of the original signal The instantaneous frequency for: (Formula 5) Although HT is a convenient, effective and fast tool for nonlinear and non-stationary signals, it cannot be used alone. In order for the output of HT to have accurate physical meaning, the input of HT must be a narrowband signal. , and In order to accurately represent the original signal from different angles That is to say, before a multi-component signal is processed by HT, it must be processed by a narrowband filter.

[0018] like Figure 3-7 As shown, the cochlear model and encoding method proposed in this application are as follows: the external speech signal is pre-processed by noise reduction, windowing, pre-emphasis, etc., and then decomposed by EMD to obtain the intrinsic mode function IMF arranged in order from low to high frequency n , IMF n-1 …IMF2, IMF1, and then HT is performed on these intrinsic mode functions. After HT, the signal is accurately extracted through equations 4 and 5 to extract the instantaneous frequency IF information corresponding to the IMF of each channel; the instantaneous frequency of each channel is subjected to bandwidth limitation BW processing according to the frequency perception range of the cochlea, and the bandwidth and quality factor of the bandpass filter are the same as the IMF function of the corresponding channel; the processed signal is frequency modulated with a sinusoidal signal of the center frequency fc of the corresponding channel, and then the output signal is sent to the low-pass filter LPF for mixing. The slow-varying signal output by the LPF is processed by zero point detection ZC, and a pulse is generated at the zero point position. Finally, each IMF outputs a pulse sequence SS.

[0019] The cochlear pulse neural coding method of the present application is different from the traditional cochlear model and its coding method based on the filter group or transmission line model. It obtains the local narrowband signal IMF with different scale characteristics through layer-by-layer screening according to the characteristics of the speech signal itself, and uses HT on this basis to accurately extract the instantaneous frequency parameters for encoding to generate a pulse sequence. The whole process does not use any prior knowledge, gets rid of the limitations of the traditional filter group / wavelet basis window function and the uncertainty principle as well as the huge computational complexity of the transmission line model, has good adaptability, reduces computational complexity, is easy to implement in hearing aids, electronic cochleas, and machine hearing devices, and can also bring performance improvements to subsequent pulse neural networks.

Claims

1. A cochlear pulse neural coding method, comprising: The original speech signal is pre-processed by noise reduction, windowing, and pre-emphasis to obtain a pre-processed signal; The preprocessed signal is subjected to empirical mode decomposition to obtain the intrinsic mode functions of multiple channels arranged in order from low to high frequency; For each channel's intrinsic mode function, performing Hilbert transform on the channel's intrinsic mode function; Based on the Hilbert transform result, the instantaneous frequency information corresponding to the intrinsic mode function of the channel is accurately extracted; the instantaneous frequency corresponding to the channel is subjected to bandwidth limitation processing according to the frequency perception range of the cochlea to obtain a bandwidth-limited processing signal; the bandwidth and quality factor of the bandpass filter used in the bandwidth limitation processing are the same as those of the intrinsic mode function of the channel; the bandwidth-limited processing signal is frequency modulated to obtain a frequency-shifted bandwidth-limited processing signal; a sinusoidal signal of the center frequency of the intrinsic mode function of the channel is frequency modulated to obtain a frequency-modulated signal to achieve frequency shift; the frequency-modulated signal is output to a low-pass filter to achieve mixing to obtain a mixed slow-varying signal; The mixed slow-varying signal undergoes zero-point detection and generates a pulse at the zero-crossing position. The pulse is the output pulse corresponding to the intrinsic mode function of the channel.

2. The cochlear pulse neural coding method according to claim 1, characterized in that: The frequency modulation of the bandwidth-limited processing signal is performed by frequency modulating a sinusoidal signal of the center frequency of the intrinsic mode function of the channel to achieve frequency shift.

3. The cochlear pulse neural coding method according to claim 1, characterized in that: The output pulses corresponding to the intrinsic mode functions of each channel are sent to the spiking neural network.

Images