Artificial cochlea stimulation method and device and hearing aid equipment

Abstract

The invention discloses an artificial cochlea stimulation method, an artificial cochlea stimulation device and hearing aid equipment. The artificial cochlea stimulation method comprises the steps that a sound signal of an environment is collected, the sound signal is converted into a first digital signal, and the first digital signal comprises multiple frames of audio signals; determining a signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; determining a plurality of formants based on the first digital signal and a preset formant neural network estimation model; based on the signal-to-noise ratio of the audio signal, the formants and a gain factor, frequency band energy corresponding to the formants is determined, and the gain factor is determined based on the signal-to-noise ratio; the frequency band energy is converted into electrode stimulation energy, and auditory nerves of the patient are stimulated based on the electrode stimulation energy. The method and the device have the technical effects of high voice distinguishing capability and high intelligibility in a noisy environment.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of neural stimulation technology, and more particularly to a cochlear implant stimulation method, apparatus, and hearing aid device. Background Art

[0002] A cochlear implant is an electronic device used to compensate for sensorineural hearing loss and is widely used in auditory reconstruction for people with severe or above hearing impairments. Its basic principle is to pick up external sound signals, analyze, process and electrically stimulate them, directly act on the auditory nerve, bypass the damaged inner ear structure, and achieve replacement of auditory function. Cochlear implants in existing technologies can often provide basic speech perception functions in static environments, but because they only rely on the size of the frequency band energy as the basis for electrode stimulation and lack in-depth use of the internal structural characteristics of speech, cochlear implant users still have obvious perception defects in the following aspects. Therefore, the speech discrimination ability of cochlear implant users in actual noise scenarios deserves attention. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a cochlear implant stimulation method, apparatus, and hearing aid device, in order to improve the speech discrimination ability of the cochlear implant in a noisy environment. In a first aspect, a cochlear implant stimulation method is provided, comprising: collecting an environmental sound signal and converting the sound signal into a first digital signal, wherein the first digital signal comprises a plurality of frames of audio signal; determining the signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; determining a plurality of resonance peaks based on the first digital signal and a preset resonance peak neural network estimation model; determining the frequency band energy corresponding to the plurality of resonance peaks based on the signal-to-noise ratio of the audio signal, the plurality of resonance peaks, and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio; converting the frequency band energy into electrode stimulation energy, and stimulating the patient's auditory nerve based on the electrode stimulation energy.

[0004] The above cochlear implant stimulation method significantly enhances the ability to extract speech structural features. Using a formant neural network estimation model to determine multiple formants in the first digital signal, it accurately detects key speech structures, making the stimulation signal more closely aligned with the essential structure of natural speech and enhancing the user's perception of vowel clarity and speech fluency. The introduction of a noise assessment algorithm, coupled with a gain factor, allows for dynamic adjustment of stimulation intensity and frequency band selection based on the environment, ensuring high speech intelligibility even in noisy environments and improving speech resolution.

[0005] Optionally, when collecting the sound signal of the environment, multiple sampling points are included, and the multiple sampling points include: a first sampling point and a second sampling point, and the time sequence of the first sampling point is before the time sequence of the second sampling point; converting the sound signal into a first digital signal also includes: based on the noise signal of the first sampling point, the pre-emphasis coefficient and the noise signal of the second sampling point, pre-emphasis processing is performed on the noise signal of the second sampling point.

[0006] Optionally, determining the signal-to-noise ratio of an audio signal based on a first digital signal and a preset noise estimation algorithm includes: using audio signals of one or more frames in a preset order in the first digital signal as an initial noise signal; determining a first noise energy based on the initial noise signal, and determining the audio energy of the audio signal of each frame; when there is an audio signal of a frame in the first digital signal whose audio energy is less than the preset noise energy, updating the first noise energy of the audio signal of the frame in combination with a smoothing factor to determine a second noise energy; and determining the signal-to-noise ratio of the audio signal based on the second noise energy and the preset noise energy.

[0007] Optionally, the construction of a preset resonance peak neural network estimation model includes: determining the resonance peaks of multiple noise-free audio signals based on clean speech samples and a linear predictive coding algorithm in a Chinese speech data set; adding multiple white noises with different signal-to-noise ratios to the clean speech samples to generate noisy speech samples; using the spectral features of the noisy speech samples as input and the resonance peaks of the noise-free audio signal as output, performing supervised learning training to generate a resonance peak neural network estimation model.

[0008] Optionally, based on the signal-to-noise ratio of the audio signal, multiple resonance peaks and a gain factor, determining the frequency band energies corresponding to the multiple resonance peaks includes: when the signal-to-noise ratio of the audio signal is greater than or equal to a first signal-to-noise ratio threshold, multiplying the frequency band energies corresponding to the multiple resonance peaks by a first gain factor to determine multiple first frequency band energies; when the signal-to-noise ratio of the audio signal is less than the first signal-to-noise ratio threshold and greater than a second signal-to-noise ratio threshold, multiplying the frequency band energies of the frequency bands where the multiple resonance peaks are located by a second gain factor to determine multiple second frequency band energies; when the signal-to-noise ratio of the audio signal is less than the second signal-to-noise ratio threshold, multiplying the frequency band energies of the frequency bands where the multiple resonance peaks are located by a third gain factor to determine multiple third frequency band energies.

[0009] Optionally, the first frequency band energy, the second frequency band energy and the third frequency band energy are respectively converted into electrode stimulation energy.

[0010] Optionally, stimulating the patient's auditory nerve based on electrode stimulation energy includes: generating an electric pulse sequence for stimulating the auditory nerve based on the correspondence between the cochlear electrode and the electrode stimulation energy, and stimulating through the cochlear electrode.

[0011] Optionally, the multiple resonance peaks include: a first resonance peak, a second resonance peak and a third resonance peak; wherein, the physical feature of pronunciation corresponding to the first resonance peak is the tongue height, and the sound feature is the vowel opening and closing degree; the physical feature of pronunciation corresponding to the second resonance peak is the front and back position of the tongue, and the sound feature is the front and back nature of the vowel; the physical feature of pronunciation corresponding to the third resonance peak is the tongue tip position and lip shape, and the sound feature is timbre and emotion.

[0012] In a second aspect, a cochlear implant stimulation device is provided, comprising: an acquisition unit for acquiring sound signals from the environment and converting the sound signals into a first digital signal, wherein the first digital signal comprises a plurality of frames of audio signals; a signal-to-noise ratio determination unit for determining the signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; a resonance peak determination unit for determining a plurality of resonance peaks based on the first digital signal and a preset resonance peak neural network estimation model; a frequency band energy determination unit for determining frequency band energies corresponding to a plurality of resonance peaks based on the signal-to-noise ratio of the audio signal, a plurality of resonance peaks and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio; and a conversion unit for converting the frequency band energy into electrode stimulation energy, and stimulating the patient's auditory nerve based on the electrode stimulation energy.

[0013] In a third aspect, a cochlear implant is provided, comprising: a plurality of electrodes; and a processor for implementing the cochlear implant stimulation method of the first aspect and stimulating the patient's auditory nerve through the electrodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The following is a brief introduction to the drawings used in describing the embodiments of the present disclosure:

[0015] Figure 1 A schematic flow chart of a cochlear implant stimulation method provided in some embodiments of the present application is shown;

[0016] Figure 2 A schematic flow chart of a method for determining a signal-to-noise ratio of an audio signal provided in some embodiments of the present application is shown;

[0017] Figure 3 A schematic flow chart showing a method for constructing a preset resonance peak neural network estimation model provided in some embodiments of the present application is shown;

[0018] Figure 4 A schematic structural diagram of a cochlear implant stimulation device provided in some embodiments of the present application is shown. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, examples of implementation of the present disclosure will be described below with reference to the accompanying drawings. The drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other drawings and other implementations can be obtained based on these drawings without inventive work. Adjustments and improvements made without departing from the concept of the present disclosure are all within the scope of protection of the present disclosure.

[0020] To simplify the drawings, the figures schematically illustrate only the portions relevant to the embodiments and do not represent the actual structure of the products. Furthermore, to simplify the drawings and facilitate understanding, in some figures, only a portion of components with the same structure or function are schematically depicted; in practice, more or fewer components with the same structure or function may exist.

[0021] In the present disclosure, unless otherwise expressly specified and limited, ordinal numbers such as "first" and "second" are only used to distinguish and describe associated objects, and cannot be understood as indicating or implying the relative importance or order between associated objects; in addition, they do not represent the number of associated objects. "Multiple" includes two or more, and other quantifiers are similar. " / " is used to describe the relationship between associated objects, which indicates the "or" relationship between associated objects. "And / or" is used to describe the relationship between associated objects, which includes any combination relationship between associated objects, for example, "a and / or b" includes: "alone a", "alone b", or "a and b". "One or more" or "at least one" in multiple objects refers to any object or any combination of multiple objects, for example, "one or more of a1, a2, a3" or "at least one of a1, a2, a3" includes: "alone a1", "alone a2", "alone a3", "a1 and a2", "a1 and a3", "a2 and a3" or "a1, a2 and a3".

[0022] A cochlear implant is an electronic device used to compensate for sensorineural hearing loss and is widely used in auditory reconstruction for people with severe or above hearing impairments. Its basic principle is to pick up external sound signals, analyze and process them, and output electrical stimulation, directly acting on the auditory nerve, bypassing the damaged inner ear structure, and achieving auditory function replacement. Existing mainstream cochlear implant systems mostly use stimulation strategies based on short-time spectrum analysis, such as Continuous Interleaved Sampling (CIS), Fine Structure Processing (FS4), and Advanced Combination Encoder (ACE). They generally select stimulation channels and control current amplitudes based on the frequency band energy distribution of the input speech signal, thereby simulating sound information. Although these strategies can provide basic speech perception in a quiet environment, their frequency resolution and structure recognition capabilities are limited, making it difficult to accurately restore the complex acoustic features in speech. In addition, existing methods generally ignore the accurate extraction and utilization of dynamic structural features in speech (such as formant frequency and change trend), which affects the comprehension ability and user experience of cochlear implant users in actual auditory scenarios. For example, existing cochlear implant stimulation strategies mainly rely on frequency band energy as a judgment criterion, which is a relatively rough acoustic feature extraction method. It lacks in-depth analysis and utilization of the internal structural features of the speech signal, cannot distinguish subtle speech differences, and the spectrum energy cannot effectively describe the formant structure of vowels, resulting in similar vowels and consonants being difficult to distinguish, and speech clarity is reduced. Moreover, the energy feature is easily affected by environmental noise, the stimulation strategy is unstable in extracting the target area of ​​speech, and the speech intelligibility is significantly reduced in noisy scenes. Therefore, the present application provides a cochlear implant stimulation method, device, and hearing aid device, which can deeply detect and extract the structural features of speech, especially the formant frequency and its dynamic evolution, on the basis of traditional frequency band division, and effectively encode this information into the stimulation pattern to enhance the speech perception ability and auditory discrimination effect of cochlear implant users in natural and noisy environments.

[0023] The following is a description with reference to the accompanying drawings:

[0024] Figure 1 A schematic flow chart of a cochlear implant stimulation method provided in some embodiments of the present application is shown. The cochlear implant stimulation method includes at least the following steps:

[0025] S110: Collecting environmental sound signals and converting the sound signals into first digital signals, wherein the first digital signals include multiple frames of audio signals;

[0026] S120: Determine a signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm;

[0027] S130: Determine a plurality of resonance peaks based on the first digital signal and a preset resonance peak neural network estimation model;

[0028] S140: Determining frequency band energies corresponding to the multiple formants based on a signal-to-noise ratio of the audio signal, the multiple formants, and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio;

[0029] S150: Convert the frequency band energy into electrode stimulation energy, and stimulate the patient's auditory nerve based on the electrode stimulation energy.

[0030] The above cochlear implant stimulation method can be applied to improve the speech perception effect of cochlear implant users, especially in an environment with background noise, and can enhance vowel recognition ability and speech clarity. When collecting environmental sound signals, the microphone in the cochlear implant can be used to collect the sound signal of natural speech in the environment. The sound signal may include white noise, human voice or other sounds in the environment. The human voice may include normal speaking speed, different volume, and voice of different speakers. The sound signal is an analog signal. For ease of processing, it can be converted into a first digital signal by the analog-to-digital conversion module in the cochlear implant. The first digital signal may include multiple consecutive frames of audio signals. The collected sound signal is often interfered by background noise. The present application can dynamically evaluate the local signal-to-noise ratio (SNR) of each frame of audio signal by a preset noise estimation algorithm. The noise corresponding to the audio signal is determined based on a frame of audio signal, and the signal-to-noise ratio is determined based on the audio signal and the corresponding noise, as shown in the following formula 1:

[0031]

[0032] Wherein, E1 is the frequency band energy of the frequency band where the current audio signal is located, and EZ is the frequency band energy of the noise corresponding to the current audio signal determined by the noise estimation algorithm. The noise estimation algorithm can be based on a dynamic threshold algorithm of short-time energy, which calculates the short-time energy of each frame of speech signal and combines it with a set energy threshold or coefficient. When the frame energy is lower than the threshold, it is regarded as a noise frame and used to update the noise energy estimation value, and then the signal-to-noise ratio is calculated based on the ratio between the speech signal frame energy and the current noise estimation energy. Alternatively, a noise update algorithm based on minimum mean square error estimation can be used to dynamically estimate the noise power spectrum density by minimizing the signal estimation error using the statistical characteristics of the speech frame and the noise frame, and then determine the signal-to-noise ratio. In step S130 of the present application, multiple resonance peaks can be determined using the first digital signal and a preset resonance peak neural network estimation model. Resonance peaks are important frequency components that reflect the timbre and vowel characteristics of speech. Accurately extracting resonance peak information plays a key role in improving speech perception. The traditional formant extraction method is implemented by the cepstrum method or linear prediction analysis method, which has poor robustness and is more sensitive to noise environments. It is difficult to determine accurate formant estimation in a noisy environment. The formant neural network estimation model of the present application can analyze each frame of audio in the input first digital signal and extract formants representing the structural characteristics of the speech, such as extracting 2, 3, 4... and other multiple formants. The neural network model can adopt a convolutional neural network (CNN), a lightweight deep model (such as MobileNetV3) or other network architectures with frequency extraction capabilities, which can identify and output the frequency position of the formant in a noisy environment, thereby being used for subsequent frequency band gain adjustment and stimulation electrode selection. By introducing a neural network model to extract speech formants, unlike the traditional processing strategy that only relies on frequency band energy, this method can learn key features from the overall spectral structure and improve the accuracy and robustness of formant estimation. Especially in a high noise background, important frequency points expressing vowel categories and timbre in speech can still be stably detected, thereby improving the speech clarity and resolution of cochlear implant users and significantly improving the quality of speech perception. The energy of the frequency bands corresponding to the formants can be dynamically adjusted using the audio signal's signal-to-noise ratio (SNR), the determined multiple formants, and a gain factor. The gain factor is used to weight the energy of the frequency bands corresponding to the formants to enhance the structural features of the speech signal. For example, the gain factor can be positively correlated with the audio signal's SNR, meaning that the gain factor increases as the SNR increases.For example, when the signal-to-noise ratio of the audio signal is high (e.g., greater than 10 dB), the position of the formant can be detected more accurately. At this time, a larger gain factor (e.g., Q=1.5) can be used to significantly enhance the energy of the frequency band corresponding to the formant. When the signal-to-noise ratio is at a medium level (e.g., 5-10 dB), a medium-level gain factor (e.g., Q=1.2) is used. Under low signal-to-noise ratio conditions (e.g., less than 5 dB), to avoid frequency band drift caused by erroneous enhancement of noise, the gain factor can be appropriately reduced (e.g., Q=1) or the enhancement operation can be suspended. The gain factor can be determined by establishing a mapping relationship with the signal-to-noise ratio through a lookup table, a linear mapping function, or a simple piecewise function to achieve adaptive adjustment. After determining the frequency band energy, the obtained multiple frequency band energies are mapped to the multi-channel electrode array in the cochlear implant. Each frequency band corresponds to one or more electrode channels. The system selects the stimulation position and electrical stimulation amplitude based on the energy distribution, thereby generating an electrical pulse sequence, which ultimately stimulates the auditory nerve through the cochlear electrodes, allowing the patient to produce clear sound perception. The above cochlear implant stimulation method significantly enhances the ability to extract speech structural features. Using a formant neural network estimation model to determine multiple formants in the first digital signal, it accurately detects key speech structures, making the stimulation signal more closely aligned with the essential structure of natural speech and enhancing the user's perception of vowel clarity and speech fluency. The introduction of a noise assessment algorithm, coupled with a gain factor, allows for dynamic adjustment of stimulation intensity and frequency band selection based on the environment, ensuring high speech intelligibility even in noisy environments and improving speech resolution.

[0033] In some embodiments of the present application, collecting an environmental sound signal includes multiple sampling points, the multiple sampling points including: a first sampling point and a second sampling point, the time sequence of the first sampling point being before the time sequence of the second sampling point; step S110 converts the sound signal into a first digital signal, and also includes: performing pre-emphasis processing on the noise signal of the second sampling point based on the noise signal of the first sampling point, the pre-emphasis coefficient, and the noise signal of the second sampling point.

[0034] When collecting environmental sound signals, the present application may, for example, use a sampling frequency of 16 kHz and a 16-bit width. Furthermore, to facilitate signal processing, the first digital signal may be pre-processed (pre-emphasized) to lay the foundation for subsequent processing, thereby ensuring that the system can obtain high-quality input signals that reflect the environmental characteristics and the interlocutor's voice as completely as possible. Formula 2 shows a pre-processing calculation method:

[0035] y(n)=x(n)-a*y(n-1) Formula 2

[0036] a is the pre-emphasis coefficient, y(n) is the audio signal after preprocessing at the second sampling point, x(n) is the audio signal before preprocessing at the second sampling point, and y(n-1) is the audio signal preprocessed at the first sampling point.

[0037] Figure 2 A flow chart of a method for determining the signal-to-noise ratio of an audio signal provided in some embodiments of the present application is shown. The method includes:

[0038] S210: Using audio signals of one or more frames in a preset sequence as initial noise signals in the first digital signal;

[0039] S220: Determine a first noise energy based on the initial noise signal, and determine the audio energy of the audio signal of each frame;

[0040] S230: When the audio energy of an audio signal of a frame in the first digital signal is less than a preset noise energy, the first noise energy is updated based on the audio signal of the frame and a smoothing factor to determine a second noise energy;

[0041] S240: Determine a signal-to-noise ratio of the audio signal based on the second noise energy and the preset noise energy.

[0042] Since cochlear implants need to face normal communication scenarios with rapid responses when used and have a high degree of real-time processing capabilities, the short-time energy of the audio signal frame can be used to determine the signal-to-noise ratio in this application. For example, in real life, the energy of noise may change over time, but compared to normal human voice signals (such as the sound signal emitted by a person speaking to the user), the noise energy is generally lower than the human voice signal. Therefore, when performing noise estimation, the audio signal of the first frame of voice input or a preset number of frames (the first few frames collected) can be defaulted to the initial noise signal, and the initial noise signal energy can be expressed based on Formula 3:

[0043]

[0044] Among them, E noise represents the energy of the initial noise signal, and x(n) represents the energy of a frame of audio signal as the initial noise signal.

[0045] Based on the initial noise signal energy of Formula 3, it is updated by the audio signals of other frames. The updated noise signal energy can be expressed by Formula 4:

[0046]

[0047] Among them, E signal is the first noise energy, E noise_newis the updated second noise energy, μ is the smoothing factor. The smoothing factor μ can be 0.98, which can be determined by those skilled in the art through experiments. signal When the noise energy is less than the preset noise energy, the frame of audio signal can be determined to be a noise frame. The preset noise energy can be determined based on the noise energy of non-noise human voices when normal people communicate face to face. For example, the preset noise energy can take a fixed value of 0.45, or a preset noise energy range can be determined. When the first noise energy falls within the range, the frame of audio signal is determined to be a noise frame. At this time, the noise energy is updated, and the signal-to-noise ratio of the current frame is calculated using the updated noise energy, as shown in Formula 5:

[0048]

[0049] This application can estimate the background noise energy in audio signals, thereby effectively improving the accuracy and stability of signal-to-noise ratio assessment. By comparing the preset noise energy with the current first noise energy and dynamically updating the noise estimate based on a smoothing factor, it can effectively adapt to changes in background noise and avoid overall distortion caused by sudden sounds or initial estimation errors. It can be applied to the front-end perception and processing of speech fragments by cochlear implant stimulation systems in noisy environments, thereby accurately estimating the signal-to-noise ratio. It improves the accuracy of the signal-to-noise ratio used to guide formant enhancement, gain factor adjustment, electrode selection, and other links, enhances the reliability of cochlear implant stimulation methods, and improves the speech clarity and intelligibility of cochlear implant users.

[0050] In some embodiments of the present application, Figure 3 A flow chart of a method for constructing a preset resonance peak neural network estimation model provided in some embodiments of the present application is shown. The construction method includes:

[0051] S310: determining formants of multiple noise-free audio signals based on clean speech samples in the Chinese speech dataset and a linear predictive coding algorithm;

[0052] S320: adding a plurality of white noises with different signal-to-noise ratios to the clean speech sample to generate a noisy speech sample;

[0053] S330: Using the spectral features of the noisy speech sample as input and the formant of the noise-free audio signal as output, supervised learning training is performed to generate a formant neural network estimation model.

[0054] The preset resonance peak neural network estimation model of the present application can be a lightweight neural network model obtained through training, for example, a MobileNetV3 architecture can be used. During training, a Chinese speech data set can be first selected as a training sample, and the speech data therein is preprocessed, including removing silent segments, frame segmentation and window function processing, to obtain a set of clean speech samples, and then the linear predictive coding (LPC) algorithm is used for each frame of speech to calculate the true resonance peak corresponding to the clean speech sample as a training label. Then, a plurality of different Gaussian white noises are added to the clean speech samples, for example, they can be set according to the same noise gradient for easy recognition, or a random noise gradient can be added to close to the real scene, thereby constructing a noisy speech sample. Based on the constructed regression neural network based on MobileNetV3, the spectral features of the noisy speech sample are input into the model to output the resonance peak of the noise-free audio signal (the aforementioned clean speech sample). A large number of labeled samples are used for training, for example, 16,000 sets of labeled samples are used for training, and 4,000 sets of samples are used as a validation set, and the L1 loss function is used to measure the error between the output and the true resonance peak frequency. After training is completed, the structural parameters and weight coefficients of the neural network model can be derived and applied in an integrated circuit that specifically implements the artificial cochlear stimulation method of the present application, and run as a resonance peak estimation module to realize resonance peak detection of environmental sound signals.

[0055] In some embodiments of the present application, step S140 determines frequency band energies corresponding to multiple resonance peaks based on the signal-to-noise ratio of the audio signal, multiple resonance peaks, and the gain factor, including: when the signal-to-noise ratio of the audio signal is greater than or equal to a first signal-to-noise ratio threshold, multiplying the frequency band energies corresponding to the multiple resonance peaks by a first gain factor to determine multiple first frequency band energies; when the signal-to-noise ratio of the audio signal is less than the first signal-to-noise ratio threshold and greater than a second signal-to-noise ratio threshold, multiplying the frequency band energies of the frequency bands where the multiple resonance peaks are located by a second gain factor to determine multiple second frequency band energies; when the signal-to-noise ratio of the audio signal is less than the second signal-to-noise ratio threshold, multiplying the frequency band energies of the frequency bands where the multiple resonance peaks are located by a third gain factor to determine multiple third frequency band energies.

[0056] In an embodiment of the present application, the first digital signal may be subjected to a fast Fourier transform to determine an expression for frequency band energy, as shown in Formula 6:

[0057]

[0058] N is the number of fast Fourier transform points, which indicates the number of sample points used when performing discrete Fourier transform on the signal, that is, the window size of the transform. In the embodiment, N=256, which means that the length of the signal extracted per frame is 256 sampling points. K is the length of the transformed spectrum, which can be the same as N, that is, it is also 256 frequency points. Taking the resonance peaks as three examples, the energy of the frequency band where the first resonance peak F1 is located, which is determined by the preset resonance peak neural network estimation model, is set to E_F1, the energy of the frequency band where the second resonance peak F2 is located is set to E_F2, and the energy of the frequency band where the third resonance peak F3 is located is set to E_F3. The signal-to-noise ratio of the audio signal is dynamically adjusted, as shown in the following formula 7:

[0059]

[0060] The gain factor can be determined to have different values ​​based on different signal-to-noise ratios. For example, the first gain factor Q1 is 2, the second gain factor Q2 is 1.5, and the third gain factor Q3 is 1, thereby determining the frequency band energy of the frequency band where the resonance peak is located under different signal-to-noise ratio conditions. For example, when the signal-to-noise ratio is greater than or equal to 10, the first gain factor Q1 is 2, and the frequency band energies corresponding to the multiple resonance peaks are weighted and calculated to determine multiple first frequency band energies E_F11, E_F12, and E_F12. After adjusting the frequency band energies of the multiple resonance peaks, the ACE strategy in the field of traditional cochlear implants can be used to convert the frequency band energy into the corresponding electrode stimulation energy, and the first frequency band energy, the second frequency band energy, and the third frequency band energy are respectively converted into electrode stimulation energy, and the patient's auditory nerve is stimulated based on the electrode stimulation energy. For example, based on the correspondence between the cochlear electrodes and the electrode stimulation energy, an electrical pulse sequence for stimulating the auditory nerve can be generated, and stimulation can be performed through the cochlear electrodes.

[0061] In some embodiments of the present application, multiple resonance peaks may include: a first resonance peak, a second resonance peak and a third resonance peak; wherein the physical feature of pronunciation corresponding to the first resonance peak is the tongue height, and the sound feature is the vowel opening and closing degree; the physical feature of pronunciation corresponding to the second resonance peak is the front and back position of the tongue, and the sound feature is the front and back nature of the vowel; the physical feature of pronunciation corresponding to the third resonance peak is the tongue tip position and lip shape, and the sound feature is timbre and emotion.

[0062] Figure 4A schematic diagram of the structure of a cochlear implant stimulation device provided in some embodiments of the present application is shown. The device comprises: an acquisition unit 410 for acquiring ambient sound signals and converting the sound signals into a first digital signal, wherein the first digital signal comprises multiple frames of audio signals; a signal-to-noise ratio determination unit 420 for determining the signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; a formant determination unit 430 for determining multiple formants based on the first digital signal and a preset formant neural network estimation model; a frequency band energy determination unit 440 for determining frequency band energies corresponding to the multiple formants based on the signal-to-noise ratio of the audio signal, the multiple formants, and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio; and a conversion unit 450 for converting the frequency band energy into electrode stimulation energy and stimulating the patient's auditory nerve based on the electrode stimulation energy.

[0063] The division of the above units is only a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or they can be physically separated. In addition, the above units can be implemented in the form of a processor calling software; for example, a cochlear implant stimulation device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU), and the memory is a memory within the device or a memory outside the device. Alternatively, the above units can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in some embodiments, the hardware circuit is an application specific integrated circuit (ASIC), and the functions of some or all units above can be realized by designing the logical relationships between the components within the circuit; for example, in another implementation, the hardware circuit can be implemented by a programmable logic device (PLD), which can include a large number of logic gate circuits. The logical relationships between the logic gate circuits are configured through a configuration file to realize the functions of some or all units above. All units of the above apparatus may be implemented entirely in the form of a processor calling a program, or entirely in the form of a hardware circuit, or partially in the form of a processor calling a program and the rest in the form of a hardware circuit.

[0064] Based on the same technical concept, the present application also provides a cochlear implant, comprising: a plurality of electrodes; a processor, for implementing the cochlear implant stimulation method provided in the above embodiments, and stimulating the patient's auditory nerve through the electrodes.

[0065] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. In addition, the above embodiments can be freely combined as needed.

Claims

1. A cochlear implant stimulation method, characterized in that: include: Collecting a sound signal from an environment and converting the sound signal into a first digital signal, wherein the first digital signal includes a plurality of frames of audio signals; determining a signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; Determining a plurality of resonance peaks based on the first digital signal and a preset resonance peak neural network estimation model; determining frequency band energies corresponding to the plurality of formants based on a signal-to-noise ratio of the audio signal, the plurality of formants, and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio; The frequency band energy is converted into electrode stimulation energy, and the patient's auditory nerve is stimulated based on the electrode stimulation energy.

2. The cochlear implant stimulation method according to claim 1, wherein: The sound signal of the environment is collected at a plurality of sampling points, wherein the plurality of sampling points include: a first sampling point and a second sampling point, wherein the time sequence of the first sampling point is before the time sequence of the second sampling point; The converting the sound signal into a first digital signal further includes: Based on the noise signal of the first sampling point, the pre-emphasis coefficient, and the noise signal of the second sampling point, pre-emphasis processing is performed on the noise signal of the second sampling point.

3. The cochlear implant stimulation method according to claim 2, wherein: The determining the signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm includes: In the first digital signal, the audio signal of one or more frames in a preset order is used as an initial noise signal; determining a first noise energy based on the initial noise signal, and determining an audio energy of the audio signal of each frame; When the audio energy of an audio signal of a frame in the first digital signal is less than a preset noise energy, the first noise energy is updated by combining the audio signal of the frame with a smoothing factor to determine a second noise energy; A signal-to-noise ratio of the audio signal is determined based on the second noise energy and the preset noise energy.

4. The cochlear implant stimulation method according to claim 3, wherein: The construction of the preset resonance peak neural network estimation model includes: Determine the formants of multiple noise-free audio signals based on clean speech samples from a Chinese speech dataset and a linear predictive coding algorithm. Adding a plurality of white noises with different signal-to-noise ratios to the clean speech sample to generate a noisy speech sample; The spectrum characteristics of the noisy speech sample are used as input and the formant of the noise-free audio signal is used as output, and supervised learning training is performed to generate the formant neural network estimation model.

5. The cochlear implant stimulation method according to claim 4, wherein: The determining, based on the signal-to-noise ratio of the audio signal, the plurality of formants and the gain factor, frequency band energies corresponding to the plurality of formants includes: When the signal-to-noise ratio of the audio signal is greater than or equal to a first signal-to-noise ratio threshold, multiplying frequency band energies corresponding to the plurality of resonance peaks by a first gain factor to determine a plurality of first frequency band energies; When the signal-to-noise ratio of the audio signal is less than the first signal-to-noise ratio threshold and greater than a second signal-to-noise ratio threshold, multiplying the frequency band energies of the frequency bands where the plurality of resonance peaks are located by a second gain factor respectively to determine a plurality of second frequency band energies; When the signal-to-noise ratio of the audio signal is less than the second signal-to-noise ratio threshold, the frequency band energies of the frequency bands where the multiple resonance peaks are located are multiplied by a third gain factor respectively to determine a plurality of third frequency band energies.

6. The cochlear implant stimulation method according to claim 5, characterized in that: The first frequency band energy, the second frequency band energy and the third frequency band energy are respectively converted into the electrode stimulation energy.

7. The cochlear implant stimulation method according to claim 6, wherein: The stimulating the patient's auditory nerve based on the electrode stimulation energy includes: Based on the correspondence between the cochlear electrodes and the electrode stimulation energy, an electrical pulse sequence for stimulating the auditory nerve is generated, and the stimulation is performed through the cochlear electrodes.

8. The cochlear implant stimulation method according to any one of claims 1 to 7, characterized in that: The multiple resonance peaks include: a first resonance peak, a second resonance peak and a third resonance peak; The physical characteristic of pronunciation corresponding to the first formant is tongue height, and the sound characteristic is vowel opening and closing degree; The physical characteristic of pronunciation corresponding to the second formant is the front-back position of the tongue, and the acoustic characteristic is the front-back nature of the vowel; The physical characteristics of pronunciation corresponding to the third resonance peak are the tongue tip position and lip shape, and the sound characteristics are the timbre and emotion.

9. A cochlear implant stimulation device, characterized in that: include: A collection unit, configured to collect a sound signal from an environment and convert the sound signal into a first digital signal, wherein the first digital signal includes a plurality of frames of audio signals; a signal-to-noise ratio determining unit, configured to determine a signal-to-noise ratio of the audio signal based on the first digital signal and a preset noise estimation algorithm; a formant determination unit, configured to determine a plurality of formant peaks based on the first digital signal and a preset formant neural network estimation model; a frequency band energy determining unit, configured to determine frequency band energies corresponding to the plurality of formants based on a signal-to-noise ratio of the audio signal, the plurality of formants, and a gain factor, wherein the gain factor is determined based on the signal-to-noise ratio; A conversion unit is used to convert the frequency band energy into electrode stimulation energy, and stimulate the patient's auditory nerve based on the electrode stimulation energy.

10. A hearing aid device, characterized in that: include: Multiple electrodes; A processor is used to implement the cochlear implant stimulation method according to any one of claims 1 to 8, and stimulate the patient's auditory nerve through the electrode.

Images