Sound signal reconstruction method, system and equipment based on artificial cochlea

Abstract

The invention provides a sound signal reconstruction method, system and device based on an artificial cochlea, and the method comprises the steps: obtaining an external sound signal, carrying out the spatial filtering of the sound signal, and obtaining a spatial filtering signal; performing frequency domain decomposition on the spatial domain filtering signal to determine an initial energy spectrum; converting the energy spectrum into a frequency spectrum; calculating a signal voice gain according to the estimation result of the noise spectrum of the sound signal; amplitude modulation is carried out on the signal voice gain in combination with the electric auditory dynamic range of an implanter; and applying amplitude-modulated signal voice gain to the frequency spectrum, and outputting a reconstructed de-noised voice energy spectrum. According to the invention, the noise in the signal is eliminated, and the signal with an overlarge amplitude can be dynamically compressed in combination with the electric hearing dynamic range of an implanter, so that the effective bit depth of the output voice signal is improved, and the artificial hearing speech intelligibility is further enhanced.

Description

Technical Field

[0001] This application belongs to the technical field of sound signal processing, and relates to a signal reconstruction method, in particular to a method, system and device for reconstructing sound signals based on cochlear implants. Background Art

[0002] Cochlear implant technology is currently the only effective method and device recognized in the world for enabling patients with bilateral severe or profound sensorineural deafness to regain their hearing. For most mainstream cochlear implant systems, external sounds are first collected by a microphone and converted into electrical signals. After processing, they are transmitted to the body through a transmitting coil behind the ear. After the receiving coil of the implant senses the signal, it is decoded by a decoding chip, causing the stimulating electrode of the implant to generate an electric current, thereby stimulating the auditory nerve to produce hearing.

[0003] Generally speaking, the current amplitude and depth that the electrodes of a cochlear implant can output are limited. For common implants, usually 8-bit depth of stimulating current (Current Level, CL) can be provided for reconstructing the input acoustic signal (i.e., 0 - 255CL, and the stimulating current amplitude generally increases non-linearly with the increase of the CL value, and the current amplitude corresponding to the CL value is related to the specific product model). However, not all intensities of stimulating current are effective or acceptable to the implant recipient: when the CL value is too small, the stimulating current cannot trigger sufficient auditory nerve responses, that is, no artificial hearing can be produced; when the CL value is too large, the stimulating current is likely to cause adverse reactions such as pain to the implant recipient.

[0004] The electroacoustic dynamic range (Electrode Dynamic Range, EDR) is the range of CL values that can cause sufficient responses in the implant recipient's auditory nerve without making them feel uncomfortable. The lower limit of this range is called the electrode threshold of the implant recipient, abbreviated as the T value. The electrostimulation output below this threshold is inaudible to the implant recipient; correspondingly, its upper limit is called the comfort threshold, abbreviated as the C value. The current stimulation above this threshold is likely to cause discomfort to the implant recipient. Generally speaking, the common EDR is 130CL - 250CL, that is, the actual current amplitude depth available for signal reconstruction may be less than 7 bits or even 6 bits, which poses relatively high requirements for how the cochlear implant system processes the input signal and improves the intelligibility of the output speech.

[0005] In addition, although the deep neural network algorithm has achieved certain breakthroughs and developments in recent years in fields such as speech noise reduction, a relatively large model is usually required to achieve a significant noise control effect. Most common cochlear implant sound processor devices are implemented based on low-power digital signal processors (DSPs). Considering that many implant recipients are sensitive to device power consumption, the method of deploying large neural network models on such edge devices has not yet become mainstream. Summary of the Invention

[0006] The present application provides a method, system, and device for reconstructing a sound signal based on a cochlear implant, which is used to solve the problem of how to reconstruct the sound signal of a cochlear implant on a low-power processing platform.

[0007] In a first aspect, the present application provides a method for reconstructing a sound signal based on a cochlear implant, the method comprising: obtaining an external sound signal, performing spatial domain filtering on the sound signal to obtain a spatially filtered signal; performing frequency domain decomposition on the spatially filtered signal to determine an initial energy spectrum; transforming the energy spectrum into a frequency spectrum; calculating a signal speech gain according to an estimation result of the noise spectrum of the sound signal; performing amplitude modulation on the signal speech gain in combination with the electro-auditory dynamic range of the implant recipient; applying the amplitude-modulated signal speech gain to the frequency spectrum, and outputting a reconstructed denoised speech energy spectrum.

[0008] In an implementation manner of the first aspect, the step of performing spatial domain filtering on the sound signal to obtain a spatially filtered signal comprises: performing spatial domain filtering on the sound signal based on a multi-channel beamforming method to obtain a spatially filtered signal.

[0009] In an implementation manner of the first aspect, the estimation process of the noise spectrum of the sound signal comprises: determining an energy feature of a current signal frame of the sound signal; judging whether the current signal frame is a noise frame according to a dynamic noise threshold; in response to the current signal frame being a noise frame, updating the estimation of the noise spectrum according to the energy spectrum of the current signal frame.

[0010] In an implementation manner of the first aspect, the step of judging whether the current signal frame is a noise frame according to a dynamic noise threshold comprises: judging whether the current signal frame is a noise frame based on a currently dynamically set noise threshold and the energy feature; in response to the energy feature being lower than the noise threshold, determining that the current signal frame is a noise frame; in response to the energy feature being higher than or equal to the noise threshold, determining that the current signal frame is not a noise frame.

[0011] In one implementation of the first aspect, after the step of determining whether the current signal frame is a noise frame according to the dynamic noise threshold, the method further includes: if the energy feature is lower than the noise threshold, using the value of the energy feature as the new noise threshold; if the energy feature is higher than the noise threshold, increasing the noise threshold according to a preset rule.

[0012] In one implementation of the first aspect, the step of amplitude modulating the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient includes: in response to the signal speech gain being greater than a preset gain threshold, multiplying the signal speech gain by a specific compression coefficient less than 1 to limit the signal speech gain, where the preset gain threshold and the specific compression coefficient are both related to the electroacoustic dynamic range.

[0013] In one implementation of the first aspect, the step of outputting the reconstructed denoised speech energy spectrum by applying the amplitude-modulated signal speech gain to the spectrum includes: transforming the amplitude-modulated signal speech gain into a linear value, multiplying the linear gain of each subband by the corresponding subband amplitude of the spectrum in sequence, and the output result is the denoised speech energy spectrum.

[0014] In a second aspect, the present application provides a reconstruction system for a sound signal based on a cochlear implant. The system includes: a beamforming module configured to obtain an external sound signal, perform spatial domain filtering on the sound signal based on a multi-channel beamforming method to obtain a spatially filtered signal; a frequency domain decomposition module configured to perform frequency domain decomposition on the spatially filtered signal to determine an initial energy spectrum; transform the energy spectrum into a frequency spectrum; a gain calculation module configured to calculate a signal speech gain according to an estimation result of the noise spectrum of the sound signal; amplitude modulate the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient; a gain application module configured to apply the amplitude-modulated signal speech gain to the spectrum and output the reconstructed denoised speech energy spectrum.

[0015] In one implementation of the second aspect, the system further includes: a frame energy calculation module configured to determine the energy feature of the current signal frame of the sound signal; determine whether the current signal frame is a noise frame according to a dynamic noise threshold; a noise spectrum estimation module configured to, in response to the current signal frame being a noise frame, update the estimation of the noise spectrum according to the energy spectrum of the current signal frame.

[0016] In a third aspect, the present application provides an electronic device. The electronic device includes: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the electronic device executes the method described above.

[0017] As described above, the method, system, and device for reconstructing sound signals based on a cochlear implant according to the present application have the following

[0018] Advantageous effects:

[0019] The present application provides an adaptive speech noise reduction and compression method based on directional beamforming and spectral attenuation, which can eliminate the noise components in the signal and achieve the purpose of improving the signal-to-noise ratio; through multi-channel gain control based on the electroacoustic dynamic range of the implant recipient, the signal with too large an amplitude is dynamically compressed to improve the effective bit depth of the output speech signal and further enhance the intelligibility of the artificial auditory sense. The present application can perform adaptive noise reduction and compression on the input signal according to the EDR data of the implant recipient by channel, which helps to improve the activity of inner ear hair cells in different regions of the cochlea of the implant recipient. Especially in the case of a narrow EDR range, it can improve the amplitude resolution of the electroacoustic reconstruction signal as much as possible and achieve a better intelligibility improvement effect, and is applicable to the cochlear implant sound processor device (such as DSP) of a low-power processing platform. Description of the Drawings

[0020] Figure 1 It shows a schematic diagram of the application scenario of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0021] Figure 2 It shows a principle flowchart of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0022] Figure 3 It shows a spatial domain filtering flowchart of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0023] Figure 4 It shows a noise estimation flowchart of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0024] Figure 5 It shows a frame energy calculation flowchart of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0025] Figure 6 It shows a schematic diagram of the relationship between the energy characteristics, noise threshold, and noise frames of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0026] Figure 7 It shows a frequency domain decomposition flowchart of the method for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0027] Figure 8It shows a flowchart of gain calculation for the method of reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0028] Figure 9 It shows a flowchart of gain application for the method of reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0029] Figure 10 It shows a comparison schematic diagram between the spectrum of a noisy signal and the spectrum of a denoised speech signal for the method of reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0030] Figure 11 It shows a comparison schematic diagram between the waveform of a noisy signal and the waveform of a denoised speech signal for the method of reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0031] Figure 12 It shows a schematic diagram of the structure principle of the system for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0032] Figure 13 It shows a flowchart of the system for reconstructing sound signals based on a cochlear implant according to an embodiment of the present application.

[0033] Figure 14 It shows a schematic diagram of the structural connection of the electronic device according to an embodiment of the present application.

[0034] Description of component numbers

[0035] 1 System for reconstructing sound signals based on a cochlear implant

[0036] 11 Beamforming module

[0037] 12 Frequency domain decomposition module

[0038] 13 Gain calculation module

[0039] 14 Gain application module

[0040] 15 Frame energy calculation module

[0041] 16 Noise spectrum estimation module

[0042] 2 Electronic device

[0043] 21 Processor

[0044] 22 Memory

[0045] Steps S11 to S19 Detailed implementation manners

[0046] The following describes the implementation manners of the present application through specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0047] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0048] The following embodiments of the present application provide a method, system, and device for reconstructing sound signals based on a cochlear implant, including but not limited to being applied to a processing system composed of a microphone and an electronic device. The following will describe this application scenario as an example.

[0049] As Figure 1 shown, this embodiment provides a processing system composed of a microphone and an electronic device. External sound is first collected by the microphone and converted into an electrical signal. The external sound electrical signal is then handed over to the electronic device to execute the method for reconstructing sound signals based on a cochlear implant described in the present application, and noise reduction and compression processing are performed to obtain the signal energy spectrum after noise reduction and compression, that is, the denoised speech energy spectrum.

[0050] Since the cochlear implant system has very few bit depths that can be used to represent the pulse amplitude when generating electrical stimulation output, it is usually necessary to consider using the limited bit depth as much as possible to reconstruct the most important external acoustic signals for the implant recipient, such as medium and low-loudness speech. This process consists of two parts: noise reduction and compression. Among them, noise reduction refers to separating and filtering the noise components in the speech to achieve the effect of speech enhancement, which can be achieved through adaptive filtering methods, spectral subtraction, or multi-channel beamforming methods; compression refers to limiting the amplitude of excessive signals, which can be achieved through automatic gain control (AGC) or wide dynamic range control (WDRC) algorithms.

[0051] Next, the technical solutions in the embodiments of the present application will be described in detail with reference to the accompanying drawings in the embodiments of the present application.

[0052] As Figure 2As shown in the figure, this embodiment provides a method for reconstructing an acoustic signal based on a cochlear implant, which specifically includes the following steps:

[0053] S11. Obtain an external acoustic signal, perform spatial domain filtering on the acoustic signal, and obtain a spatially filtered signal.

[0054] In one embodiment, the step of performing spatial domain filtering on the acoustic signal to obtain a spatially filtered signal includes:

[0055] Perform spatial domain filtering on the acoustic signal based on a multi-channel beamforming method to obtain a spatially filtered signal.

[0056] Specifically, it includes but is not limited to implementing microphone directivity through the Delay&Sum Beam-Forming (DSBF) algorithm to achieve the effect of spatial domain filtering.

[0057] As Figure 3 shown in the figure, a feasible beamforming method is shown, including two sub-processes of DSBF and high-pass filtering, and its output is a single-channel signal frame after spatial domain filtering.

[0058] The DSBF module is configured to: apply a specific time delay to one of the two-channel input signals, Mic_0, and subtract it from the other signal, Mic_1. In practical applications, the time delay is related to the relative positions of the microphone components and the desired beam direction, and this value can also be dynamic under certain conditions.

[0059] The high-pass filter is configured to: the signal processed by the DSBF module has a relatively obvious high-frequency attenuation, and it can be compensated by a high-pass filter. High-pass filtering can be implemented by a Finite Impulse Response (FIR) or Infinite Impulse Response (IIR) filter.

[0060] S12. Decompose the spatially filtered signal in the frequency domain to determine an initial energy spectrum.

[0061] Specifically, it includes but is not limited to transforming the original time-domain signal into a frequency-domain signal composed of M frequency bands through a Weighted Over-Lap&Add (WOLA) filter bank analysis algorithm. The output of the WOLA filter bank is represented by a complex vector. The calculation method of the energy spectrum is the square of the absolute value of the obtained P complex frequency bands. According to the number of electrode channels of the corresponding cochlear implant system, the above P complex frequency bands are combined into M sub-bands (M≤P).

[0062] S13. Transform the energy spectrum into a frequency spectrum.

[0063] As shown Figure 7 in the figure, the frequency-domain decomposition consists of 4 sub-processes: WOLA analysis filter bank, calculating the energy spectrum, band merging, and unit conversion, which are used to convert the input frame into an energy spectrum output composed of M sub-bands.

[0064] The WOLA analysis filter bank is configured to transform the input signal frame from the time domain into a complex frequency spectrum composed of P frequency bands. Among them, the WOLA analysis filter bank is an efficient implementation of the Discrete Fourier Transform (DFT) filter bank.

[0065] Calculating the energy spectrum: Taking the square of the modulus of the complex frequency spectrum S output by the WOLA analysis as its energy spectrum E (P) , that is

[0066]

[0067] where S p represents the complex output of the p-th frequency band, and represents the energy value of this frequency band.

[0068] Band merging: According to the requirements, formulate a band allocation table, and sum the above energy spectrum E (P) by frequency band according to this allocation table, and use it as the sub-band amplitude of the new energy spectrum E (M) , that is

[0069]

[0070] where represents the energy amplitude corresponding to the m-th sub-band in the energy spectrum composed of M sub-bands, and i represents the frequency band serial number in the energy spectrum E (P) corresponding to this sub-band.

[0071] Converting to decibels: Representing the energy spectrum E (M) in units of dB, that is

[0072] E (dB) = 10 · log 10 E (M) ,

[0073] where E (dB) represents the M-channel energy spectrum expressed in dB.

[0074] S14. Calculate the signal speech gain according to the estimation result of the noise spectrum of the sound signal.

[0075] Specifically, the signal speech gain is the difference (in dB) between the energy spectrum obtained in step S13 and the noise spectrum.

[0076] S15, amplitude-modulate the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient.

[0077] In one embodiment, the step of amplitude-modulating the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient includes:

[0078] In response to the signal speech gain being greater than a preset gain threshold, limit the signal speech gain by multiplying the signal speech gain by a specific compression coefficient less than 1, where the preset gain threshold and the specific compression coefficient are both related to the electroacoustic dynamic range.

[0079] Specifically, as Figure 8 shown, the calculation of the signal speech gain consists of three sub-processes: initial gain calculation, compression coefficient calculation, and gain compression, to calculate the system noise reduction gain in this way.

[0080] Initial gain calculation: Denote the initial signal gain G (iniy) as the difference between the energy spectrum E (dB) and the noise spectrum i.e.,

[0081]

[0082] Compression coefficient calculation: Calculate the compression parameters according to the EDR data of the implant recipient, including the compression threshold Thrd (G) and the compression coefficient γ. When the EDR is large, the implant recipient's hearing threshold is large, and a higher Thrd (G) and a lower γ value are taken; conversely, when the EDR is small, the implant recipient's hearing threshold is small, and a lower Thrd (G) and a larger γ value are taken. Among them, the EDR needs to be obtained through programming.

[0083] Gain compression: Limit the initial signal gain by channel according to the compression threshold Thrd (G) and the compression coefficient γ, i.e.,

[0084]

[0085] where G m represents the compression gain output of the m-th sub-band.

[0086] S16, apply the amplitude-modulated signal speech gain to the spectrum and output the reconstructed denoised speech energy spectrum.

[0087] In one embodiment, the step of applying the amplitude-modulated signal speech gain to the spectrum and outputting the reconstructed denoised speech energy spectrum includes:

[0088] Convert the speech gain of the amplitude-modulated signal into a linear value, multiply the linear gain of each subband by the amplitude of the corresponding subband of the spectrum in turn, and the output result is the denoised speech energy spectrum.

[0089] As Figure 9 shown, the application of the signal speech gain consists of two sub-processes: unit conversion and gain application, and its output is the denoised speech energy spectrum.

[0090] Unit conversion: Convert the compression gain G in dB to the linear gain coefficient G (lnr) , that is

[0091]

[0092] where represents the linear gain coefficient of the m-th subband.

[0093] Gain application: According to the requirements, formulate a frequency band allocation table, and multiply the initial energy spectrum E (P) by the linear gain coefficient G (lnr) channel by channel to obtain the denoised speech energy spectrum V, that is

[0094]

[0095] where V p represents the output of the p-th frequency band of the denoised speech energy spectrum, and there is a corresponding relationship between the frequency band p and the subband m. Typical noise reduction outputs are as Figure 10 , Figure 11 shown.

[0096] In an embodiment, as Figure 4 shown, the estimation process of the noise spectrum of the sound signal includes steps S17 - S19:

[0097] S17, determine the energy characteristics of the current signal frame of the sound signal.

[0098] Specifically, it includes but is not limited to using the root mean square (RMS) of the signal frame as its energy characteristic.

[0099] S18, judge whether the current signal frame is a noise frame according to the dynamic noise threshold.

[0100] In an embodiment, the step of judging whether the current signal frame is a noise frame according to the dynamic noise threshold includes:

[0101] Based on the currently dynamically set noise threshold and the energy characteristics, judge whether the current signal frame is a noise frame;

[0102] In response to the energy feature being lower than the noise threshold, determine that the current signal frame is a noise frame (assuming that the signal frame determined to be a noise frame does not contain voice components); in response to the energy feature being higher than or equal to the noise threshold, determine that the current signal frame is not a noise frame.

[0103] In one embodiment, after the step of determining whether the current signal frame is a noise frame according to the dynamic noise threshold, the method further includes:

[0104] If the energy feature is lower than the noise threshold, use the value of the energy feature as the new noise threshold; if the energy feature is higher than the noise threshold, increase the noise threshold according to a preset rule.

[0105] As Figure 5 shown, the frame energy calculation includes three sub - processes: calculating the energy feature, comparing the energy feature, and updating the noise threshold, which are used to determine whether the current frame is a noise frame.

[0106] Calculating the energy feature: By calculating the root - mean - square value \(X_{rms}\) of the output signal frame \(X = [x_1,\ldots,x_{L}]\) as its energy feature, that is n ,\ldots,x_{L} L as its energy feature, that is RMS where \(L\) is the length of the signal frame.

[0107]

[0108] where \(L\) is the length of the signal frame.

[0109] Energy feature discrimination: Compare the energy feature with the noise threshold. If the energy feature \(X_{rms}[n]\) of the current frame is less than the threshold \(Thrd[n - 1]\), then determine it as a noise frame; conversely, if it is greater than the threshold, it is considered that there is a voice component in the current signal frame. RMS [n] less than the threshold \(Thrd (N) [n - 1]\), then determine it as a noise frame; conversely, if it is greater than the threshold, it is considered that there is a voice component in the current signal frame.

[0110] Noise threshold update: Update the noise threshold according to the result of the energy feature discrimination. If the energy feature is less than the noise threshold, use the energy feature as the new threshold value, that is

[0111] Thrd (N) [n]=X_{rms} RMS [n];

[0112] Conversely, if it is greater than the threshold, multiply the current threshold by a variable coefficient, that is

[0113] Thrd (N) [n]=τ·Thrd (N) [n - 1],

[0114] where τ > 1 is the noise threshold update coefficient, which can be fixed or vary according to a specific strategy. For example, Figure 6 shows a typical energy feature X RMS , the relationship between the noise threshold Thrd and the noise frames. Among them, the upper X RMS曲线 is the energy feature curve of a noisy speech signal within a certain period of time.

[0115] S19. In response to the current signal frame being a noise frame, the estimation of the noise spectrum is updated according to the energy spectrum of the current signal frame.

[0116] Specifically, the method for updating the noise spectrum is the weighted sum (in dB) of the energy spectrum obtained in step S13 and the current noise spectrum.

[0117] Specifically, the noise spectrum estimation process is as follows: If it is determined that the current frame is a noise frame, then its energy spectrum E (dB) is weighted and added to the current noise spectrum estimation as the new noise spectrum estimation, that is

[0118]

[0119] where 0 < α < 1 is the weighting coefficient.

[0120] The protection scope of the method for reconstructing a sound signal based on a cochlear implant according to the embodiments of the present application is not limited to the execution order of the steps listed in this embodiment. Any solution achieved by adding or reducing steps of the prior art and replacing steps according to the principle of the present application is included in the protection scope of the present application.

[0121] The embodiments of the present application further provide a system for reconstructing a sound signal based on a cochlear implant. The system for reconstructing a sound signal based on a cochlear implant can implement the method for reconstructing a sound signal based on a cochlear implant described in the present application. However, the implementation devices of the method for reconstructing a sound signal based on a cochlear implant described in the present application include but are not limited to the structure of the system for reconstructing a sound signal based on a cochlear implant listed in this embodiment. Any structural deformation and replacement of the prior art made according to the principle of the present application are included in the protection scope of the present application.

[0122] For example, Figure 12 as shown, this embodiment provides a system 1 for reconstructing a sound signal based on a cochlear implant, including: a beamforming module 11, a frequency-domain decomposition module 12, a gain calculation module 13, a gain application module 14, a frame energy calculation module 15, and a noise spectrum estimation module 16.

[0123] The beamforming module 11 is configured to obtain an external sound signal and perform spatial filtering on the sound signal based on a multi-channel beamforming method to obtain a spatially filtered signal.

[0124] The frequency-domain decomposition module 12 is configured to perform frequency-domain decomposition on the spatial-domain filtered signal to determine an initial energy spectrum; and transform the energy spectrum into a frequency spectrum.

[0125] The gain calculation module 13 is configured to calculate a signal speech gain according to an estimation result of a noise spectrum of the sound signal; and perform amplitude modulation on the signal speech gain in combination with an electro-auditory dynamic range of an implant user.

[0126] The gain application module 14 is configured to apply the amplitude-modulated signal speech gain to the frequency spectrum and output a reconstructed denoised speech energy spectrum.

[0127] The frame energy calculation module 15 is configured to determine an energy feature of a current signal frame of the sound signal; and determine whether the current signal frame is a noise frame according to a dynamic noise threshold.

[0128] The noise spectrum estimation module 16 is configured to update an estimation of the noise spectrum according to the energy spectrum of the current signal frame in response to the current signal frame being a noise frame.

[0129] As Figure 13 shown, the reconstruction system of the sound signal based on a cochlear implant according to the present application, as a voice adaptive noise reduction and compression system, includes a microphone (Mic) input, a beamforming module 11 (corresponding to method step S11), a frequency-domain decomposition module 12 (corresponding to method steps S12 and S13), a gain calculation module 13 (corresponding to method steps S14 and S15), a gain application module 14 (corresponding to method step S16), a frame energy calculation module 15 (corresponding to method steps S17 and S18), and a noise spectrum estimation module 16 (corresponding to method step S19). Its input is a dual-channel signal frame composed of time-domain signals of a specific length, which is composed of two channels, Mic_0 and Mic_1; its output is a denoised speech energy spectrum after noise reduction and compression.

[0130] The specific implementation process of the sound signal reconstruction system based on the cochlear implant is as follows: The beamforming module 11 combines the dual-channel Mic signals into a single channel according to a certain strategy, with the frame length remaining unchanged; the frame energy calculation module 15 calculates the energy characteristics within the frame, determines whether the current frame is a noise frame, and updates the noise threshold value; the WOLA analysis filter bank in the frequency-domain decomposition module 12 decomposes the input signal into a complex frequency spectrum consisting of N frequency bands, and combines P frequency bands into M sub-bands (M ≤ P) according to a certain strategy based on the number of electrode channels of the corresponding cochlear implant system; the noise spectrum estimation module 16 updates the noise spectrum estimation based on the energy spectrum of the noise frame; the gain calculation module 13 calculates the signal energy gain based on the current frame energy spectrum and the noise spectrum, and performs limiting compression on this gain according to the implantee's EDR data; the gain application module 14 applies the obtained signal energy gain to the initial energy spectrum to obtain the system output, that is, the signal energy spectrum after noise reduction and compression.

[0131] In several embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of modules / units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or modules or units can be in an electrical, mechanical or other forms.

[0132] The modules / units described as separate components may or may not be physically separated. The components shown as modules / units may or may not be physical modules, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules / units can be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, in each embodiment of the present application, the various functional modules / units can be integrated in a processing module, or each module / unit can exist physically alone, or two or more modules / units can be integrated in one module / unit.

[0133] Those of ordinary skill in the art should also be further aware that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0134] As Figure 14 shown, this embodiment provides an electronic device 2, including: a processor 21 and a memory 22; the memory 22 is used to store a computer program, and the processor 21 is used to execute the computer program stored in the memory 22 so that the electronic device 2 executes the method described above.

[0135] The above-mentioned processor 21 may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; it may also be a digital signal processor (Digital Signal Processing, abbreviated as DSP), an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), a field programmable gate array (Field Programmable Gate Array, abbreviated as FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0136] The above-mentioned memory 22 may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0137] The descriptions of the processes or structures corresponding to the above-mentioned respective drawings have their own focuses. For parts not detailed in a certain process or structure, reference can be made to the relevant descriptions of other processes or structures.

[0138] The above embodiments are only illustrative of the principles and effects of this application, and are not used to limit this application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by this application should still be covered by the claims of this application.

Claims

1. A method for reconstructing a sound signal based on a cochlear implant, characterized in that, The method includes: Obtaining an external sound signal, performing spatial domain filtering on the sound signal to obtain a spatially filtered signal; Performing frequency domain decomposition on the spatially filtered signal to determine an initial energy spectrum; Transforming the energy spectrum into a frequency spectrum; Calculating a signal speech gain according to an estimation result of the noise spectrum of the sound signal; Amplitude modulating the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient; Applying the amplitude-modulated signal speech gain to the frequency spectrum and outputting a reconstructed denoised speech energy spectrum.

2. The method according to claim 1, characterized in that The step of performing spatial domain filtering on the sound signal to obtain a spatially filtered signal includes: Performing spatial domain filtering on the sound signal based on a multi-channel beamforming method to obtain a spatially filtered signal.

3. The method according to claim 1, characterized in that, The estimation process of the noise spectrum of the sound signal includes: Determining an energy feature of a current signal frame of the sound signal; Judging whether the current signal frame is a noise frame according to a dynamic noise threshold; In response to the current signal frame being a noise frame, updating the estimation of the noise spectrum according to the energy spectrum of the current signal frame.

4. The method according to claim 3, characterized in that, The step of judging whether the current signal frame is a noise frame according to a dynamic noise threshold includes: Judging whether the current signal frame is a noise frame based on a currently dynamically set noise threshold and the energy feature; In response to the energy feature being lower than the noise threshold, determining that the current signal frame is a noise frame; in response to the energy feature being higher than or equal to the noise threshold, determining that the current signal frame is not a noise frame.

5. The method according to claim 4, wherein After the step of judging whether the current signal frame is a noise frame according to a dynamic noise threshold, the method further includes: If the energy feature is lower than the noise threshold, using the value of the energy feature as a new noise threshold; if the energy feature is higher than the noise threshold, increasing the noise threshold according to a preset rule.

6. The method according to claim 1, characterized in that, The step of amplitude modulating the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient includes: In response to the signal speech gain being greater than a preset gain threshold, limiting the signal speech gain by multiplying the signal speech gain by a specific compression coefficient less than 1, where the preset gain threshold and the specific compression coefficient are both related to the electroacoustic dynamic range.

7. The method according to claim 1, wherein The step of applying the amplitude-modulated signal speech gain to the frequency spectrum and outputting a reconstructed denoised speech energy spectrum includes: Transforming the amplitude-modulated signal speech gain into a linear value, multiplying the linear gain of each sub-band by the amplitude of the corresponding sub-band of the frequency spectrum in turn, and the output result is the denoised speech energy spectrum.

8. A sound signal reconstruction system based on a cochlear implant, characterized in that, The system includes: A beamforming module configured to obtain an external sound signal and perform spatial domain filtering on the sound signal based on a multi-channel beamforming method to obtain a spatially filtered signal; A frequency domain decomposition module configured to perform frequency domain decomposition on the spatially filtered signal to determine an initial energy spectrum; transform the energy spectrum into a frequency spectrum; A gain calculation module configured to calculate a signal speech gain according to an estimation result of the noise spectrum of the sound signal; amplitude modulate the signal speech gain in combination with the electroacoustic dynamic range of the implant recipient; A gain application module, configured to apply an amplitude-modulated signal speech gain to the spectrum and output a reconstructed denoised speech energy spectrum.

9. The system according to claim 8, wherein The system further includes: A frame energy calculation module, configured to determine the energy feature of the current signal frame of the sound signal; and determine whether the current signal frame is a noise frame according to a dynamic noise threshold. A noise spectrum estimation module, configured to, in response to the current signal frame being a noise frame, update the estimation of the noise spectrum according to the energy spectrum of the current signal frame.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory. The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

Images