Sound signal processing method and apparatus based on closed-loop control, device, and medium

Abstract

The application provides a sound signal processing method, device, equipment and medium based on closed-loop control. The method comprises the following steps: converting a time-domain sound signal into a frequency-domain sound signal to obtain an energy spectrum; distributing the energy spectrum to sub-bands in an artificial cochlea electrode, performing a pooling operation on the sub-bands to obtain a sub-band output value corresponding to each sub-band; performing data processing on the sub-band output value to obtain a current value, inputting the current value into a corresponding electrode implanted in a body respectively to obtain a biphasic electric stimulation pulse, stimulating a nerve fiber in the cochlea based on the biphasic electric stimulation pulse to obtain a telemetry waveform; removing an electric stimulation artifact in the telemetry waveform to obtain a nerve response signal; obtaining an ECAP voltage based on the nerve response signal; updating a maximum clinical current value of the implant based on the ECAP voltage, adjusting an electric auditory dynamic range of the implant through the updated maximum clinical current value, and forming a closed-loop regulation and control on the electric auditory dynamic range, thereby solving the problem of T / C value mismatch of the artificial cochlea.

Description

Technical Field

[0001] This application belongs to the field of cochlear implant signal processing technology, and relates to a sound signal processing method based on closed-loop control, and particularly to a sound signal processing method, device, equipment and medium based on closed-loop control. Background Technology

[0002] Cochlear implant technology is currently recognized worldwide as one of the most effective methods and devices for restoring hearing in patients with bilateral severe or profound sensorineural hearing loss. For most mainstream cochlear implant systems, external sound is first collected by a microphone and converted into an electrical signal. After filtering, compression, noise reduction, and encoding, the signal is transmitted into the body via a transmitting coil located behind the ear. The receiving coil in the implant senses the signal, and the decoding chip decodes it, causing the stimulating electrodes to generate a current, thereby stimulating the auditory nerve to produce hearing. The amplitude of the electrical pulse is usually determined by the sensitivity and tolerance of the nerve fibers in the cochlea to electrical stimulation at the electrode implantation site. The minimum clinical current value that can be perceived by the implantee is called the electrical stimulation threshold or T-value, while the maximum clinical current value that will not cause discomfort is called the comfort threshold or C-value. The area between the T-value and the C-value is the implantee's electroauditory dynamic range (EDR). In practical applications, the range of the T / C value is not constant; changes in the T / C value can lead to a mismatch with the original tuning results. Therefore, based on the actual needs of different implant recipients, how to improve the T / C value mismatch in the implant has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a sound signal processing method, apparatus, device, and medium based on closed-loop control, which is used to solve the technical problem of T / C value mismatch in implantees after cochlear implantation in the prior art.

[0004] In a first aspect, embodiments of this application provide a sound signal processing method based on closed-loop control, the method comprising:

[0005] Acquire a time-domain sound signal; convert the time-domain sound signal into a frequency-domain sound signal and obtain the energy spectrum corresponding to the frequency-domain sound signal; allocate the energy spectrum to a preset number of sub-bands within the cochlear implant electrode, and sequentially perform pooling operations on each sub-band to obtain the sub-band output value corresponding to each sub-band; perform data processing on the sub-band output values ​​to obtain the current value corresponding to the sub-band output value; input the current value into the corresponding electrode implanted in the body to obtain biphasic electrical stimulation pulses; based on the biphasic electrical stimulation pulses, traverse the ear according to a preset electrode stimulation order. Cochlear nerve fibers are stimulated to generate neural action potentials. After each traversal, the neural action potentials generated by the telemetry electrodes are telemetryd to obtain a telemetry waveform. Electrical stimulation artifacts in the telemetry waveform are removed to obtain the neural response signal corresponding to the telemetry waveform. Based on the minimum and maximum values ​​of the neural response signal, the electrically evoked compound action potential voltage is obtained. The maximum clinical current value of the implant is updated based on the electrically evoked compound action potential voltage, and the electroauditory dynamic range of the implant is adjusted by the updated maximum clinical current value.

[0006] In one implementation of the first aspect, the step of processing the sub-band output value to obtain the current value corresponding to the sub-band output value includes: performing a compression operation on the sub-band output value to obtain a sub-band compressed value corresponding to the sub-band output value; and performing a mapping process on the sub-band compressed value to obtain the current value corresponding to the sub-band compressed value.

[0007] In one implementation of the first aspect, the expression for compressing the sub-band output value to obtain the sub-band compressed value corresponding to the sub-band output value is:

[0008]

[0009] in, E represents the subband compression value. j X represents the output value of the j-th sub-band. min X thrd X max These represent the initial compression parameters.

[0010] In one implementation of the first aspect, the step of mapping the sub-band compression value to obtain the current value corresponding to the sub-band compression value includes: performing logarithmic processing on the sub-band compression value to obtain the sub-band logarithmic value corresponding to the sub-band compression value; and performing linear scaling processing on the sub-band logarithmic value between the maximum clinical current value and the minimum clinical current value of the implant to obtain the current value corresponding to the sub-band logarithmic value.

[0011] In one implementation of the first aspect, the method further includes: the telemetry electrode is one of the electrodes implanted in the body.

[0012] In one implementation of the first aspect, the telemetry waveform includes a first telemetry waveform and a second telemetry waveform. The method for determining the telemetry waveform includes: recording the first telemetry waveform using a recording electrode, wherein the recording electrode is an electrode other than the current telemetry electrode; determining a second biphasic electrical stimulation pulse based on a first biphasic electrical stimulation pulse corresponding to the first telemetry waveform, wherein the first biphasic electrical stimulation pulse and the second biphasic electrical stimulation pulse have the same current amplitude but opposite phase; and stimulating the cochlear nerve fibers again using the telemetry electrode based on the second biphasic electrical stimulation pulse to obtain the second telemetry waveform recorded by the recording electrode.

[0013] In one implementation of the first aspect, the expression for removing electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform is:

[0014] S=(N+P) / 2-D

[0015] Where S represents the neural response signal, N represents the first telemetry waveform, P represents the second telemetry waveform, and D represents the amplifier offset baseline.

[0016] In the closed-loop control-based sound signal processing method provided in this application embodiment, the time-domain sound signal is converted into a frequency-domain sound signal, and the energy spectrum corresponding to the frequency-domain sound signal is obtained. The energy spectrum is allocated to a preset number of sub-frequency bands within the cochlear implant electrode, and pooling is performed on each sub-frequency band sequentially to obtain the sub-frequency band output value corresponding to each sub-frequency band. The sub-frequency band output value is processed to obtain the current value corresponding to the sub-frequency band output value. The current value is input to the corresponding electrode implanted in the body to obtain biphasic electrical stimulation pulses. Based on the biphasic electrical stimulation pulses, the nerve fibers within the cochlea are traversed according to a preset electrode stimulation order to stimulate the nerve fibers within the cochlea. Stimulation is applied to generate neural action potentials. After each traversal, the neural action potentials generated by the telemetry electrodes are telemetryd to obtain telemetry waveforms. Electrical stimulation artifacts in the telemetry waveforms are removed to obtain the neural response signals corresponding to the telemetry waveforms. Based on the minimum and maximum values ​​in the neural response signals, the electrically evoked compound action potential voltage is obtained. The maximum clinical current value of the implant is updated based on the electrically evoked compound action potential voltage. The updated maximum clinical current value is used to adjust the electroauditory dynamic range of the implant, forming a closed-loop control of the electroauditory dynamic range of the implantee, thus solving the problem of T / C value mismatch after cochlear implantation.

[0017] Secondly, embodiments of this application provide a sound signal processing device based on closed-loop control. The closed-loop control-based sound signal processing device includes: a signal acquisition module for acquiring a time-domain sound signal; a signal conversion module for converting the time-domain sound signal into a frequency-domain sound signal and obtaining an energy spectrum corresponding to the frequency-domain sound signal; a sub-band output value determination module for allocating the energy spectrum to a preset number of sub-bands within the cochlear implant electrode, and sequentially performing pooling operations on each sub-band to obtain a sub-band output value corresponding to each sub-band; a current value determination module for processing the sub-band output values ​​to obtain a current value corresponding to the sub-band output value; a pulse determination module for inputting the current values ​​into corresponding electrodes implanted in the body to obtain biphasic electrical stimulation pulses; and a telemetry waveform. The module is configured to: determine the neural response action potential (NPT) by traversing the cochlear nerve fibers according to a preset electrode stimulation sequence based on the biphasic electrical stimulation pulses, stimulating the cochlear nerve fibers to generate NPT, and telemetry the NPT generated by the telemetry electrodes after each traversal to obtain a telemetry waveform; determine the neural response signal by removing electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform; determine the voltage of the electrically evoked compound action potential (ECP) by obtaining the voltage of the ECP based on the minimum and maximum values ​​of the neural response signal; and determine the dynamic range of the electro-auditory system by updating the maximum clinical current value of the implant based on the voltage of the ECP and adjusting the dynamic range of the implant based on the updated maximum clinical current value.

[0018] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the sound signal processing method based on closed-loop control as described in any one of the first aspects of embodiments of this application.

[0019] Fourthly, embodiments of this application provide an electronic device, the electronic device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the sound signal processing method based on closed-loop control as described in any one of the first aspects of the embodiments of this application. Attached Figure Description

[0020] Figure 1A The diagram shown is an application scenario diagram corresponding to the sound signal processing method based on closed-loop control provided in an embodiment of this application.

[0021] Figure 1B The flowchart shown is a sound signal processing method based on closed-loop control provided in an embodiment of this application.

[0022] Figure 2 The diagram shown is a schematic diagram illustrating the calculation of sub-channel output values ​​using the max pooling method in one embodiment of this application.

[0023] Figure 3 The diagram shown is a schematic of removing electrical stimulation artifacts from telemetry waveforms according to an embodiment of this application.

[0024] Figure 4 The curve shown is a neural response waveform in one embodiment of this application.

[0025] Figure 5 The figure shows the curve of the voltage of the electrically evoked compound action potential as a function of the amplitude of the stimulation current in one embodiment of this application.

[0026] Figure 6 Shown as |C0-C in one embodiment of this application j A diagram illustrating the correspondence between | and γ.

[0027] Figure 7 This is a flowchart illustrating data processing of sub-band output values ​​in one embodiment of this application.

[0028] Figure 8 The diagram shown is a schematic diagram of the compression curve provided in one embodiment of this application.

[0029] Figure 9 The diagram shown is a current mapping schematic provided in an embodiment of this application.

[0030] Figure 10 The diagram shown is a schematic diagram of an implanted electrode provided in an embodiment of this application.

[0031] Figure 11 The flowchart shown is a process for determining a telemetry waveform according to an embodiment of this application.

[0032] Figure 12 The diagram shown illustrates the electrode stimulation sequence provided in one embodiment of this application.

[0033] Figure 13 The flowchart shown is a corresponding to another sound signal processing method based on closed-loop control provided in an embodiment of this application.

[0034] Figure 14 The diagram shown is a schematic of a sound signal processing device based on closed-loop control provided in one embodiment of this application.

[0035] Figure 15 The diagram shown is a structural schematic of an electronic device according to an embodiment of this application.

[0036] Component designation explanation

[0037] Steps S11~S119

[0038] Steps S71 to S73

[0039] S111~S113 Step 140 Sound signal processing device based on closed-loop control

[0041] 141 Signal Acquisition Module

[0042] 142 Signal Conversion Module

[0043] 143 Sub-band Output Value Determination Module

[0044] 144 Current Value Determination Module

[0045] 145 Pulse Determination Module

[0046] 146 Telemetry Waveform Determination Module

[0047] 147 Neural Response Signal Determination Module

[0048] 148 Electrically Evoked Composite Action Potential Voltage Determination Module

[0050] 149 Electro-auditory Dynamic Range Determination Module

[0051] 150 electronic devices

[0052] 151 processor

[0053] 152 Non-volatile storage media

[0054] 153 System Bus

[0055] 154 internal memory

[0056] 155 Network Interface Detailed Implementation

[0057] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0058] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0059] The T / C value within implanted cochlear implants in current technologies is not constant. For many recipients, the T / C value changes by approximately 20–60 CL over a period of 2–4 years after cochlear implantation. In some recipients, this change shows an increasing trend; while in others, it does not exhibit a clear trend. This change in T / C value leads to a mismatch with the initial device settings, further contributing to a decline in the recipient's recovery outcome. This is something that current AGC or ASC algorithms cannot address. Therefore, there is currently a lack of mature sound signal processing methods to promptly improve the T / C value mismatch problem in recipients.

[0060] At least to address the aforementioned problems, embodiments of this application provide a sound signal processing method based on closed-loop control. This sound signal processing method based on closed-loop control can solve the technical problem of the lack of mature sound signal processing methods in the prior art, and timely improve the T / C value mismatch in implant recipients.

[0061] Figure 1A The diagram shown illustrates an application scenario of the closed-loop control-based sound signal processing method provided in an embodiment of this application. Figure 1A As shown, this application scenario includes two parts: an external sound processor and an internal cochlear implant. The sound processor should at least include components for acquiring sound signals, such as a microphone, a processor chip, a radio frequency transmission chip, an antenna, and a power supply; the implant should at least include components for decoding stimulation, implanted electrodes, and an antenna. During use, the internal and external parts of the cochlear implant system establish bidirectional communication via wireless radio frequency technology.

[0062] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0063] Figure 1B The flowchart shown is a sound signal processing method based on closed-loop control provided in an embodiment of this application. Figure 1B As shown, the sound signal processing method based on closed-loop control provided in this application includes the following steps S11 to S19.

[0064] S11, acquire time-domain audio signal.

[0065] Specifically, the sound processor collects time-domain sound signals through a microphone, and the process includes: sound waves propagate through the air to the diaphragm of the microphone, the diaphragm vibrates along with the air and generates corresponding electrical signals based on the amplitude of the vibration; then the analog-to-digital converter in the microphone converts the collected electrical signals into time-domain sound signals, so as to obtain time-domain sound signals.

[0066] S12, converting the time-domain sound signal into a frequency-domain sound signal, and obtaining an energy spectrum corresponding to the frequency-domain sound signal.

[0067] For example, based on an N-point fast Fourier transform (FFT) method, the time-domain sound signal can be transformed into a complex frequency spectrum with a length of N / 2, expressed as F1, F2, ..., F N / 2 .

[0068] Specifically, the calculation method of the energy spectrum is: sequentially process F1, F2, ..., F N / 2 take the modulus, and calculate the square value |F1| 2 , |F2| 2 , ..., |F N / 2 | 2 .

[0069] S13, distributing the energy spectrum to a preset number of sub-bands in the cochlear implant electrode, and sequentially performing a pooling operation on each of the sub-bands to obtain a sub-band output value corresponding to each of the sub-bands.

[0070] Wherein, the preset number of sub-bands can be represented by M.

[0071] It should be noted that the number of sub-bands is not greater than the number of electrodes of the cochlear implant, which aims to ensure that each electrode corresponds to one stimulation channel. If the number of sub-bands exceeds the number of electrodes, there will not be enough electrodes to independently stimulate these sub-bands.

[0072] For example, the distribution method of the energy spectrum is: for any sub-band, its lower cut-off frequency is expressed as f (l) , the upper cut-off frequency is expressed as f (h) , then there must be definite F m , F n (m<n), such that the lower cut-off frequency of sub-band E j satisfies satisfies the upper cut-off frequency satisfies then a distribution relationship is defined between sub-band E j and F m , F n to form a distribution relationship.

[0073] Among them, sub-bands E1, ..., E M The cutoff frequencies are consecutive, that is...

[0074] When performing pooling operations on each sub-band sequentially to obtain the sub-band output value corresponding to each sub-band, the max pooling method is used, that is, for any j∈[1,2,...,M], let E j =max(|F m | 2 , ..., |F n | 2 E at this time j This refers to the sub-band output value. A schematic diagram illustrating the calculation of the sub-channel output value using the max-pooling method can be found in [reference needed]. Figure 2 .

[0075] It should be noted that the method of constructing the allocation relationship between the energy spectrum and sub-bands determines the response range of the physical channel output by the cochlear implant system to sound frequencies. For example, a pre-designed cochlear implant frequency response range can be mapped to a Mel-Cepstral spectrum, and then evenly divided into M equal parts within the Mel-Cepstral spectrum. Finally, the cutoff frequencies of these M bands are remapped to the frequency spectrum, thus obtaining a frequency allocation relationship similar to the frequency topology of the human cochlea. However, the embodiments of this application do not limit the specific frequency response range and frequency allocation method of the cochlear implant system.

[0076] S14, perform data processing on the sub-frequency band output value to obtain the current value corresponding to the sub-frequency band output value.

[0077] Specifically, data processing includes at least one of compression operations and mapping operations.

[0078] It should be noted that, in addition to the two processing methods listed above, data processing of the sub-frequency band output values ​​may also include other suitable data processing methods, such as data preprocessing, etc., and this application does not impose any restrictions on this.

[0079] S15, the current values ​​are input into the corresponding electrodes implanted in the body to obtain biphasic electrical stimulation pulses.

[0080] Specifically, M electrodes generate biphasic electrical stimulation pulses with negative pulses preceding positive pulses, and the stimulation current amplitudes are A1, A2, ..., A... M .

[0081] S16, based on the biphasic electrical stimulation pulse, the cochlear nerve fibers are traversed according to the preset electrode stimulation sequence to stimulate the cochlear nerve fibers and generate nerve response action potentials. After each traversal, the nerve response action potentials generated by the telemetry electrodes are telemetryd to obtain telemetry waveforms.

[0082] The telemetry electrode is one of the electrodes implanted in the body.

[0083] For example, the method for selecting telemetry electrodes is as follows: each electrode performs telemetry in a sequential cycle. That is, for any j∈[1,2,...,M-1], after the electrode with index j completes its telemetry, the next round of traversal stimulation is performed by the electrode with index j+1; after the electrode with index M completes its telemetry, the next round of traversal stimulation is performed by the electrode with index 1, thus obtaining the telemetry waveform.

[0084] S17, Remove electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform.

[0085] Specifically, the telemetry waveform includes neural response activity, stimulus artifacts, and baseline deviation.

[0086] In some embodiments, the expression for removing electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform is:

[0087] S=(N+P) / 2-D

[0088] Where S represents the neural response signal, N represents the first telemetry waveform, P represents the second telemetry waveform, and D represents the amplifier offset baseline.

[0089] For a schematic diagram of removing electrical stimulation artifacts from telemetry waveforms, please refer to [link / reference needed]. Figure 3 .

[0090] S18. Based on the minimum and maximum values ​​in the neural response signal, the voltage of the electrically evoked compound action potential is obtained.

[0091] Specifically, before determining the voltage of the electrically evoked compound action potential (ECAP), it is necessary to determine the minimum value S of the neural response signal. - and maximum value S + Does the timing and magnitude of the ECAP voltage meet expectations? If they do, then... If the data does not meet the requirements, discard the current data and skip the subsequent steps of updating the maximum clinical current value.

[0092] For example, the minimum value S of the neural response signal - and maximum value S+ The occurrence time is related to the selection of parameters such as the recording electrode and the sampling delay. For example, the minimum value S - Most of them occur within 0.8ms after the start of sampling, with a maximum value S. + Then it should appear later than the minimum value, and the minimum value S - The amplitude is generally negative, while the maximum value S + The amplitude is generally positive.

[0093] For example, the calculation method for the neural response waveform corresponding to the neural response signal is as follows: Perform electrical stimulation once in the order of negative pulse first, followed by positive pulse, and acquire the first telemetry waveform N corresponding to the neural response waveform; then perform electrical stimulation once in the order of positive pulse first, followed by negative pulse, and acquire the second telemetry waveform P corresponding to the neural response waveform; according to the formula... Calculate the average of the first and second telemetry waveforms. Repeat the above steps to calculate the average, superimposing the first and second telemetry waveforms to obtain the average waveform, i.e., according to the formula... Calculate the average waveform, where k represents the number of repetitions of the stimulus. Subtracting the amplifier offset baseline D from the neural response signal S, the neural response waveform expression is obtained as follows:

[0094] It should be noted that the value of k can be any positive integer, such as 20, 30, etc. The specific value of k can be reasonably determined according to the specific application scenario, and this application does not impose any restrictions on it.

[0095] It should be noted that a higher number of repetitions helps improve the stability of the measurement results and enhances the effectiveness of the closed-loop control-based sound signal processing method of this invention. However, when the number of repetitions is too high, the improvement in the stability of the measurement results will be very limited, and it can easily lead to excessively long testing times, causing fatigue in the implant recipient. In practical applications, the value of k should be reasonably determined.

[0096] The curve representation of the neural response waveform S can be found in [reference needed]. Figure 4 .Depend on Figure 4 It can be seen that in the neural response waveform S, the minimum value S - Located at the lowest point of the neural response waveform, the maximum value S + It is located at the highest peak of the neural response waveform.

[0097] S19, update the maximum clinical current value of the implant based on the electrically evoked compound action potential voltage, and adjust the electro-auditory dynamic range of the implant using the updated maximum clinical current value.

[0098] The maximum clinical current value can be represented by C.

[0099] Specifically, the method for updating the maximum clinical current value includes: for any j∈[1,2,...,M], the updated C value C j 'Can be achieved through the current C value C j Adding an adjustment term, the formula is expressed as follows: Where γ represents the normalization coefficient. This represents the reference voltage, which is independent of the current telemetry results, and α represents the parameter update rate.

[0100] Among them, the reference voltage is determined. The corresponding expression is: Where C0 and T0 represent the initial mapping parameters, and A represents the amplitude of the stimulation current. This represents the ECAP voltage when the stimulation current amplitude is C0. Specifically, the curve showing the change of ECAP voltage with the stimulation current amplitude is as follows: Figure 5 As shown.

[0101] The formula for calculating the normalization coefficient γ is as follows: The normalization coefficient γ serves to limit the range of the updated C value, preventing it from becoming too large or too small. It is worth noting that, in this embodiment of the invention, the specific value of the updated range of the C value is not limited.

[0102] For example, when C j It is very close to C0, i.e., |C0-C j When |→0, the normalization coefficient γ→1, meaning it will not limit the update rate of the C value. When |C0-C j |→20 - When γ→0 + When |C0-C j When |>20, γ<0. Clearly, the updated C value is now constrained to within ±20 current levels (CL) of the initial C value C0. The above |C0-C j The correspondence between | and γ is as follows: Figure 6 As shown.

[0103] Specifically, after the maximum clinical current value is updated, the T / C ratio is redefined, and the electro-auditory dynamic range of the implant is adjusted based on the redefined T / C ratio.

[0104] This application provides a sound signal processing method based on closed-loop control. In this method, a time-domain sound signal is converted into a frequency-domain sound signal, and the energy spectrum corresponding to the frequency-domain sound signal is allocated to a preset number of sub-frequency bands within the cochlear implant electrode. Through pooling, the output value of each sub-frequency band is obtained. Data processing is performed on the sub-frequency band output values ​​to obtain the current value corresponding to the frequency band output value. The current value is then input to the corresponding electrodes implanted in the body to obtain biphasic electrical stimulation pulses. Based on the biphasic electrical stimulation pulses, nerve fibers within the cochlea are stimulated, generating neural action potentials. After each traversal, the neural action potentials generated by the telemetry electrode are telemetryd to obtain a telemetry waveform. By performing an operation to remove electrical stimulation artifacts from the telemetry waveform, the neural response signal corresponding to the telemetry waveform is obtained. Based on the neural response signal... The minimum and maximum values ​​are used to obtain the electrically evoked compound action potential (ECAP) voltage. The maximum clinical current value of the implant is updated using the ECAP voltage. Then, the electro-auditory dynamic range of the implant is adjusted using the updated maximum clinical current value. This method updates the maximum clinical current value in a scientific way, without relying on the user's subjective feedback. It can objectively and accurately update the maximum clinical current value of the implant in real time using the data corresponding to the acquired time-domain sound signal, thereby adjusting the electro-auditory dynamic range of the implant. This effectively solves the problem of T / C mismatch after cochlear implantation. For some implantees who are not convenient to adjust the device on-site, this method can effectively extend the adjustment interval and improve the problem of reduced adjustment effect caused by EDR changes over time during long-term cochlear implant use. This meets the actual needs of different users and improves the user's rehabilitation effect.

[0105] Figure 7 This is a flowchart illustrating data processing of sub-band output values ​​in one embodiment of this application. For example... Figure 7 As shown, the process of data processing for the sub-band output value in this embodiment includes the following steps S71 to S72.

[0106] S71, perform a compression operation on the sub-frequency band output value to obtain the sub-frequency band compressed value corresponding to the sub-frequency band output value.

[0107] Specifically, the compression method is as follows: for any j∈[1,2,...,M], if E j <X min Then let the compressed output If X min <E j ≤X thrd Then keep Unchanged; if E j >X thrd Then E j Compress at a ratio of 3:1; when Ej =X max hour, It reaches its maximum value and no longer follows E. j It changes with the improvement.

[0108] In some embodiments, the expression for the sub-frequency band compressed value corresponding to the sub-frequency band output value is as follows:

[0109]

[0110] in, E represents the subband compression value. j X represents the output value of the j-th sub-band. min X thrd X max These represent the initial compression parameters.

[0111] Please see Figure 8 , Figure 8 The diagram shown is a schematic diagram of the compression curve provided in an embodiment of this application.

[0112] It should be noted that the compression method in this embodiment uses the energy spectrum sub-band output based on frequency relationships as the control object, and independently controls the compression gain for each sub-band. Its advantages include preserving the frequency details of the signal; for implant recipients with a narrow EDR range, this method also helps improve the overall loudness resolution.

[0113] S72, perform mapping processing on the sub-band compression value to obtain the current value corresponding to the sub-band compression value.

[0114] In some embodiments, the step of mapping the sub-band compression value to obtain the current value corresponding to the sub-band compression value includes: performing logarithmic processing on the sub-band compression value to obtain the sub-band logarithmic value corresponding to the sub-band compression value; and performing linear scaling processing on the sub-band logarithmic value between the maximum clinical current value and the minimum clinical current value of the implant to obtain the current value corresponding to the sub-band logarithmic value.

[0115] Specifically, for any j∈[1, 2, ..., M], if The logarithms of the compressed sub-band outputs are then calculated sequentially, and expressed as:

[0116]

[0117] like Then it is not correct. Calculate its logarithm.

[0118] It should be noted that, due to the monotonicity of logarithmic function data, therefore It is bounded, and its range of values ​​is...

[0119]

[0120] like Then the above A j Scale linearly to a value T. j and C value C j Between, denoted as like Then let Please see Figure 9 , Figure 9 The diagram shown is a current mapping schematic provided in an embodiment of this application.

[0121] The expression corresponding to the above current mapping is:

[0122]

[0123] in,

[0124] It should be noted that the above current mapping output These represent the clinical current values ​​of the stimulation pulses generated by the implanted electrodes numbered 1, 2, ..., M, respectively, in CL; when the value is 0, it indicates no stimulation output.

[0125] The current value output by the above current mapping These correspond to the output responses of the cochlear implant system to different frequency ranges, and the frequency response range should be increasing. Preferably, the implanted electrodes should be positioned as follows: Figure 10 The electrodes are arranged evenly as shown. In this case, the cochlear implant electrodes that are implanted deeper respond to low-frequency sounds, while the electrodes that are implanted shallower respond to high-frequency sounds, thus forming a correspondence with the frequency topology of the cochlea.

[0126] This application provides a method for data processing of sub-band output values. In this method, the sub-band output values ​​are compressed to obtain a sub-band compressed value. The sub-band compressed value is then mapped to obtain a current value, providing accurate current data for subsequently inputting the corresponding current value to the corresponding electrode implanted in the body and obtaining biphasic electrical stimulation pulses.

[0127] Figure 11 This is a flowchart illustrating the determination of a telemetry waveform according to an embodiment of this application. The telemetry waveform includes a first telemetry waveform and a second telemetry waveform. For example... Figure 11As shown, the process of determining the telemetry waveform in this embodiment includes the following steps S111 to S113.

[0128] S111, the first telemetry waveform is recorded by the recording electrode.

[0129] The recording electrode is any electrode other than the current telemetry electrode.

[0130] Specifically, the voltage drop between the recording electrode and ground is sampled using a set of programmable gain amplifiers (PGA) and analog-to-digital converters (ADC), and then transmitted back to the sound processor via radio frequency for recording.

[0131] For example, a programmable gain amplifier can provide a sampling gain of at least 40dB to 70dB.

[0132] For example, the sampling interval of the above ADC should not be higher than 50 μs, the sampling bit depth should be at least 10 bits, and the sampling length should be at least 32 points.

[0133] For example, the method for determining the electrode stimulation sequence is as follows: stimulation is first performed on the electrodes adjacent to the telemetry electrode, and then alternated in the order of "increase first, decrease later". Finally, stimulation is performed through the telemetry electrode, and a neural response telemetry is performed after the stimulation is completed. The first telemetry waveform is denoted by N.

[0134] The first telemetry waveform N includes neural response activity + stimulus artifacts + baseline offset.

[0135] It should be noted that the "increment-decrease" order means: for any j∈[1, 2, ..., M-1], stimulation begins with the electrode numbered j+1, proceeding in ascending order of j+1, ..., M; after the electrode numbered M has finished stimulating, stimulation begins again with the electrode numbered j-1, proceeding in descending order of j-1, ..., 1, until the electrode numbered 1 has finished stimulating. When the electrode numbered M is used as a telemetry electrode, stimulation begins with the electrode numbered M-1, proceeding in descending order of M-1, M-2, ..., 1, until the electrode numbered 1 has finished stimulating. This electrode stimulation sequence is as follows: Figure 12 As shown, its purpose is to reduce interference from electrical artifacts during neural response telemetry.

[0136] S112, based on the first biphasic electrical stimulation pulse corresponding to the first telemetry waveform, determine the second biphasic electrical stimulation pulse.

[0137] The first biphasic electrical stimulation pulse and the second biphasic electrical stimulation pulse have the same current amplitude but opposite phase.

[0138] Specifically, the positive pulse of the second biphasic electrical stimulation pulse comes first, followed by the negative pulse.

[0139] S113, based on the second biphasic electrical stimulation pulse, the nerve fibers in the cochlea are stimulated again through the telemetry electrode to obtain the second telemetry waveform recorded by the recording electrode.

[0140] Specifically, the second telemetry waveform can be represented by P.

[0141] The second telemetry waveform P includes neural response activity - stimulus artifacts + baseline offset.

[0142] This application provides a method for determining telemetry waveforms. In this method, a first telemetry waveform is recorded by recording electrodes, and a second biphasic electrical stimulation pulse is obtained based on the first biphasic electrical stimulation pulse corresponding to the first telemetry waveform. The first biphasic electrical stimulation pulse and the second biphasic electrical stimulation pulse have the same current amplitude but opposite phase. Based on the second biphasic electrical stimulation pulse, the nerve fibers in the cochlea are stimulated again by telemetry electrodes to obtain the second telemetry waveform recorded by the recording electrodes. For a set of stimulation pulses with the same amplitude and opposite phase, the resulting neural responses are usually very similar. Both the first and second telemetry waveforms obtained include neural response activity, stimulation artifacts, and baseline deviation, providing an accurate waveform basis for subsequently obtaining accurate neural response signal waveforms corresponding to the neural response activity.

[0143] Please see Figure 13 , Figure 13 The flowchart shown is for another sound signal processing method based on closed-loop control provided in an embodiment of this application. Figure 13 It can be seen that the sound signal processing method based on closed-loop control also includes initializing system parameters. These initialized system parameters include initial mapping parameters T0 and C0, and initial compression parameters X. min X thrd X max The parameter update rate α, the amplifier offset baseline D, and the ECAP voltage when the stimulation current amplitude is C0.

[0144] It should be noted that T0 and C0 are different for each electrode channel.

[0145] It should be noted that this application does not limit the specific value of the parameter update rate α. In practice, this value can be any value within the range of (0, 1). However, if the value is too large, the implantee's T value will change too quickly, which is not conducive to the implantee adapting to this change; if the value is too small, the change in T value will not be significant enough to achieve the effect of adaptive regulation. In practical applications, an appropriate value of α can be selected according to the specific application requirements, and this application does not impose any restrictions on this.

[0146] For example, the method for measuring the amplifier offset baseline D is as follows: telemetry is performed on the neural response without stimulation, and its waveform is recorded. After repeating this several times, the above waveforms are superimposed and their average waveform is calculated. The average waveform is the amplifier offset baseline D.

[0147] For example, the initial compression parameter X max The calculation method is as follows: Play a single-frequency pure tone with a frequency of 1000Hz and adjust the volume until the sound pressure level at the microphone of the sound processor is measured to be 105dB(A)SPL. Check the maximum amplitude of the system input time domain signal at this time. This amplitude is the compression parameter X. max The value of .

[0148] Furthermore, the volume was adjusted to make the sound pressure level at the sound processor microphone 28dB(A)SPL and 65dB(A)SPL respectively, and the maximum amplitude of the system input time-domain signal was recorded sequentially, which were respectively the compression parameter X. min X thrd The value of .

[0149] The specific implementation process in the remaining steps is the same as that shown in Figures 1 to 14 above. Figure 12 The corresponding steps are similar, and will not be repeated here.

[0150] The scope of protection of the sound signal processing method based on closed-loop control in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0151] This application also provides a sound signal processing device based on closed-loop control. The sound signal processing device based on closed-loop control can implement the sound signal processing method based on closed-loop control of this application. However, the implementation device of the sound signal processing method based on closed-loop control of this application includes, but is not limited to, the structure of the sound signal processing device based on closed-loop control listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0152] like Figure 14As shown, in one embodiment, the sound signal processing device 140 based on closed-loop control of this application includes a signal acquisition module 141, a signal conversion module 142, a sub-band output value determination module 143, a current value determination module 144, a pulse determination module 145, a telemetry waveform determination module 146, a neural response signal determination module 147, an electrically evoked compound action potential voltage determination module 148, and an electro-auditory dynamic range determination module 149.

[0153] The signal acquisition module 141 is used to acquire time-domain sound signals.

[0154] The signal conversion module 142 is used to convert the time-domain sound signal into a frequency-domain sound signal and obtain the energy spectrum corresponding to the frequency-domain sound signal.

[0155] The sub-band output value determination module 143 is used to allocate the energy spectrum to a preset number of sub-bands within the cochlear implant electrode, and sequentially perform pooling operations on each sub-band to obtain the sub-band output value corresponding to each sub-band.

[0156] The current value determination module 144 is used to process the sub-frequency band output value to obtain the current value corresponding to the sub-frequency band output value.

[0157] The pulse determination module 145 is used to input the current value into the corresponding electrode implanted in the body to obtain biphasic electrical stimulation pulses.

[0158] The telemetry waveform determination module 146 is used to traverse the cochlear nerve fibers based on the biphasic electrical stimulation pulse and in accordance with the preset electrode stimulation sequence, stimulate the cochlear nerve fibers to generate nerve response action potentials, and telemetry the nerve response action potentials generated by the telemetry electrodes after each traversal to obtain the telemetry waveform.

[0159] The neural response signal determination module 147 is used to remove electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform.

[0160] The electrically evoked compound action potential voltage determination module 148 is used to obtain the electrically evoked compound action potential voltage based on the minimum and maximum values ​​in the neural response signal.

[0161] The electroauditory dynamic range determination module 149 is used to update the maximum clinical current value of the implant based on the electrically evoked compound action potential voltage, and adjust the electroauditory dynamic range of the implant using the updated maximum clinical current value.

[0162] The structure and principle of the signal acquisition module 141, signal conversion module 142, sub-frequency band output value determination module 143, current value determination module 144, pulse determination module 145, telemetry waveform determination module 146, neural response signal determination module 147, electrically evoked compound action potential voltage determination module 148, and electro-auditory dynamic range determination module 149 correspond one-to-one with the steps in the above-mentioned sound signal processing method based on closed-loop control, and therefore will not be described in detail here.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed apparatus or method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0164] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0165] Those skilled in the art will further recognize that the units and steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] This application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0167] This application also provides an electronic device. Figure 15 The diagram shown is a structural schematic of an electronic device 150 in one embodiment of this application. The closed-loop control-based audio signal processing method provided in this embodiment can be applied to… Figure 15 The electronic device shown is 150, but it is not limited to this. For example... Figure 15 As shown, the electronic device 150 includes a processor 151, a memory, a system bus 153, and a network interface 155. The memory may include a non-volatile storage medium 152 and internal memory 154.

[0168] The non-volatile storage medium 152 can store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform any of the closed-loop control-based sound signal processing methods provided in the embodiments of this application.

[0169] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0170] The internal memory 154 provides an environment for the execution of a computer program in a non-volatile storage medium. When the computer program is executed by the processor, it can cause the processor to execute any of the closed-loop control-based sound signal processing methods provided in the embodiments of this application.

[0171] This network interface 155 is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 1AThe structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0172] It should be understood that processor 151 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, the general-purpose processor can be a microprocessor or any conventional processor.

[0173] The electronic device 150 in this application embodiment may include terminal devices such as tablet computers, laptop computers, mobile phones, supercomputers, and smart wearable devices. It can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. This application embodiment does not impose any restrictions on the specific type of electronic device.

[0174] For example, electronic devices can be stations (STAION, ST) in WLANs, cellular phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, computers, laptops, handheld communication devices, handheld computing devices, and / or other devices for communicating over wireless systems, as well as next-generation communication systems, such as mobile terminals in 5G networks, mobile terminals in future evolved Public Land Mobile Networks (PLMNs), or mobile terminals in future evolved Non-terrestrial Networks (NTNs).

[0175] As an example, and not a limitation, when an electronic device is a wearable device, the term can also refer to any device that utilizes wearable technology to intelligently design and develop everyday wearables, such as gloves and watches equipped with near-field communication modules. Wearable devices are portable devices worn directly on the body or integrated into a user's clothing or accessories. By attaching to the user and using a pre-linked electronic card, they perform operations such as payment and authentication. Wearable devices are not merely hardware devices; they achieve powerful functions through software support, data interaction, and cloud interaction. Broadly defined, wearable smart devices include those with comprehensive functions, large sizes, and the ability to perform complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those focused on a specific application function that require interaction with other devices such as smartphones, such as various smartwatches and smart bracelets with displays.

[0176] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0177] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A sound signal processing method based on closed-loop control, characterized in that, The method includes: Acquire time-domain sound signals; The time-domain sound signal is converted into a frequency-domain sound signal, and the energy spectrum corresponding to the frequency-domain sound signal is obtained; The energy spectrum is distributed to a preset number of sub-bands within the cochlear implant electrode, and a pooling operation is performed on each sub-band in sequence to obtain the sub-band output value corresponding to each sub-band. The sub-frequency band output value is processed to obtain the current value corresponding to the sub-frequency band output value; The current values ​​are input into the corresponding electrodes implanted in the body to obtain biphasic electrical stimulation pulses; Based on the biphasic electrical stimulation pulse, the cochlear nerve fibers are traversed according to the preset electrode stimulation sequence to stimulate the cochlear nerve fibers and generate neural action potentials. After each traversal, the neural action potentials generated by the telemetry electrodes are telemetryd to obtain telemetry waveforms. Remove electrical stimulation artifacts from the telemetry waveform to obtain the neural response signal corresponding to the telemetry waveform; Based on the minimum and maximum values ​​in the neural response signal, the voltage of the electrically evoked compound action potential is obtained; The maximum clinical current value of the implant is updated based on the voltage of the electrically evoked compound action potential, and the electroauditory dynamic range of the implant is adjusted by the updated maximum clinical current value.

2. The sound signal processing method based on closed-loop control according to claim 1, characterized in that, The step of processing the sub-frequency band output value to obtain the current value corresponding to the sub-frequency band output value includes: The sub-band output value is compressed to obtain the sub-band compressed value corresponding to the sub-band output value. The sub-band compression value is mapped to obtain the current value corresponding to the sub-band compression value.

3. The sound signal processing method based on closed-loop control according to claim 2, characterized in that, The expression for the sub-band compressed value corresponding to the sub-band output value obtained by performing compression operation on the sub-band output value is as follows: in, E represents the subband compression value. j X represents the output value of the j-th sub-band. min X thrd X max These represent the initial compression parameters.

4. The sound signal processing method based on closed-loop control according to claim 2, characterized in that, The mapping process of the sub-band compression value to obtain the current value corresponding to the sub-band compression value includes: Logarithmic processing is performed on the sub-band compression value to obtain the sub-band logarithmic value corresponding to the sub-band compression value; Between the maximum clinical current value and the minimum clinical current value of the implant, the logarithmic value of the sub-band is linearly scaled to obtain the current value corresponding to the logarithmic value of the sub-band.

5. The sound signal processing method based on closed-loop control according to claim 1, characterized in that, The method further includes: the telemetry electrode is one of the electrodes implanted in the body.

6. The sound signal processing method based on closed-loop control according to claim 1, characterized in that, The telemetry waveform includes a first telemetry waveform and a second telemetry waveform, and the method for determining the telemetry waveform includes: The first telemetry waveform is recorded by a recording electrode, which is an electrode other than the current telemetry electrode. Based on the first biphasic electrical stimulation pulse corresponding to the first telemetry waveform, a second biphasic electrical stimulation pulse is determined, wherein the first biphasic electrical stimulation pulse and the second biphasic electrical stimulation pulse have the same current amplitude but opposite phase. Based on the second biphasic electrical stimulation pulse, the nerve fibers in the cochlea are stimulated again through the telemetry electrode to obtain the second telemetry waveform recorded by the recording electrode.

7. The sound signal processing method based on closed-loop control according to claim 6, characterized in that, The expression for the neural response signal corresponding to the telemetry waveform obtained by removing electrical stimulation artifacts from the telemetry waveform is as follows: S=(N+P) / 2-D Where S represents the neural response signal, N represents the first telemetry waveform, P represents the second telemetry waveform, and D represents the amplifier offset baseline.

8. A sound signal processing device based on closed-loop control, characterized in that, The device includes: The signal acquisition module is used to acquire time-domain sound signals; The signal conversion module is used to convert the time-domain sound signal into a frequency-domain sound signal and obtain the energy spectrum corresponding to the frequency-domain sound signal. The sub-band output value determination module is used to allocate the energy spectrum to a preset number of sub-bands within the cochlear implant electrode, and sequentially perform a pooling operation on each sub-band to obtain the sub-band output value corresponding to each sub-band. The current value determination module is used to process the sub-frequency band output value to obtain the current value corresponding to the sub-frequency band output value. The pulse determination module is used to input the current value into the corresponding electrodes implanted in the body to obtain biphasic electrical stimulation pulses; The telemetry waveform determination module is used to traverse the cochlear nerve fibers according to the preset electrode stimulation sequence based on the biphasic electrical stimulation pulse, stimulate the cochlear nerve fibers to generate nerve response action potentials, and telemetry the nerve response action potentials generated by the telemetry electrodes after each traversal to obtain the telemetry waveform. A neural response signal determination module is used to remove electrical stimulation artifacts from the telemetry waveform and obtain the neural response signal corresponding to the telemetry waveform. The electrically evoked compound action potential voltage determination module is used to obtain the electrically evoked compound action potential voltage based on the minimum and maximum values ​​in the neural response signal; The electro-auditory dynamic range determination module is used to update the maximum clinical current value of the implant based on the voltage of the electrically evoked compound action potential, and to adjust the electro-auditory dynamic range of the implant using the updated maximum clinical current value.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the sound signal processing method based on closed-loop control as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes: A memory that stores a computer program; The processor, which is communicatively connected to the memory, executes the sound signal processing method based on closed-loop control as described in any one of claims 1 to 7 when calling the computer program.

Images