Audio data processing method for personal hearing aid product and personal hearing aid product

Abstract

The invention relates to an audio data processing method for a personal hearing aid product and the personal hearing aid product, and relates to the technical field of audio data processing, and the method comprises the following steps: a self-adaptive feedback canceller cancels acoustic feedback in an audio signal; the self-adaptive wave trap suppresses a howling signal in the audio signal; the conversion module is used for converting an audio signal acquired by the microphone into a time-frequency signal; the multi-sub-band dynamic range compressor adjusts the dynamic range of the time-frequency signal to obtain the gain of a frequency point; the filter conversion module calculates a filter coefficient based on the gain of the frequency point; the time domain filter performs time domain filtering on the audio signal output by the self-adaptive wave trap based on a filter coefficient, the equalizer adjusts the frequency response of the audio signal obtained by filtering, and the amplitude limiter limits the amplitude of the audio signal to be within a preset range; the digital-to-analog converter converts the audio signal into an electric signal; and the loudspeaker converts the electric signal into sound for playing. According to the invention, the delay of the PSAP can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of audio data processing, and more specifically, to an audio data processing method for a personal listening assistant product and a personal listening assistant product. Background Art

[0002] As hearing loss becomes more common, the personal sound amplification products (PSAP) market has ushered in unprecedented development opportunities. These products, such as smart hearing aids and hearing-aid headphones, are designed to provide hearing-impaired users with a clearer and more accurate hearing experience through advanced audio processing technology.

[0003] Existing methods usually process audio data in the frequency domain, such as dynamic range compression, noise suppression, and acoustic feedback elimination in the frequency domain.

[0004] However, when processing audio data in the frequency domain, it is generally necessary to perform fast Fourier transform after framing, and in order to achieve the desired effect, the frequency resolution must be high enough, that is, the frequency band interval must be small enough. Assuming the frame length is Tf, the frequency band interval is 1 / Tf. For example, to achieve a frequency resolution of 125Hz, the frame length needs to be 8ms, which will cause the PSAP to have a higher delay. Summary of the invention

[0005] The present application is provided to solve the above problems existing in the prior art. According to an audio data processing method for a personal listening assistant product and a personal listening assistant product according to an embodiment of the present application, the delay of PSAP can be reduced.

[0006] In a first aspect, the present application provides an audio data processing method for a personal hearing aid product, the personal hearing aid product comprising: a microphone, an adaptive feedback eliminator, an adaptive notch filter, a transformation module, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter and a speaker; the method comprises:

[0007] The adaptive feedback canceller cancels acoustic feedback in the audio signal collected by the microphone;

[0008] The adaptive notch filter suppresses a howling signal in the audio signal output by the adaptive feedback canceller;

[0009] The conversion module converts the audio signal collected by the microphone into a time-frequency signal;

[0010] The multi-subband dynamic range compressor performs dynamic range adjustment on the time-frequency signal to obtain a gain of a frequency point;

[0011] The filter conversion module calculates the filter coefficient based on the gain of the frequency point;

[0012] The time domain filter performs time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficient;

[0013] The equalizer adjusts the frequency response of the audio signal after time domain filtering;

[0014] The limiter limits the amplitude of the audio signal output by the equalizer to within a preset range;

[0015] The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal;

[0016] The speaker converts the electrical signal into sound for playback.

[0017] In a second aspect, an embodiment of the present application provides a personal hearing aid product, including: a microphone, an adaptive feedback canceller, an adaptive notch filter, a transformation module, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker;

[0018] The adaptive feedback canceller is used to cancel the acoustic feedback in the audio signal collected by the microphone;

[0019] The adaptive notch filter is used to suppress the howling signal in the audio signal output by the adaptive feedback canceller;

[0020] The conversion module is used to convert the audio signal collected by the microphone into a time-frequency signal;

[0021] The multi-subband dynamic range compressor is used to adjust the dynamic range of the time-frequency signal to obtain the gain of the frequency point;

[0022] The filter conversion module is used to calculate the filter coefficient based on the gain of the frequency point;

[0023] The time domain filter is used to perform time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficient;

[0024] The equalizer is used to adjust the frequency response of the audio signal after time domain filtering;

[0025] The limiter is used to limit the amplitude of the audio signal output by the equalizer to within a preset range;

[0026] The digital-to-analog converter is used to convert the audio signal processed by the limiter into an electrical signal;

[0027] The speaker is used to convert the electrical signal into sound for playback.

[0028] The beneficial effect of the embodiments of the present application is that the processing process does not require frame processing of the audio signal, and modules such as the adaptive feedback eliminator and the adaptive notch filter all process the audio data in the time domain, reducing the delay caused by frequency domain processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar parts. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the specification and claims, are used to illustrate the disclosed embodiments. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0030] Figure 1 It is a schematic diagram of a personal hearing aid product provided by the prior art;

[0031] Figure 2 is a flowchart of an audio data processing method for a personal auxiliary listening product provided by an embodiment of the present application;

[0032] Figure 3 is a schematic diagram of a personal hearing aid product provided by an embodiment of the present application;

[0033] Figure 4 It is a schematic diagram of a personal hearing aid product provided by another embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present application. The terms "first", "second" and "third" used in the present application are only intended to distinguish the corresponding features, do not represent the need for such an order, and do not necessarily represent only the singular form.

[0035] like Figure 1The figure shows a schematic diagram of a personal hearing aid product provided by the prior art. The dual microphone Dual MIC collects audio signals, which are transformed to the frequency domain by the STFT (Short-Time Fourier Transform) module, and then input into the processing module for beamforming processing, and then processed by the adaptive feedback eliminator, adaptive notch filter, noise estimator, and multi-subband dynamic range compressor in sequence. The processed audio signal is transformed from the frequency domain signal to the time domain signal by the inverse short-time Fourier transform (ISTFT) module, and then processed by the equalizer, limiter, and digital-to-analog converter before being played by the speaker.

[0036] Among them, beamforming processing, adaptive feedback eliminator, adaptive notch filter, and noise estimator are all processed in the frequency domain. The frequency resolution of the frequency domain transformation is the inverse of the frequency band interval. For example, to achieve a frequency resolution of 250Hz, it is necessary to perform a short-time Fourier transform on the 4ms time domain data. This transformation increases the data processing delay.

[0037] In view of this, if Figure 2 As shown, an embodiment of the present invention provides an audio data processing method for a personal auxiliary listening product, the personal auxiliary listening product comprising: a microphone, an adaptive feedback eliminator, an adaptive notch filter, a transformation module, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter and a speaker; the method comprises:

[0038] Step 201: The adaptive feedback canceller cancels the acoustic feedback in the audio signal collected by the microphone.

[0039] An adaptive feedback canceller is a device that can automatically detect and eliminate acoustic feedback. It adaptively estimates the path response parameters of the feedback path of the feedback acoustic signal, uses the path response parameters to filter the signal before the speaker plays, generates a set of feedback sound estimation signals that are identical to the feedback sound signal, and subtracts the feedback sound estimation signals from the original audio signal collected by the microphone, thereby achieving the purpose of automatically eliminating acoustic feedback. The number of microphones is not limited here, and can be single, two, or other numbers.

[0040] Step 202: The adaptive notch filter suppresses the howling signal in the audio signal output by the adaptive feedback canceller.

[0041] The working principle of the adaptive notch filter is based on adaptive filtering algorithms, such as the minimum mean square error algorithm, etc. By analyzing the audio signal, the audio signal frequency that produces howling (i.e. howling frequency) is quickly found, and a set of narrowband filters with the same frequency as these howling frequencies are automatically generated to cut off or attenuate these frequency signals, thereby achieving the purpose of automatic notching and suppressing howling.

[0042] Step 203: The conversion module converts the audio signal collected by the microphone into a time-frequency signal.

[0043] The transformation module converts the audio signal into a time-frequency signal through short-time Fourier transform. Specifically, the transformation module divides the audio signal into multiple short-time windows in the time domain and assumes that the signal is stable in each window. Then, the signal in each window is Fourier transformed to obtain the spectrum information of the time period. By moving the window and repeating the above process, the spectrum representation of the signal at different time points can be obtained to form a time-frequency diagram.

[0044] Step 204: The multi-subband dynamic range compressor performs dynamic range adjustment on the time-frequency signal to obtain a gain of a frequency point.

[0045] The Multi-Subband Dynamic Range Compressor splits the frequency spectrum of an audio signal into multiple sub-bands and applies dynamic range adjustment independently on each sub-band. This means that each sub-band can have its gain adjusted independently based on its specific sound characteristics, allowing for more precise and detailed sound control.

[0046] Step 205: The filter conversion module calculates the filter coefficient based on the gain of the frequency point.

[0047] Step 206: The time domain filter performs time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients.

[0048] The time domain filter may include a FIR filter and an IIR filter, among others.

[0049] Step 207: The equalizer adjusts the frequency response of the audio signal after time domain filtering.

[0050] An equalizer is an effector that can adjust the size of different frequency signals separately. Its main function is to adjust the frequency response of audio signals, improve sound quality, eliminate unnecessary noise or enhance specific audio characteristics.

[0051] The working principle of an equalizer is based on the processing of different frequency components of an audio signal. It usually consists of multiple frequency bands, each of which can be adjusted independently. Users can enhance or attenuate specific frequencies as needed to achieve the desired sound quality effect. This adjustment is achieved by changing the gain of different frequency components without affecting the signals in other frequency bands.

[0052] Step 208: The limiter limits the amplitude of the audio signal output by the equalizer to within a preset range.

[0053] A limiter is a circuit that can flatten the signal voltage amplitude within a limited range, also known as a clipper. Its main function is to limit the output signal amplitude within a certain range. When the input voltage exceeds or falls below a certain reference value, the output voltage will be limited to a certain level (called the limit level) and will no longer change with the input voltage.

[0054] Step 209: The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal.

[0055] Step 210: The speaker converts the electrical signal into sound for playback.

[0056] In the embodiment of the present application, the processing process does not need to frame the audio signal, and modules such as the adaptive feedback eliminator and the adaptive notch filter process the audio data in the time domain, reducing the delay caused by frequency domain processing.

[0057] In one embodiment of the present application, in order to suppress noise and further improve the quality of sound output by the speaker, the personal hearing aid product further includes: a noise estimator;

[0058] Before the filter conversion module calculates the filter coefficient based on the gain of the frequency point, the method further includes: a noise estimator performs noise estimation on the time-frequency signal to obtain the gain of the frequency point;

[0059] The filter conversion module calculates the filter coefficients based on the gain of the frequency point, including:

[0060] The filter conversion module fuses the gains of the frequency points output by the multi-subband dynamic range compressor and the gains of the frequency points output by the noise estimator in the frequency domain, and calculates the filter coefficients based on the gains of the fused frequency points.

[0061] Noise estimators are devices that can estimate the intensity of noise howling at each frequency point. They are widely used in various audio devices, mobile phones, and radio communication systems. Gain refers to the attenuation multiple of a signal at a certain frequency point, which reflects the intensity or energy of the noise at that frequency point. In the noise estimation process, the calculation of gain is usually based on the intensity estimation of the noise signal.

[0062] In one embodiment of the present application, the filter coefficient is calculated based on the gain of the fused frequency point, including:

[0063] Based on the gains of the fused frequency points, the filter coefficients of the FIR filter are calculated by inverse fast Fourier transform.

[0064] The fused frequency gain is used as the input of the inverse fast Fourier transform, and the time domain coefficients of the FIR filter are calculated through the inverse fast Fourier transform algorithm. These coefficients will be directly used in the implementation of the FIR filter.

[0065] FIR filter, finite impulse response filter, also known as non-recursive filter, is a linear filter whose output signal is the result of linear convolution of the input signal with a finite length sequence of the filter's unit impulse response.

[0066] In one embodiment of the present application, the filter coefficient is calculated based on the gain of the fused frequency point, including:

[0067] Based on the gains of the fused frequency points, the filter coefficients of the IIR filter are calculated through a filter parameter space search algorithm.

[0068] IIR filter, that is, infinite impulse response filter (Infinite Impulse Response). Filter parameter space search algorithms include grid search, random search, genetic algorithm, particle swarm optimization, etc.

[0069] The above two embodiments adopt different algorithms to calculate filter coefficients for different filters, and can be applicable to different filtering requirements.

[0070] In one embodiment of the present application, the personal auxiliary listening product further includes: a processing module;

[0071] Before the adaptive feedback canceller cancels the acoustic feedback in the audio signal collected by the microphone, the method further comprises:

[0072] The processing module performs beamforming processing on the audio signal collected by the microphone;

[0073] The adaptive feedback canceller eliminates acoustic feedback in the audio signal collected by the microphone, including:

[0074] The adaptive feedback canceller cancels the acoustic feedback in the audio signal after the beamforming process.

[0075] Beamforming, also known as spatial filtering or beamforming, is a signal processing technology that uses sensor arrays (such as antenna arrays, microphone arrays, etc.) to send and receive signals in a direction. In the beamforming process, the perceived direction of the signal is enhanced or attenuated by adjusting the weights and delays of each unit in the receiver (such as microphone arrays, camera arrays). The processing module can enhance the audio signal in a specific direction while suppressing noise and howling from other directions.

[0076] like Figure 3 As shown, an embodiment of the present application provides a personal hearing aid product, including: a microphone, an adaptive feedback eliminator, an adaptive notch filter, a transformation module, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter and a speaker.

[0077] An adaptive feedback canceller for canceling acoustic feedback in the audio signal collected by the microphone;

[0078] An adaptive notch filter, used for suppressing a howling signal in an audio signal output by an adaptive feedback canceller;

[0079] A conversion module, used to convert the audio signal collected by the microphone into a time-frequency signal;

[0080] A multi-subband dynamic range compressor is used to adjust the dynamic range of the time-frequency signal to obtain the gain of the frequency point;

[0081] A filter conversion module is used to calculate the filter coefficient based on the gain of the frequency point;

[0082] A time domain filter, used for performing time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficient;

[0083] an equalizer for adjusting the frequency response of the time-domain filtered audio signal;

[0084] A limiter, used to limit the amplitude of the audio signal output by the equalizer to within a preset range;

[0085] A digital-to-analog converter, used for converting the audio signal processed by the limiter into an electrical signal;

[0086] Speakers are used to convert electrical signals into sound.

[0087] In one embodiment of the present application, the personal hearing aid product further includes: a noise estimator;

[0088] A noise estimator is used to estimate the noise of the time-frequency signal to obtain the gain of the frequency point before the filter conversion module calculates the filter coefficient based on the gain of the frequency point;

[0089] The filter conversion module is used to fuse the gains of the frequency points output by the multi-subband dynamic range compressor and the gains of the frequency points output by the noise estimator in the frequency domain, and calculate the filter coefficients based on the gains of the fused frequency points.

[0090] In one embodiment of the present application, the filter conversion module is used to calculate the filter coefficients of the FIR filter through an inverse fast Fourier transform based on the gains of the fused frequency points.

[0091] In one embodiment of the present application, the filter conversion module is used to calculate the filter coefficients of the IIR filter through a filter parameter space search algorithm based on the gains of the fused frequency points.

[0092] In one embodiment of the present application, the personal auxiliary listening product further includes: a processing module;

[0093] A processing module, configured to perform beamforming processing on the audio signal collected by the microphone before the adaptive feedback canceller cancels the acoustic feedback in the audio signal collected by the microphone;

[0094] The adaptive feedback canceller is used to cancel the acoustic feedback in the audio signal after the beamforming processing.

[0095] like Figure 4 As shown, an embodiment of the present application provides a personal hearing aid product, including: a microphone, a processing module, an adaptive feedback eliminator, an adaptive notch filter, a transformation module, a noise estimator, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter and a speaker.

[0096] The personal hearing aid product provided in the embodiment of the present application converts audio data from the frequency domain into the time domain for processing, thereby eliminating the delay caused by the frequency domain processing and making the visual effect of the personal hearing aid product more natural.

[0097] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. Although several embodiments are listed and described for the wireless communication method and the wireless communication component, it should be noted that the method details involved in the description of the wireless communication component can also be incorporated into the various embodiments of the wireless communication method, and vice versa.

[0098] The elements in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of the application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the claims and the full scope of their equivalents.

[0099] The order of each step in the present application is merely exemplary and not restrictive. Without affecting the implementation of the present application (without destroying the required logical relationship between the steps), the execution order of the steps can be adjusted, and the various embodiments obtained after the adjustment still fall within the scope of the present application.

[0100] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of a specific disclosed embodiment. Thus, the claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

Claims

1. An audio data processing method for a personal listening aid product, characterized in that: The personal hearing aid product includes: a microphone, an adaptive feedback eliminator, an adaptive notch filter, a transform module, a multi-subband dynamic range compressor, a filter conversion module, a time domain filter, an equalizer, a limiter, a digital-to-analog converter and a speaker; the method includes: The adaptive feedback canceller cancels acoustic feedback in the audio signal collected by the microphone; The adaptive notch filter suppresses a howling signal in the audio signal output by the adaptive feedback canceller; The conversion module converts the audio signal collected by the microphone into a time-frequency signal; The multi-subband dynamic range compressor performs dynamic range adjustment on the time-frequency signal to obtain a gain of a frequency point; The filter conversion module calculates the filter coefficient based on the gain of the frequency point; The time domain filter performs time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficient; The equalizer adjusts the frequency response of the audio signal after time domain filtering; The limiter limits the amplitude of the audio signal output by the equalizer to within a preset range; The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal; The speaker converts the electrical signal into sound for playback.

2. The method according to claim 1, characterized in that The personal hearing aid product further includes: a noise estimator; Before the filter conversion module calculates the filter coefficient based on the gain of the frequency point, the method further comprises: the noise estimator performs noise suppression on the time-frequency signal to obtain the gain of the frequency point; The filter conversion module calculates the filter coefficient based on the gain of the frequency point, including: The filter conversion module fuses the gains of the frequency points output by the multi-subband dynamic range compressor and the gains of the frequency points output by the noise estimator in the frequency domain, and calculates filter coefficients based on the gains of the fused frequency points.

3. The method according to claim 2, characterized in that The filter coefficients are calculated based on the gains of the fused frequency points, including: Based on the gains of the fused frequency points, the filter coefficients of the FIR filter are calculated by inverse fast Fourier transform.

4. The method according to claim 2, characterized in that The filter coefficients are calculated based on the gains of the fused frequency points, including: Based on the gains of the fused frequency points, the filter coefficients of the IIR filter are calculated through a filter parameter space search algorithm.

5. The method according to claim 1, characterized in that The personal hearing aid product further includes: a processing module; Before the adaptive feedback canceller cancels the acoustic feedback in the audio signal collected by the microphone, the method further comprises: The processing module performs beamforming processing on the audio signal collected by the microphone; The adaptive feedback canceller cancels acoustic feedback in the audio signal collected by the microphone, including: The adaptive feedback canceller cancels acoustic feedback in the audio signal that has been processed by beamforming.

6. A personal hearing aid product, characterized in that: include: microphone, adaptive feedback canceller, adaptive notch filter, transform module, multi-subband dynamic range compressor, filter conversion module, time domain filter, equalizer, limiter, digital-to-analog converter and speaker; The adaptive feedback canceller is used to cancel the acoustic feedback in the audio signal collected by the microphone; The adaptive notch filter is used to suppress the howling signal in the audio signal output by the adaptive feedback canceller; The conversion module is used to convert the audio signal collected by the microphone into a time-frequency signal; The multi-subband dynamic range compressor is used to adjust the dynamic range of the time-frequency signal to obtain the gain of the frequency point; The filter conversion module is used to calculate the filter coefficient based on the gain of the frequency point; The time domain filter is used to perform time domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficient; The equalizer is used to adjust the frequency response of the audio signal after time domain filtering; The limiter is used to limit the amplitude of the audio signal output by the equalizer to within a preset range; The digital-to-analog converter is used to convert the audio signal processed by the limiter into an electrical signal; The speaker is used to convert the electrical signal into sound for playback.

7. The personal hearing aid product according to claim 6, characterized in that: Also includes: Noise estimator; The noise estimator is used to perform noise estimation on the time-frequency signal to obtain the gain of the frequency point before the filter conversion module calculates the filter coefficient based on the gain of the frequency point; The filter conversion module is used to fuse the gains of the frequency points output by the multi-subband dynamic range compressor and the gains of the frequency points output by the noise estimator in the frequency domain, and calculate the filter coefficients based on the gains of the fused frequency points.

8. The personal hearing aid product according to claim 7, characterized in that: The filter conversion module is used to calculate the filter coefficients of the FIR filter through inverse fast Fourier transform based on the gains of the fused frequency points.

9. The personal hearing aid product according to claim 7, characterized in that: The filter conversion module is used to calculate the filter coefficients of the IIR filter through a filter parameter space search algorithm based on the gains of the fused frequency points.

10. The personal hearing aid product according to claim 6, characterized in that: Also includes: Processing module; The processing module is used to perform beamforming processing on the audio signal collected by the microphone before the adaptive feedback canceller eliminates the acoustic feedback in the audio signal collected by the microphone; The adaptive feedback canceller is used to cancel the acoustic feedback in the audio signal that has been processed by beamforming.

Images