Audio data processing method for personal sound amplification product, and personal sound amplification product

Abstract

The present application relates to the technical field of audio data processing, and relates to an audio data processing method for a personal sound amplification product, and a personal sound amplification product. The method comprises: an adaptive feedback canceller eliminates acoustic feedback in an audio signal; an adaptive notch filter suppresses a howling signal in the audio signal; a transform module converts an audio signal acquired by a microphone into a time-frequency signal; a multi-subband dynamic range compressor performs dynamic range adjustment on the time-frequency signal to obtain gains of frequency points; a filter conversion module calculates filter coefficients on the basis of the gains of the frequency points; a time domain filter performs, on the basis of the filter coefficients, time domain filtering on an audio signal outputted by the adaptive notch filter; an equalizer adjusts a frequency response of the filtered audio signal; an amplitude limiter limits an amplitude of the obtained audio signal to be within a preset range; a digital-to-analog converter converts the audio signal into an electrical signal; and a speaker converts the electrical signal into sound for playback. The present application can reduce PSAP latency.

Description

Audio data processing method for personal hearing aids and personal hearing aids Technical Field

[0001] This application relates to the field of audio data processing technology, and more specifically, to an audio data processing method for personal hearing aids and the personal hearing aid itself. Background Technology

[0002] With the increasing prevalence of hearing loss, the market for Personal Sound Amplification Products (PSAPs) has ushered in unprecedented development opportunities. These products, such as smart hearing aids and hearing headphones, are designed to provide hearing-impaired users with a clearer and more accurate auditory experience through advanced audio processing technology.

[0003] Existing methods typically process audio data in the frequency domain, such as dynamic range compression, noise suppression, and acoustic feedback cancellation.

[0004] However, when processing audio data in the frequency domain, it is generally necessary to perform a fast Fourier transform after dividing the data into frames. In order to achieve the desired effect, the frequency resolution must be high enough, that is, the frequency band spacing must be small enough. Assuming the frame length is Tf, the frequency band spacing is 1 / Tf. For example, to achieve a frequency resolution of 125Hz, the frame length needs to be 8ms, which will result in a high latency for PSAP. Summary of the Invention

[0005] This application is provided to address the aforementioned problems existing in the prior art. An audio data processing method for a personal hearing aid product and a personal hearing aid product according to embodiments of this application can reduce PSAP latency.

[0006] In a first aspect, this application provides an audio data processing method for a personal hearing aid product, the personal hearing aid product including: a microphone, an adaptive feedback canceller, an adaptive notch filter, a conversion module, a multi-subband dynamic range compressor, a filter conversion module, a time-domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker; the method includes:

[0007] The adaptive feedback canceller eliminates acoustic feedback in the audio signal acquired by the microphone;

[0008] The adaptive notch filter suppresses the howling signal in the audio signal output by the adaptive feedback canceller;

[0009] The conversion module converts the audio signal collected by the microphone into a time-frequency signal;

[0010] The multi-subband dynamic range compressor dynamically adjusts the time-frequency signal to obtain the gain at the frequency point;

[0011] The filter conversion module calculates the filter coefficients based on the gain at the frequency point;

[0012] The time-domain filter performs time-domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients;

[0013] The equalizer adjusts the frequency response of the time-domain filtered audio signal;

[0014] The limiter restricts the amplitude of the audio signal output by the equalizer to a preset range;

[0015] The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal;

[0016] The speaker converts the electrical signal into sound for playback.

[0017] Secondly, embodiments of this application provide a personal hearing aid product, including: a microphone, an adaptive feedback canceller, an adaptive notch filter, a conversion module, a multi-subband dynamic range compressor, a filter conversion module, a time-domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker;

[0018] The adaptive feedback canceller is used to eliminate acoustic feedback in the audio signal collected by the microphone;

[0019] The adaptive notch filter is used to suppress howling signals in the audio signal output by the adaptive feedback canceller;

[0020] The conversion module is used to convert the audio signal collected by the microphone into a time-frequency signal;

[0021] The multi-subband dynamic range compressor is used to dynamically adjust the time-frequency signal to obtain the gain at the frequency point;

[0022] The filter conversion module is used to calculate the filter coefficients based on the gain at the frequency point;

[0023] The time-domain filter is used to perform time-domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients.

[0024] The equalizer is used to adjust the frequency response of the time-domain filtered audio signal;

[0025] The limiter is used to limit the amplitude of the audio signal output by the equalizer to a preset range;

[0026] The digital-to-analog converter is used to convert the audio signal processed by the limiter into an electrical signal;

[0027] The speaker is used to convert the electrical signal into sound for playback.

[0028] The beneficial effects of the embodiments of this application are as follows: the processing does not require frame-by-frame processing of the audio signal, and the adaptive feedback canceller, adaptive notch filter and other modules process the audio data in the time domain, which reduces the delay caused by frequency domain processing. Attached Figure Description

[0029] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with or without letter suffixes may indicate different instances of similar parts. The drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the description and claims, to explain the disclosed embodiments. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0030] Figure 1 is a schematic diagram of a personal hearing aid product provided by the prior art;

[0031] Figure 2 is a flowchart of an audio data processing method for a personal hearing aid product according to an embodiment of this application;

[0032] Figure 3 is a schematic diagram of a personal hearing aid product provided in an embodiment of this application;

[0033] Figure 4 is a schematic diagram of a personal hearing aid product provided in another embodiment of this application. Detailed Implementation

[0034] To enable those skilled in the art to better understand the technical solutions of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the application. The terms "first," "second," and "third" used in this application are merely intended to distinguish the corresponding features and do not imply a necessary order, nor do they necessarily represent only the singular form.

[0035] Figure 1 shows a schematic diagram of a personal hearing aid product provided by the prior art. Dual microphones acquire audio signals. These signals are transformed to the frequency domain by an STFT (Short-Time Fourier Transform) module, then input to a processing module for beamforming. The signals are then processed sequentially through an adaptive feedback canceller, an adaptive notch filter, a noise estimator, and a multi-subband dynamic range compressor. The processed audio signal is then transformed from a frequency domain signal to a time domain signal by an Inverse Short-Time Fourier Transform (ISTFT) module. After further processing by an equalizer, a limiter, and a digital-to-analog converter, the signal is finally played by a speaker.

[0036] Among them, beamforming, adaptive feedback canceller, adaptive notch filter, and noise estimator are all processed in the frequency domain. The frequency resolution of the frequency domain transformation is the reciprocal of the frequency band interval. For example, to achieve a frequency resolution of 250Hz, a short-time Fourier transform is required for 4ms of time domain data. This transformation leads to an increase in data processing delay.

[0037] Therefore, as shown in Figure 2, this embodiment of the invention provides an audio data processing method for a personal hearing aid product. The personal hearing aid product includes: a microphone, an adaptive feedback canceller, an adaptive notch filter, a conversion module, a multi-subband dynamic range compressor, a filter conversion module, a time-domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker; the method includes:

[0038] Step 201: The adaptive feedback canceller cancels acoustic feedback in the audio signal acquired by the microphone.

[0039] An adaptive feedback canceller is a device that automatically detects and eliminates acoustic feedback. It adaptively estimates the path response parameters of the feedback signal's path, filters the signal before playback from the speaker using these parameters, and generates a set of estimated feedback signals identical to the original feedback signal. This estimated feedback signal is then subtracted from the original audio signal captured by the microphone, thus automatically eliminating the acoustic feedback. The number of microphones is not limited here; it can be one, two, or any other number.

[0040] Step 202: The adaptive notch filter suppresses the howling signal in the audio signal output by the adaptive feedback canceller.

[0041] The working principle of an adaptive notch filter is based on adaptive filtering algorithms, such as the minimum mean square error algorithm. By analyzing the audio signal, it quickly identifies the audio signal frequency that generates howling (i.e., howling frequency) and automatically generates a set of narrowband filters with the same frequency to cut off or attenuate these frequency signals, thereby achieving automatic notch filtering and suppressing howling.

[0042] Step 203: The conversion module converts the audio signal collected by the microphone into a time-frequency signal.

[0043] The transformation module converts the audio signal into a time-frequency signal using a short-time Fourier transform. Specifically, the transformation module divides the audio signal into multiple short time windows in the time domain, assuming that the signal is stationary within each window. Then, a Fourier transform is performed on the signal within each window to obtain the spectral information for that time period. By moving the windows and repeating the above process, the spectral representation of the signal at different time points can be obtained, forming a time-frequency diagram.

[0044] Step 204: The multi-subband dynamic range compressor performs dynamic range adjustment on the time-frequency signal to obtain the gain at the frequency point.

[0045] Multi-subband dynamic range compressors divide the spectrum of an audio signal into multiple subbands and apply dynamic range adjustment independently to each subband. This means that each subband can be independently gain-adjusted according to its specific sonic characteristics, resulting in more precise and nuanced sound control.

[0046] Step 205: The filter conversion module calculates the filter coefficients based on the gain at the frequency point.

[0047] Step 206: Time-domain filter performs time-domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients.

[0048] Time-domain filters can include FIR filters and IIR filters, etc.

[0049] Step 207: The equalizer adjusts the frequency response of the time-domain filtered audio signal.

[0050] An equalizer is an effect that allows you to adjust the magnitude of signals at different frequencies. Its main function is to adjust the frequency response of audio signals, improve sound quality, eliminate unwanted noise, or enhance specific audio characteristics.

[0051] An equalizer works by processing different frequency components of an audio signal. It typically consists of multiple frequency bands, each with its gain adjustable independently. Users can boost or attenuate specific frequencies to achieve desired sound quality. This adjustment is achieved by changing the gain of different frequency components without affecting signals in other frequency bands.

[0052] Step 208: The limiter restricts the amplitude of the audio signal output by the equalizer to a preset range.

[0053] A limiter is a circuit that can flatten the amplitude of a signal voltage within a defined range; it is also called a clipper. Its main function is to limit the amplitude of the output signal within a certain range. When the input voltage exceeds or falls below a certain reference value, the output voltage will be limited to a certain level (called the limiting level) and will no longer change with the input voltage.

[0054] Step 209: The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal.

[0055] Step 210: The speaker converts the electrical signal into sound and plays it.

[0056] In this embodiment of the application, the processing does not require frame-by-frame processing of the audio signal, and the adaptive feedback canceller, adaptive notch filter and other modules process the audio data in the time domain, which reduces the delay caused by frequency domain processing.

[0057] In one embodiment of this application, in order to suppress noise and further improve the quality of the speaker output sound, the personal hearing aid product further includes: a noise estimator;

[0058] Before the filter conversion module calculates the filter coefficients based on the gain at the frequency point, the method further includes: a noise estimator performing noise estimation on the time-frequency signal to obtain the gain at the frequency point;

[0059] The filter conversion module calculates filter coefficients based on the gain at a frequency point, including:

[0060] The filter conversion module fuses the gain of the frequency point output by the multi-subband dynamic range compressor and the gain of the frequency point output by the noise estimator in the frequency domain, and calculates the filter coefficients based on the gain of the fused frequency point.

[0061] A noise estimator is a device that estimates the intensity of noise feedback at various frequencies and is widely used in various audio devices, mobile phones, and wireless communication systems. Gain refers to the attenuation factor of a signal at a certain frequency, reflecting the intensity or energy of noise at that frequency. In noise estimation, gain calculation is usually based on an estimate of the noise signal intensity.

[0062] In one embodiment of this application, calculating the filter coefficients based on the gain of the fused frequency points includes:

[0063] Based on the gain of the fused frequency points, the filter coefficients of the FIR filter are calculated using inverse fast Fourier transform.

[0064] The fused frequency gain is used as input to the inverse fast Fourier transform (IFF), and the time-domain coefficients of the FIR filter are calculated using the IFT algorithm. These coefficients will be directly used in the implementation of the FIR filter.

[0065] An FIR filter, or Finite Impulse Response filter, also known as a non-recursive filter, is a type of linear filter whose output signal is the result of linear convolution of the input signal with a finite-length sequence of the filter's unit impulse response.

[0066] In one embodiment of this application, calculating the filter coefficients based on the gain of the fused frequency points includes:

[0067] Based on the gain of the fused frequency points, the filter coefficients of the IIR filter are calculated using a filter parameter space search algorithm.

[0068] IIR filters, or Infinite Impulse Response filters, utilize various parameter space search algorithms, including grid search, random search, genetic algorithms, and particle swarm optimization.

[0069] The two embodiments described above employ different algorithms to calculate the filter coefficients for different filters, thus being applicable to various filtering requirements.

[0070] In one embodiment of this application, the personal hearing aid product further includes: a processing module;

[0071] Before the adaptive feedback canceller cancels acoustic feedback in the audio signal acquired by the microphone, the method further includes:

[0072] The processing module performs beamforming processing on the audio signals captured by the microphone;

[0073] An adaptive feedback canceller eliminates acoustic feedback in the audio signal captured by a microphone, including:

[0074] An adaptive feedback canceller eliminates acoustic feedback in beamformed audio signals.

[0075] Beamforming, also known as spatial filtering or beamforming, is a signal processing technique that uses sensor arrays (such as antenna arrays, microphone arrays, etc.) to transmit and receive signals in a specific direction. In beamforming, the weights and delays of individual units in a receiver (such as a microphone array or camera array) are adjusted to enhance or attenuate the perceived direction of the signal. Processing modules can enhance audio signals from a specific direction while suppressing noise and feedback from other directions.

[0076] As shown in Figure 3, this application provides a personal hearing aid product, including: a microphone, an adaptive feedback canceller, an adaptive notch filter, a conversion module, a multi-subband dynamic range compressor, a filter conversion module, a time-domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker.

[0077] An adaptive feedback canceller is used to eliminate acoustic feedback in the audio signal captured by a microphone.

[0078] An adaptive notch filter is used to suppress howling signals in the audio signal output by an adaptive feedback canceller.

[0079] The conversion module is used to convert the audio signals captured by the microphone into time-frequency signals;

[0080] Multi-subband dynamic range compressors are used to dynamically adjust the gain of time-frequency signals to obtain the gain at a frequency point.

[0081] The filter conversion module is used to calculate filter coefficients based on the gain at a frequency point;

[0082] A time-domain filter is used to perform time-domain filtering on the audio signal output by an adaptive notch filter based on the filter coefficients.

[0083] An equalizer is used to adjust the frequency response of a time-domain filtered audio signal.

[0084] A limiter is used to limit the amplitude of the audio signal output by an equalizer within a preset range.

[0085] A digital-to-analog converter is used to convert audio signals processed by a limiter into electrical signals;

[0086] A loudspeaker is used to convert electrical signals into sound for playback.

[0087] In one embodiment of this application, the personal hearing aid product further includes: a noise estimator;

[0088] The noise estimator is used to estimate the noise of the time-frequency signal before the filter conversion module calculates the filter coefficients based on the gain at the frequency point, and obtains the gain at the frequency point.

[0089] The filter conversion module is used to fuse the gain of the frequency point output by the multi-subband dynamic range compressor and the gain of the frequency point output by the noise estimator in the frequency domain, and calculate the filter coefficients based on the gain of the fused frequency point.

[0090] In one embodiment of this application, a filter conversion module is used to calculate the filter coefficients of an FIR filter by means of inverse fast Fourier transform based on the gain of the fused frequency points.

[0091] In one embodiment of this application, a filter conversion module is used to calculate the filter coefficients of an IIR filter based on the gain of the fused frequency points using a filter parameter space search algorithm.

[0092] In one embodiment of this application, the personal hearing aid product further includes: a processing module;

[0093] The processing module is used to perform beamforming processing on the audio signal acquired by the microphone before the adaptive feedback canceller cancels the acoustic feedback in the audio signal acquired by the microphone.

[0094] An adaptive feedback canceller is used to eliminate acoustic feedback in beamformed audio signals.

[0095] As shown in Figure 4, this application provides a personal hearing aid product, including: a microphone, a processing module, an adaptive feedback canceller, an adaptive notch filter, a transformation module, a noise estimator, a multi-subband dynamic range compressor, a filter conversion module, a time-domain filter, an equalizer, a limiter, a digital-to-analog converter, and a speaker.

[0096] The personal hearing aid product provided in this application converts audio data from the frequency domain to the time domain for processing, eliminating the delay caused by frequency domain processing and making the visual effect of the personal hearing aid product more natural.

[0097] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this application that have equivalent elements, modifications, omissions, combinations (e.g., schemes where various embodiments overlap), adaptations, or changes. While several embodiments of wireless communication methods and wireless communication components have been described separately, it should be understood that the method details described in the wireless communication component description can also be incorporated into various embodiments of the wireless communication method, and vice versa.

[0098] The elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, the examples of which will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered merely illustrative, and the true scope and spirit are indicated by the claims and the full scope of their equivalents.

[0099] The order of the steps in this application is merely exemplary and not restrictive. The execution order of the steps can be adjusted without affecting the implementation of this application (without disrupting the logical relationship between the required steps), and the various embodiments obtained after the adjustment still fall within the scope of this application.

[0100] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated to be combined with each other in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. An audio data processing method for personal hearing aids, characterized in that, Personal hearing aids include: microphones, adaptive feedback cancellers, adaptive notch filters, converter modules, multi-subband dynamic range compressors, filter conversion modules, time-domain filters, equalizers, limiters, digital-to-analog converters, and speakers; the method includes: The adaptive feedback canceller eliminates acoustic feedback in the audio signal acquired by the microphone; The adaptive notch filter suppresses the howling signal in the audio signal output by the adaptive feedback canceller; The conversion module converts the audio signal collected by the microphone into a time-frequency signal; The multi-subband dynamic range compressor dynamically adjusts the time-frequency signal to obtain the gain at the frequency point; The filter conversion module calculates the filter coefficients based on the gain at the frequency point; The time-domain filter performs time-domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients; The equalizer adjusts the frequency response of the time-domain filtered audio signal; The limiter restricts the amplitude of the audio signal output by the equalizer to a preset range; The digital-to-analog converter converts the audio signal processed by the limiter into an electrical signal; The speaker converts the electrical signal into sound for playback.

2. The method as described in claim 1, characterized in that, The personal hearing aid product also includes: a noise estimator; Before the filter conversion module calculates the filter coefficients based on the gain at the frequency point, the method further includes: the noise estimator performs noise suppression on the time-frequency signal to obtain the gain at the frequency point; The filter conversion module calculates the filter coefficients based on the gain at the frequency point, including: The filter conversion module fuses the gain of the frequency point output by the multi-subband dynamic range compressor and the gain of the frequency point output by the noise estimator in the frequency domain, and calculates the filter coefficients based on the gain of the fused frequency point.

3. The method as described in claim 2, characterized in that, Calculate the filter coefficients based on the gain of the fused frequency points, including: Based on the gain of the fused frequency points, the filter coefficients of the FIR filter are calculated using inverse fast Fourier transform.

4. The method as described in claim 2, characterized in that, Calculate the filter coefficients based on the gain of the fused frequency points, including: Based on the gain of the fused frequency points, the filter coefficients of the IIR filter are calculated using a filter parameter space search algorithm.

5. The method as described in claim 1, characterized in that, The personal hearing aid product also includes: a processing module; Before the adaptive feedback canceller cancels acoustic feedback in the audio signal acquired by the microphone, the method further includes: The processing module performs beamforming processing on the audio signals collected by the microphone; The adaptive feedback canceller cancels acoustic feedback in the audio signal acquired by the microphone, including: The adaptive feedback canceller eliminates acoustic feedback in the beamforming audio signal.

6. A personal hearing aid product, characterized in that, include: Microphone, adaptive feedback canceller, adaptive notch filter, converter module, multi-subband dynamic range compressor, filter conversion module, time-domain filter, equalizer, limiter, digital-to-analog converter and speaker; The adaptive feedback canceller is used to eliminate acoustic feedback in the audio signal collected by the microphone; The adaptive notch filter is used to suppress howling signals in the audio signal output by the adaptive feedback canceller; The conversion module is used to convert the audio signal collected by the microphone into a time-frequency signal; The multi-subband dynamic range compressor is used to dynamically adjust the time-frequency signal to obtain the gain at the frequency point; The filter conversion module is used to calculate the filter coefficients based on the gain at the frequency point; The time-domain filter is used to perform time-domain filtering on the audio signal output by the adaptive notch filter based on the filter coefficients. The equalizer is used to adjust the frequency response of the time-domain filtered audio signal; The limiter is used to limit the amplitude of the audio signal output by the equalizer to a preset range; The digital-to-analog converter is used to convert the audio signal processed by the limiter into an electrical signal; The speaker is used to convert the electrical signal into sound for playback.

7. The personal hearing aid product as described in claim 6, characterized in that, Also includes: Noise estimator; The noise estimator is used to perform noise estimation on the time-frequency signal before the filter conversion module calculates the filter coefficients based on the gain at the frequency point, so as to obtain the gain at the frequency point; The filter conversion module is used to fuse the gain of the frequency point output by the multi-subband dynamic range compressor and the gain of the frequency point output by the noise estimator in the frequency domain, and calculate the filter coefficients based on the gain of the fused frequency point.

8. The personal hearing aid product as described in claim 7, characterized in that, The filter conversion module is used to calculate the filter coefficients of the FIR filter by inverse fast Fourier transform based on the gain of the fused frequency points.

9. The personal hearing aid product as described in claim 7, characterized in that, The filter conversion module is used to calculate the filter coefficients of the IIR filter based on the gain of the fused frequency points using a filter parameter space search algorithm.

10. The personal hearing aid product as described in claim 6, characterized in that, Also includes: Processing module; The processing module is used to perform beamforming processing on the audio signal acquired by the microphone before the adaptive feedback canceller cancels the acoustic feedback in the audio signal acquired by the microphone. The adaptive feedback canceller is used to eliminate acoustic feedback in audio signals that have undergone beamforming processing.

Images