Audio processing method and related device for audio transparency

Abstract

The present disclosure relates to an audio processing method and a related device for performing audio transparency. An audio processing method comprises: converting a time domain audio signal into a frequency domain audio signal; determining a noise reduction gain according to the frequency domain audio signal; selecting at least one set of time domain filter coefficients from a plurality of preset time domain filter coefficients according to the noise reduction gain; setting a time domain filter according to the selected at least one set of time domain filter coefficients, and filtering the time domain audio signal using the time domain filter.

Description

Technical Field

[0001] The present invention relates to an audio device, and more particularly to an audio processing method and a related device for realizing low-latency audio pass-through technology in a headphone system. Background Art

[0002] In-ear headphones or closed-end earmuff headphone systems usually have a certain degree of sound isolation. If you want the user to be able to hear the external environment while listening to music with this type of headphones, you usually use a microphone to receive the external sound, and the speaker of the headphone plays the ambient sound received by the microphone. This technology is called Audio Pass-Through (APT).

[0003] Audio transparency technology generally pursues a natural sense of hearing. While retaining the ambient sound, it is also hoped to remove the noise in the ambient sound, such as the sound of the air conditioner, the sound of the wind, or the background noise of the microphone. However, in the process of reducing noise, due to the involvement of some digital-to-analog conversion, time-domain-frequency conversion and signal processing processes, a certain degree of delay will be caused. In the process of audio transparency processing, part of the ambient sound that the user can hear comes from the sound waves that penetrate the sound insulation layer of the headset from the external environment, and part comes from the microphone recording played by the speaker of the headset through the noise reduction processing. Therefore, if the delay of the noise reduction processing is too high, the sound of the above two parts will be obviously out of sync, causing the user to feel an echo.

[0004] Please refer to Figure 1 , which illustrates the architecture of an audio processing device for implementing audio pass-through technology in the prior art. As shown in the figure, the analog audio signal recorded by the sound pickup device (such as a microphone) 10 is first converted into a time-domain digital audio signal x[t] by an analog-to-digital converter 11. Afterwards, the time-domain digital audio signal x[t] is converted into a frequency-domain audio signal X[f, t] through a Fourier transform unit 12. In addition, a corresponding noise reduction gain G[f, t] is generated based on the frequency-domain audio signal X[f, t] through a noise floor estimation unit 13 and a noise reduction gain calculation unit 14. The noise reduction processing unit 15 performs noise reduction processing on the frequency-domain audio signal X[f, t] according to the noise reduction gain G[f, t], thereby obtaining a frequency-domain audio signal Y[f, t] after noise reduction. The frequency domain audio signal Y[f, t] is converted back to the time domain through the inverse Fourier transform unit 16 to obtain the time domain audio signal y[t]. Then, it is combined with the audio signal z[t] (such as music, voice, etc.) that the user wants to listen to through the summing unit 17. Finally, it is converted into an analog audio signal through the digital to analog converter 18, thereby driving the speaker unit and converting the electronic signal into sound waves for the user to listen to.

[0005] In this architecture, assuming that the sampling frequency of the analog-to-digital converter 11 is fs, and the size of the Fourier transform unit 12 is N, the processed signal will have a delay of at least N / fs relative to the original sound of the external environment. In the common case of N=128 and fs=16KHz, there will be a delay of at least 8ms, and this degree of delay will obviously give users a bad feeling. Summary of the invention

[0006] In order to solve the above problems, the purpose of the present invention is to propose an audio processing method and device for realizing audio transparency technology. In the audio processing architecture proposed by the present invention, noise reduction processing is mainly performed in the time domain through a time domain filter. Compared with the conventional architecture, the delay caused by the conversion between the time domain and the frequency domain can be effectively reduced. Furthermore, after the noise estimation and analysis in the frequency domain, the present invention determines the specific time domain filter setting through the pre-stored time domain filter coefficients. Therefore, the use of frequency domain filter coefficients is avoided, which may cause potential delays caused by the conversion between the frequency domain and the time domain. In summary, the audio processing method and device of the present invention can achieve an audio transparency effect with low latency and good noise reduction effect.

[0007] An embodiment of the present invention provides an audio processing method, which includes: converting a time domain audio signal into a frequency domain audio signal; determining a noise reduction gain according to the frequency domain audio signal; selecting at least one set of time domain filter coefficients from a plurality of preset time domain filter coefficients according to the noise reduction gain; setting a time domain filter according to the selected at least one set of time domain filter coefficients, and filtering the time domain audio signal using the time domain filter.

[0008] One embodiment of the present invention provides an audio processing device, which includes: a Fourier transform unit, a noise analysis device, a filter coefficient storage unit, a filter coefficient selection unit and a time domain filter. The Fourier transform unit is used to convert a time domain audio signal into a frequency domain audio signal. The noise analysis device is coupled to the Fourier transform unit and is used to determine a noise reduction gain according to the frequency domain audio signal. The filter coefficient storage unit is used to store a plurality of preset time domain filter coefficients. The filter coefficient selection unit is coupled to the noise analysis device and the filter coefficient storage unit, and is used to select at least one set of time domain filter coefficients from the plurality of preset time domain filter coefficients according to the noise reduction gain. The time domain filter is coupled to the filter coefficient selection unit, is controlled by the selected at least one set of time domain filter coefficients, and is used to filter the time domain audio signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A diagram showing the architecture of a conventional audio processing device is shown.

[0010] Figure 2 A structural diagram of an audio processing device according to an embodiment of the present invention is shown.

[0011] Figure 3 Plot the frequency response curve of noise reduction gain.

[0012] Figure 4 The filter frequency response curves corresponding to different time-domain filter coefficient groups according to the embodiment of the present invention are shown.

[0013] Figure 5 A simplified flow chart of a sound effect processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0014] In the following text, many specific details are described to provide readers with a thorough understanding of the embodiments of the present invention. However, those skilled in the art will understand how to implement the present invention without one or more specific details, or using other methods, components, materials, etc. In other cases, well-known structures, materials, or operations will not be shown or described in detail to avoid obscuring the core concepts of the present invention.

[0015] The reference to "an embodiment" in the specification means that the specific features, structures or characteristics described in the embodiment may be included in at least one embodiment of the present invention. Therefore, the "in an embodiment" that appears in various places in this specification does not necessarily mean the same embodiment. In addition, the aforementioned specific features, structures or characteristics may be combined in one or more embodiments in any suitable form.

[0016] Please refer to Figure 2 , which illustrates an architecture diagram of an audio processing device according to an embodiment of the present invention. As shown in the figure, the audio processing device 100 of the present invention includes: an analog-to-digital converter (ADC) 110, a Fourier transform unit 120, a noise floor estimation unit 130, a gain calculation unit 135, a frequency determination unit 140, a filter coefficient selection unit 145, a filter coefficient storage unit 150, a time domain filter 160, a summing unit 170, and a digital-to-analog converter (DAC) 180.

[0017] The analog-to-digital converter 110 is used to convert an analog audio signal generated by an external sound pickup device 10 (such as a microphone) based on an external environmental sound into a digital time-domain audio signal x[t]. The Fourier transform unit 120 is used to convert the time-domain audio signal x[t] into a frequency-domain audio signal X[f, t]. In one embodiment, the Fourier transform unit 120 generates the frequency-domain audio signal X[f, t] by performing a short-time Fourier transform (STFT). The noise floor estimation unit 130 is used to estimate the noise floor of the frequency-domain audio signal X[f, t] to obtain the noise floor Nf[f, t]. According to the noise floor Nf[f, t], the gain calculation unit 135 calculates the noise reduction gain G[f, t] required to eliminate the current noise. The noise floor estimation unit 130 and the gain calculation unit 135 may estimate the noise floor Nf[f, t] and the noise reduction gain G[f, t] according to various appropriate algorithms.

[0018] According to the noise reduction gain G[f, t] calculated by the gain calculation unit 135, the frequency determination unit 140 will calculate one or more frequency parameters, and the filter coefficient selection unit 145 will select the filter coefficients accordingly. Figure 3 , which represents the noise reduction gain G[f,t] at time point t0, that is, the noise reduction gain G[f,t0]. At this time, the frequency determination unit 140 finds the maximum frequency Fmax according to the noise reduction gain G[f,t0]. The maximum frequency Fmax is the frequency when the noise reduction gain G[f,t0] is greater than a certain critical value. Figure 3 For example, when the threshold value is set at 0.9, the frequency determination unit 140 determines that the maximum frequency Fmax is 3500 Hz. Furthermore, in one embodiment, the maximum frequency Fmax can also be adjusted by performing a weighted average calculation on the maximum frequency Fmax(t0-1) determined at the previous time point and the maximum frequency Fmax(t0) determined at the current time point:

[0019] Fmax'(t0)=Fmax(t0-1)*K+Fmax(t0)*(1-K)

[0020] Thus, the adjusted maximum frequency Fmax'(t0) is obtained, and the frequency determination unit 140 provides the maximum frequency Fmax as the maximum frequency Fmax to the filter coefficient selection unit 145. In addition, in one embodiment, the frequency determination unit 140 may adjust the maximum frequency Fmax(t0) using a fixed offset L, or further adjust the maximum frequency Fmax'(t0) that has been adjusted:

[0021] Fmax”(t0)=Fmax'(t0)+L

[0022] or

[0023] Fmax”(t0)=Fmax(t0)+L

[0024] Thus, the adjusted maximum frequency Fmax" (t0) is obtained as the maximum frequency Fmax and provided to the filter coefficient selection unit 145. According to the frequency parameter provided by the frequency determination unit 140, the filter coefficient selection unit 145 selects a set of appropriate time domain filter coefficients from the multiple sets of time domain filter coefficients pre-stored in the filter coefficient storage unit 150. The multiple sets of filter coefficients pre-stored in the filter coefficient storage unit 150 are multiple combinations of coefficients related to filter characteristics, covering multiple different bandwidths. The cutoff frequencies fc corresponding to these time domain filter coefficient sets are distributed between 0 and fs / 2 (fs is the sampling frequency of the system), for example, fc = 500 Hz, 1000 Hz, ... 7500 Hz. Furthermore, the filter coefficient selection unit 145 will find the time domain filter coefficient set whose cutoff frequency fc is closest to the maximum frequency Fmax, and in a later process, bring this set of time domain filter coefficients into the time domain filter 160.

[0025] Please note that in the above examples, only the corresponding processing method for high-frequency noise is mentioned. However, this is not a limitation of the present invention. In multiple embodiments of the present invention, the high-frequency and low-frequency noises can be eliminated simultaneously by redesigning the frequency determination unit 140 and the filter coefficient type stored in the filter coefficient storage unit 150. For example, the multiple groups of time-domain filter coefficients stored in the filter coefficient storage unit 150 may include multiple groups of time-domain filter coefficients with low-pass characteristics, which correspond to the cut-off frequency fc_low, and multiple groups of time-domain filter coefficients with high-pass characteristics, which correspond to the cut-off frequency fc_high.

[0026] On the other hand, the frequency determination unit 140 finds the maximum frequency Fmax(t0) that makes G[Fmax,t0] greater than a certain critical value and the minimum frequency Fmin(t0) that makes G[Fmin,t0] greater than a certain critical value from the noise reduction gain G[f,t0]. The frequency determination unit 140 can process Fmax(t0) and Fmin(t0) by the aforementioned weighted average or offset shift, thereby outputting the adjusted maximum frequency Fmax"(t0) or Fmax'(t0) and the adjusted minimum frequency Fmin"(t0) or Fmin'(t0) to the filter coefficient selection unit 145. Afterwards, the filter coefficient selection unit 145 finds a group of time domain filter coefficients whose corresponding cutoff frequency fc_high is closest to Fmin" (t0) or Fmin' (t0) from the multiple groups of time domain filter coefficients with high-pass characteristics, and finds a group of time domain filter coefficients whose corresponding cutoff frequency fc_low is closest to Fmax" (t0) or Fmax' (t0) from the multiple groups of time domain filter coefficients with low-pass characteristics, thereby obtaining coefficients that can realize a bandpass filter, and in the subsequent process, it is brought into the time domain filter 160.

[0027] In one embodiment, in order to reduce system delay as much as possible, the pre-stored time domain filter coefficients and the time domain filter 160 can implement a minimum phase filter, and the type can be a high-shelving filter or a low-shelving filter. In addition, the time domain filter 160 can be an infinite impulse response (IIR) or finite impulse response (FINITE IMPULSE RESPONSE) type filter. In one embodiment, each set of time domain filter coefficients may include: a cutoff frequency fc, a sampling frequency fs, an amplitude A, and a quality factor Q.

[0028] Furthermore, through the following conversion formula:

[0029] cos_w0 = cos(2*pi*(fc / fs));

[0030] sin_w0 = sin(2*pi*(fc / fs));

[0031] α=sin_w0 / 2*sqrt((A+1 / A)*(1 / Q-1)+2);

[0032] a0=((A+1)-(A-1)*cos_w0+2*sqrt(A)*α);

[0033] b0=(A*((A+1)+(A-1)*cos_w0+2*sqrt(A)*α)) / a0;

[0034] b1=(-2*A*((A-1)+(A+1)*cos_w0)) / a0;

[0035] b2=(A*((A+1)+(A-1)*cos_w0-2*sqrt(A)*α)) / a0;

[0036] a1=2*((A-1)-(A+1)*cos_w0) / a0;

[0037] a2=((A+1)-(A-1)*cos_w0-2*sqrt(A)*α) / a0;

[0038] The transfer function of the time domain filter 160 can be obtained:

[0039] H(z)=(b0+b1*z^-1+b2*z^-2) / (1+a1*z^-1+a2*z^-2)

[0040] Figure 4 The various filter frequency responses that can be achieved when the cutoff frequency fc=500:500:7500 (Hz), the sampling frequency fs=16000 Hz, the amplitude A=0.5, the quality factor Q=1, etc. are shown. Please note that the above-mentioned time domain filter coefficients: cutoff frequency fc, sampling frequency fs, amplitude A, quality factor Q, etc. are not limited to the filter coefficient set pre-stored in the present invention. In different embodiments of the present invention, each set of pre-stored time domain filter coefficients may also include more different types of coefficients, so as to more finely change and present the characteristics of the time domain filter 160.

[0041] The time domain filter 160 will filter out the external environmental noise in the time domain audio signal x[t] in the time domain according to a set of time domain filter coefficients selected by the filter coefficient selection unit 145. As can be seen from the previous description, the filter coefficient selection unit 145 selects the time domain filter coefficients by referring to the noise reduction gain G[f,t] calculated by the noise reduction gain calculation unit 135. When the frequency domain audio signal X[f,t] changes, the noise reduction gain G[f,t] also changes accordingly. Therefore, the filter coefficient selection unit 145 will select different time domain filter coefficients when the signal changes. In one embodiment, in order to avoid the change of the filter characteristics of the time domain filter 160 when switching different time domain filter coefficients, thereby causing popping sounds, the audio processing device 100 of the present invention is further provided with a filter coefficient interpolation unit 155. Through the filter coefficient interpolation unit 155, the time domain filter 160 can have a more gentle characteristic conversion. Here, it is assumed that at the current time point, the filter coefficient selection unit 145 selects the time domain filter coefficient [B, A], and at the previous time point, the filter coefficient selection unit 145 selects the time domain filter coefficient [B', A'], which means that the time domain filter coefficient of the time domain filter 160 will be updated from [B', A'] to [B, A]. At this time, the filter coefficient interpolation unit 155 will interpolate multiple groups of time domain filter coefficients according to the time domain filter coefficients [B', A'] and [B, A] to achieve a gentle transition of the time domain filter characteristics. It is assumed that the filter coefficient interpolation unit 155 can perform N coefficient updates at N time points, and the update time is Nk, where k=0, 1..., and the time domain filter coefficient at the time point N(k-1) is [B', A'], and the time domain filter coefficient at the time point Nk is [B, A], then the time domain filter coefficients B_use[Nk+n] and A_use[Nk+n] used at the time point Nk+n, (n=0 to N-1) can be:

[0042] B_use[Nk+n]=B'+(B-B')*(n / N)

[0043] A_use[Nk+n]=A'+(A-A')*(n / N)

[0044] Please note that the above-mentioned time domain filter coefficients [B, A] are not limited to the pre-stored time domain filter coefficients of the present invention. For example, the pre-stored time domain filter coefficients of the present invention may include more than two sets of coefficients that need to be smoothly converted through interpolation processing.

[0045] Through the above-mentioned coefficient setting, the time domain filter 160 can filter out the noise in the time domain audio signal x[t], thereby generating a filtered time domain audio signal y[t]. The filtered time domain audio signal y[t] will be combined with the audio signal z[t] (such as music, voice, etc.) that the user wants to listen to through the summing circuit 170, and finally converted into an analog audio signal through the digital to analog converter 180, thereby driving the speaker unit and converting the electronic signal into sound waves for the user to listen to.

[0046] Figure 5 A simplified flow chart of an audio processing method according to an embodiment of the present invention is shown. The flow chart includes the following steps:

[0047] Step 510: Convert a time domain audio signal into a frequency domain audio signal;

[0048] Step 520: Determine a noise reduction gain according to the frequency domain audio signal;

[0049] Step 530: Select at least one set of time-domain filter coefficients from a plurality of sets of preset time-domain filter coefficients according to the noise reduction gain; and

[0050] Step 540: Set a time domain filter according to the selected at least one set of time domain filter coefficients, and use the time domain filter to filter the time domain audio signal.

[0051] Since the principles and specific details of the above steps have been described in detail in the previous embodiment of the signal processing device 100, they will not be described again here. It should be noted that the above process may also be able to better implement the audio processing method of the present invention by adding other additional steps.

[0052] In summary, the conventional technology involves multiple conversions between the time domain and the frequency domain, which results in a long delay time. The present invention utilizes a time domain filter and pre-stored time domain filter coefficients to reduce the conversion between the time domain and the frequency domain. Among them, the present invention converts the time domain audio signal to the frequency domain for noise floor estimation and noise reduction gain calculation, and then selects an appropriate combination from the pre-stored time domain filter coefficient group to set the time domain filter, and performs noise reduction processing accordingly. In addition, in order to avoid possible popping sounds when the filter coefficients change, the present invention also uses interpolation processing to allow the filter characteristics to change smoothly. In this way, the present invention reduces the delay and avoids the occurrence of echoes, thereby ensuring the natural hearing of the audio transparency technology, while also providing a good noise reduction effect.

[0053] The embodiments of the present invention may be implemented using hardware, software, firmware, and combinations thereof. By using an appropriate instruction execution system, the embodiments of the present invention may be implemented using software or firmware stored in a memory. As for hardware, any of the following technologies or combinations thereof may be used to implement the embodiments: an individual operation logic having logic gates that can perform logic functions according to data signals, an application specific integrated circuit (ASIC) having suitable combinational logic gates, a programmable gate array (PGA), or a field programmable gate array (FPGA), etc.

[0054] The processes and blocks in the flowcharts in the specification illustrate the architecture, functions and operations that can be implemented by the systems, methods and computer software products based on various embodiments of the present invention. In this regard, each block in the flowchart or functional block diagram can represent a module, segment or part of the program code, which includes one or more executable instructions for implementing the specified logical function. In addition, each block in the functional block diagram and / or the flowchart, as well as the combination of blocks, can basically be implemented by a dedicated hardware system that performs the specified function or action, or a combination of dedicated hardware and computer program instructions. These computer program instructions can also be stored in a computer-readable medium, which can enable a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable medium implement the functions / actions specified by the blocks in the flowchart and / or functional block diagram.

[0055] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

[0056]

Explanation of symbols

[0057] 10.105 Sound pickup device

[0058] 100 Audio Processing Device

[0059] 11.110 Analog to Digital Converter

[0060] 12, 16, 120 Fourier transform units

[0061] 13.130 Noise floor estimation unit

[0062] 14, 135 noise reduction gain calculation unit

[0063] 15 Noise Reduction Processing Unit

[0064] 140 Frequency determination unit

[0065] 145 Filter coefficient selection unit

[0066] 150 filter coefficient storage unit

[0067] 155 filter coefficient interpolation unit

[0068] 160 Time Domain Filter

[0069] 17, 170 Total units

[0070] 18, 180 Digital to Analog Converter

[0071] 19, 190 speaker units

Claims

1. An audio processing method, comprising: Converting a time domain audio signal into a frequency domain audio signal; determining a noise reduction gain according to the frequency domain audio signal; According to the noise reduction gain, at least one set of time domain filter coefficients is selected from a plurality of sets of preset time domain filter coefficients; A time domain filter is set according to the selected at least one set of time domain filter coefficients, and the time domain filter is used to filter the time domain audio signal. The step of selecting the at least one set of time domain filter coefficients according to the noise reduction gain comprises: Determining a maximum frequency according to the frequency corresponding to when the noise reduction gain is greater than a predetermined threshold value; and The at least one group of time-domain filter coefficients is selected from the plurality of groups of preset time-domain filter coefficients according to the maximum frequency.

2. The audio processing method as claimed in claim 1, wherein the step of determining the noise reduction gain according to the frequency domain audio signal comprises: estimating a noise floor of the frequency domain audio signal; and The noise reduction gain is calculated according to the noise floor.

3. The audio processing method as claimed in claim 1, wherein the step of converting the time domain audio signal into the frequency domain audio signal comprises: A short-time Fourier transform (STFT) is performed on the time-domain audio signal to obtain the frequency-domain audio signal.

4. The audio processing method as claimed in claim 1, wherein the step of selecting the at least one set of time domain filter coefficients according to the maximum frequency comprises: Performing a frequency averaging operation and / or a frequency shifting operation according to the maximum frequency, thereby obtaining an adjusted maximum frequency; and The at least one group of time-domain filter coefficients is selected from the plurality of groups of preset time-domain filter coefficients according to the adjusted maximum frequency.

5. The audio processing method as claimed in claim 1, further comprising: interpolating one or more third sets of filter coefficients according to a first set of filter coefficients and a second set of filter coefficients selected from the plurality of sets of preset time-domain filter coefficients at a first time point and a second time point respectively; and In a time interval, the time domain filter is sequentially set according to the first set of filter coefficients, the one or more sets of third set of filter coefficients and the second set of filter coefficients.

6. The audio processing method of claim 1, wherein the plurality of sets of preset time-domain filter coefficients can set the time-domain filter to be a high-shelving filter, a low-shelving filter or a band pass filter.

7. An audio processing device, comprising: a Fourier transform unit for converting a time domain audio signal into a frequency domain audio signal; a noise analysis device, coupled to the Fourier transform unit, for determining a noise reduction gain according to the frequency domain audio signal; A filter coefficient storage unit for storing a plurality of sets of preset time-domain filter coefficients; a filter coefficient selection unit, coupled to the noise analysis device and the filter coefficient storage unit, for selecting at least one set of time-domain filter coefficients from the plurality of sets of preset time-domain filter coefficients according to the noise reduction gain; a frequency determination unit coupled to the noise analysis device, for determining a maximum frequency according to a frequency corresponding to when the noise reduction gain is greater than a predetermined threshold value, wherein the filter coefficient selection unit selects the at least one set of time domain filter coefficients from the plurality of sets of preset time domain filter coefficients according to the maximum frequency; as well as A time domain filter is coupled to the filter coefficient selection unit, is controlled by the selected at least one set of time domain filter coefficients, and is used to filter the time domain audio signal.

8. The audio processing device as claimed in claim 7, further comprising: a filter coefficient interpolation unit, coupled to the filter coefficient selection unit, for interpolating one or more third sets of filter coefficients according to a first set of filter coefficients and a second set of filter coefficients selected by the filter coefficient selection unit from the plurality of sets of preset time-domain filter coefficients at a first time point and a second time point respectively; in, The time domain filter is sequentially set according to the first set of filter coefficients, the one or more third sets of filter coefficients and the second set of filter coefficients within a time interval.

Images