Head-related transfer function adaptive method and system

Abstract

The present disclosure provides a head-related transfer function (HRTF) adaptive method and system. The method includes: performing system identification, wherein the system identification includes auricle identification and occlusion identification.

Description

Technical Field

[0001] The present disclosure relates to the field of audio, and more particularly to a head-related transfer function (HRTF) adaptation method and system using a hybrid adaptive active noise canceller (ANC) loop. Background Art

[0002] In the past few years, ANC headphones have become increasingly popular because they can provide users with a relatively quiet environment in a noisy environment, reduce unnecessary ambient noise, and thus bring greater convenience and comfort to users.

[0003] As people's requirements for user experience continue to increase, spatial audio technology (also known as 3D audio technology) has received more attention and use. This technology makes it possible to create a 3D audio experience by using headphones. Applications of this technology include realizing augmented virtual reality, listening to music, and watching movies on a tablet or PC. Virtual surround headphones are a typical application of 3D audio technology. When surround sound is presented through 3D audio headphones, it will produce the same audio experience as listening to an actual speaker system.

[0004] HRTF is an advanced 3D audio rendering method that makes sounds appear to come from specific points in 3D space to synthesize binaural audio. In order to achieve fidelity and immersive experience during binaural audio reproduction, HRTF is often used as a filter to describe the sound transmission from the sound source to the listener's eardrum.

[0005] ANC headphones are another typical application, which use the HRTF from the noise source to the ear entry point (EEP) and introduce sound waves with matched amplitude but opposite phase to reduce the severity of noise pollution (e.g., street noise, airplane engine noise, and office chatter).

[0006] In short, HRTF is highly personalized and will vary from person to person. Each person has a different upper body contour and a different ear shape, and therefore a different sound filtering effect. In current practice, a common HRTF from a group of subjects is usually used offline and on headphones. This approach of using a common HRTF has two disadvantages:

[0007] 1) Once the ordinary HRTF cannot match the actual user of the terminal, a very poor sound localization effect will occur due to the front-back and top-bottom confusion (the so-called confusion cone) associated with 3D audio.

[0008] 2) Although modifying the non-personalized generic HRTF can be a labor-saving approach, it is always accompanied by undesirable audio distortion.

[0009] The existing HRTF measurement is: use a set of loudspeakers mounted on a semicircular rotating ring to generate an excitation signal (e.g., an exponential sweep signal). Place a simulated head or an individual's head at the center of the semicircular ring, and set microphones in the eardrums of the left and right ears of the simulated head or the individual's head. However, this measurement is very difficult and more time-consuming.

[0010] In addition, current ANC designs either use fixed HRTF / offline HRTF or require dedicated hardware, which is more expensive. Moreover, ANC designs with fixed HRTF have the following two disadvantages: 1) they cannot accurately adapt to different environmental noises in the real world based on field calibration / measurement; and 2) they cannot achieve user personalization, such as human differences between headphones causing ANC results to vary, and for example, sound leakage due to various different fit states between the headphones and the wearer's head.

[0011] In order to overcome the above-mentioned deficiencies of inaccurate and non-personalized HRTF, an improved solution is needed. Summary of the invention

[0012] The present disclosure provides a solution for obtaining adaptive HRTFs, for example, from far field to near field and from ear reference point (ERP) to ear entry point (EEP) through adaptive ANC. In addition, in another embodiment of the present disclosure, adaptive HRTFs are used for compensation in a variety of applications (e.g., ANC headphone applications and 3D headphone applications). In addition, the present disclosure can provide a hybrid (feedback + feedforward) adaptive ANC to adapt to different adaptive states.

[0013] According to one or more aspects of the present disclosure, a HRTF adaptation method is provided. The method includes: performing system identification. The system identification includes auricle identification and occlusion identification. The method provided according to one or more aspects of the present disclosure also includes: performing system compensation based on the adaptive HRTF obtained from the system identification. The method provided according to one or more aspects of the present disclosure also includes: generating an HRTF rendering matrix based on the auricle identification and the occlusion identification.

[0014] According to one or more aspects of the present disclosure, a HRTF adaptive system is provided. The system includes a memory and a processor. The memory is configured to store computer-readable instructions. The processor is configured to perform system identification when executing the computer-readable instructions. The system identification includes auricle identification and occlusion identification.

[0015] Another embodiment of the present disclosure provides a computer-readable medium, wherein the computer-readable medium is configured to execute the steps of the above method.

[0016] Advantageously, the method and system disclosed in the present disclosure can provide personalized HRTF according to different users, so that users can obtain a better sound experience when using headphones. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present disclosure may be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings. The various parts in the drawings are not necessarily drawn to scale, but are focused on illustrating the principles of the present invention. In addition, in the drawings, similar or identical reference numerals refer to similar or identical elements.

[0018] Figure 1 A schematic diagram illustrating the methods and systems of the present disclosure;

[0019] Figure 2 A schematic diagram illustrating an ANC feedback loop of an embodiment of the present disclosure;

[0020] Figure 3 A graph showing a left ear transfer function (TF) measured by a method according to an embodiment of the present disclosure;

[0021] Figure 4 1. shows a right ear TF curve graph measured by a method according to an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram illustrating an ANC feed-forward loop of another embodiment of the present disclosure;

[0023] Figure 6 illustrates a schematic diagram of an auditory echo cancellation system H(Z) implemented in the frequency domain (FD); and

[0024] Figure 7 Schematic diagram illustrating the adaptation of the auditory echo cancellation system H(Z) implemented in FD. DETAILED DESCRIPTION

[0025] It should be understood that the following description of the embodiments is given for illustrative purposes only and not for limiting purposes. The example division on the functional blocks, modules or units shown in the drawings should not be regarded as indicating that these functional blocks and these modules or units must be implemented as physically separated units. The functional blocks, modules or units shown or described can be implemented as individual units, circuits, chips, functions, modules or circuit elements. One or more functional blocks or units can also be implemented as a common circuit, chip, circuit element or unit.

[0026] Any one or more of the processors, memories, or systems described herein include computer executable instructions that can be compiled or interpreted according to a computer program formed using various programming languages ​​and / or techniques. In general, a processor (e.g., a microprocessor) receives instructions, such as from a memory, a computer readable medium, etc., and executes the instructions. The processor includes a non-transitory computer readable storage medium capable of executing the instructions of the software program. The computer readable medium can be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0027] Figure 1 Schematic diagram illustrating the methods and systems of the present disclosure. Figure 1 As shown in, the present disclosure provides a HRTF adaptive method and system. The method may include system identification and system compensation. The system identification is intended to determine the difference between a reference model and a user. System identification mainly focuses on auricle differences and masking functions. That is, system identification may include auricle identification and masking identification. System compensation is intended to use a mathematical modeling method to compensate for the system differences between a reference model and a user. For example, an HRTF rendering matrix is ​​generated based on the output of auricle identification and masking identification.

[0028] For system identification, we can refer to the acoustic echo canceller (AEC) in telecommunication systems for modeling. Figure 6 A schematic diagram illustrating the auditory echo cancellation system H(Z) implemented in FD is shown. Figure 6 In the embodiment of the present disclosure, by using an echo path identification algorithm (e.g., a normalized least mean square (NLMS) algorithm), the TF from the speaker (horn) spk to the internal microphone (i.e., auricle identification from ERP to EEP) and the TF from the external microphone to the internal microphone (i.e., shielding identification from the far field to EEP) can be obtained.

[0029] Pinna recognition (ERP to EEP)

[0030] The following will refer to Figure 2 Auricle recognition (ERP to EEP) of one aspect of the present disclosure is described. Figure 2 A schematic diagram illustrating the ANC feedback loop. Figure 2 The ANC feedback loop in the embodiment is used as an example to describe the system model of auricle recognition in the present disclosure. Figure 2 As shown in FIG. 1 , for the convenience of description, for example, the position of the loudspeaker (horn) spk is defined as ERP, the position of the error microphone is defined as EEP, and the HRTF from ERP to EEP is defined as H 0The controller can be implemented as an AEC system. For a more intuitive understanding, Figure 6 The AEC system shown in is described as an example. For example, the controller may implement an adaptive algorithm based on NLMS.

[0031] In order to describe the individual differences (HRTF) between the pinna (loudspeaker (spk) at the ERP) and the ear canal (error microphone (mic) at the EEP) that affect the spatial fidelity, it is necessary to use Figure 2 The feedback loop in the 0 inverse function of ).

[0032] Next, the process of auricle recognition will be described in detail.

[0033] First, obtain the data from ERP to EEP (H 0 ) HRTF. This process can be implemented in two ways. One way includes: capturing any reference audio signal from the earphone spk, recording the signal through the error microphone, and then transforming the signal from the time domain (TD) to FD through the fast Fourier transform (FFT). Another way includes: using the AEC adaptive loop to obtain the adaptive HRTF (HRTF). 0 ). For ease of understanding, Figure 6 The diagram illustrates adaptive AEC using NLMS. It will be appreciated by those skilled in the art that the present disclosure may also be used with AEC adaptive loops using other adaptive algorithms such as RLS and VLMS.

[0034] Next, by 0 Perform curve fitting to obtain the HRTF compensation curve (H 0 -1 ). For example, given a known filter, the curve fit can be modeled as an arbitrary amplitude filter design.

[0035] Finally, before the audio signal in FD is reproduced through the loudspeaker spk, the audio signal is multiplied by the HRTF compensation curve (H 0 -1 ).

[0036] Figure 3 and Figure 4 Schematic diagrams of left ear HRTF and right ear HRTF of left ear auricle identification and right ear auricle identification, respectively, the left ear HRTF and right ear HRTF are obtained by noise cancellation measurement, different user measurement and artificial head measurement in the method according to an embodiment of the present disclosure. The noise cancellation measurement is: sound absorbing foam is placed on the top of the earmuff of the headphone. Figure 3 and Figure 4It can be seen that the HRTF adaptation method of the present disclosure can obtain corresponding HRTFs (ie, different frequency response curves in the figure) based on earless ness, different users and artificial heads, so that personalized HRTF measurements can be implemented for different test targets.

[0037] Obstruction detection (far field to EEP)

[0038] The system model for occlusion recognition can be the same as the feed-forward loop designed by ANC. Figure 5 A schematic diagram of the ANC feedforward loop is shown. For ease of understanding, the system model of the masking recognition disclosed in the present invention will be described as follows: Figure 5 Take the ANC feed-forward loop in as an example.

[0039] In view of the combined binaural feed-forward ANC, the different HRTFs between the left ERP / EEP and the right ERP / EEP describe the masking effect of the head, which will be compensated individually and adaptively according to the 3D audio.

[0040] by Figure 5 Take the mono feed-forward ANC shown in for example. Figure 5 The far-field HRTF from the noise source to the reference microphone (ERP) and the near-field HRTF from the ERP to the error microphone (EEP) are shown in . The reference microphone and the error microphone usually have the same characteristics.

[0041] Figure 5 The diagram illustrates the various components and signal paths of a mono feed-forward ANC. A reference microphone 2 located outside the headphone 1 is configured to measure the far-field HRTF. An error microphone 3 located inside the headphone 1 is configured to measure the near-field HRTF. Noise 4 entering the system is filtered into a signal 5 by the ear cup of the headphone. The signal 6 played by the headphone speaker is preferably the inverse of signal 5. P(z) in the diagram represents the far-field HRTF from the noise source to the reference microphone (ERP). N(z) represents the low-pass characteristic of the headphone ear cup, which is a passive isolation function. H 0 represents the near-field HRTF from the earphone speaker (located approximately at the ERP) to the error microphone (EEP). The controller can be implemented as an AEC system, such as Figure 6 Similarly, those skilled in the art will appreciate that Figure 6 Only adaptive AEC using NLMS is illustrated. The present disclosure may also use AEC adaptive loops using other adaptive algorithms (eg, RLS and VLMS).

[0042] Assuming that the noise source captured by the reference microphone 2 will be regarded as the reference signal X(Z), and the signal captured by the error microphone 3 will be regarded as the input signal Y(Z), the final estimated echo cancellation transfer function H(z) will be combined with N(z) (e.g. low-pass filtering in FD) and H 0 Associated.

[0043] According to one aspect of the present disclosure, a priori estimation can be incorporated into the ANC feedforward loop based on the measurement results to obtain better and more stable performance. In actual operation, in the NLMS AEC system, the low-pass filtering (e.g., the cutoff frequency is 3kHz) and the H derived through the feedback loop are used. 0 Multiply by the reference microphone signal X(Z) in FD.

[0044] In the solution of the present disclosure, after performing pinna recognition and occlusion recognition on the system, the obtained adaptive HRTF can be applied to the ANC headset during use of the ANC headset to achieve accurate and personalized HRTF measurement and adaptation. In addition, a hybrid (feedback + feedforward) adaptive ANC headset design can be provided to adapt to different adaptation states.

[0045] System compensation

[0046] For example, in the case of 3D virtual surround headphones, in order to ultimately reproduce the HRTF for the customer, a reverse mapping is required to measure the near-field TF and the incomplete directional masking function and map it to a 360-degree model. In the field of statistical analysis, this process can be modeled as a sparsity problem. For example, a reference head model is used to collect a large number of near-field measurements and far-field measurements and train a deep neural network (DNN). The data is collected in the form of impulse responses that are located throughout the space and have different degrees and distances.

[0047] The measured masking function and pinna response may then be used as input during calculations to generate a 360 degree HRTF rendering matrix to achieve system compensation effects.

[0048] In binaural hearing modeling studies, HRTFs can usually be divided into two free-field spatial characteristics based on the distance from the sound source to the center of the head, namely the far field (e.g., distance greater than 1.0m) and the near field (e.g., distance less than 1.0m). How to determine the source of free-field sound mainly depends on three auditory cues: (a) interaural time difference (ITD), (b) interaural intensity difference (ILD), and (c) auditory filtering, which are spectral cues derived from the shape of the human ear, head, and body. The near-field HRTF depends on the human body structure, especially the outer ear structure consisting of the auricle, ear canal, and eardrum.

[0049] Figure 6 and Figure 7 Schematic diagrams illustrating the auditory echo cancellation system implemented in FD and the adaptive process of H(Z) implementing the echo cancellation system are respectively illustrated. It can be understood by those skilled in the art that Figure 6 and Figure 7 The purpose is to help understand the technology of the present disclosure rather than to limit the technology of the present disclosure in a narrow sense. Figure 6 and Figure 7 This is further described.

[0050] FD has become the first choice for AEC because it can implement high-order adaptive filters H(z) with high convergence speed and moderate computational complexity. Figure 6 and Figure 7 The two basic modules of NLMS AEC filtering and adaptation are shown in FIG. Figure 6 and Figure 7 The diagram illustrates the situation of the speaker-environmental space-microphone (LEM) system. It can be understood by those skilled in the art that Figure 6 and Figure 7 The purpose is to illustrate the underlying principles by way of example and not to make any specific limitations.

[0051] In FD, AEC is usually implemented using fast convolution / correlation techniques. The cross-correlation between the error signal e(i) and the reference signal x(i) in TD is equal to E(z) multiplied by X in FD. * (z)(X * (z) is the conjugate of X(z)). Figure 7 Shows the use of reference signal The inverse power spectral density (PSD) of is used as the normalization of the gradient. The step size μ(z) in FD ensures the robustness of H(z).

[0052] Figure 6 and Figure 7 The AEC version shown in can only control the linear part of the LEM system, and additional residual echo suppression (RES) is usually used to further reduce the echo to keep the echo within the range of the (linear) AEC error (i). However, it is well known that RES represents a non-linear signal processing stage and has inherent deficiencies. It may produce audible artifacts known as musical tones, which need to be avoided.

[0053] The following is a brief description of some basic mathematical principles of NLMS AEC. Those skilled in the art will appreciate that the following description is only to help understand the basic principles of NLMS AEC, rather than to make specific limitations.

[0054] Based on the Wiener-Khinchin formula and Parseval's theorem,

[0055] Φ xx(z) = X(z)·X * (z)

[0056] Adaptation of the echo canceller H(z) is implemented, for example, as follows:

[0057]

[0058] Optimal step size μ opt (z) is derived based on the relationship between E(z) and X(z), which will be used in practice to determine the optimal step size μ opt (z) for simulation, analysis, and fine-tuning. In practice, larger μ opt (z) will converge quickly, but may cause instability. Smaller μ opt (z) will converge slowly, but sometimes it is not sufficient for practical applications.

[0059] The present disclosure further provides a system, the system comprising a memory and a processor. The memory is configured to store computer-readable instructions. The processor is configured to perform system identification when executing the computer-readable instructions. The system identification includes auricle identification and occlusion identification.

[0060] The description of the embodiments has been presented for the purpose of illustration and description. Appropriate modifications and changes of the embodiments can be implemented according to the above description or can be obtained by practical methods. For example, unless otherwise indicated, one or more of the methods can be performed by a combination of suitable devices and / or systems. The method can be performed in the following manner: one or more logic devices (such as processors) are used in combination with one or more additional hardware elements (such as storage devices, memories, circuits, hardware network interfaces, etc.) to perform the stored instructions. The methods and associated actions can also be performed in parallel and / or simultaneously, in various orders other than the order described in the application. The system is illustrative in nature and may include additional elements and / or omitted elements. The subject matter of the present disclosure includes all novel and non-obvious combinations of the disclosed various methods and system configurations and other features, functions and / or properties.

[0061] As used in this application, an element or step listed in the singular and preceded by the word "a / an" should be understood as not excluding a plurality of the elements or steps described, unless such exclusion is indicated. In addition, reference to "one embodiment" or "example" in the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that also include the described features.

Claims

1. A head-related transfer function (HRTF) adaptive method, comprising: Performing system identification, wherein the system identification includes pinna identification and occlusion identification; and performing system compensation based on the adaptive HRTF obtained from said system identification; wherein the system compensation comprises generating an HRTF rendering matrix based on the outputs of the pinna identification and the occlusion identification; The auricle recognition comprises: Obtaining an adaptive HRTF from the ear reference point ERP to the ear entry point EEP; performing curve fitting based on the adaptive HRTF to obtain a compensation curve of the HRTF; and multiplying the audio signal in the frequency domain FD by the compensation curve; The mask identification includes: The audio signal received from the reference microphone and the audio signal received from the error microphone are input into the adaptive controller, and an adaptive HRTF is obtained. The method of claim 1 , wherein the compensation curve is an inverse function of the HRTF.

3. The method according to claim 1, wherein the pinna identification is implemented by a feedback loop of an active noise canceller (ANC) comprising an adaptive controller. The method of claim 1 , wherein the masking identification is implemented by a feed-forward loop of an ANC comprising an adaptive controller.

5. The method according to claim 1, further comprising: System compensation is performed via a mixed output of the adaptive HRTF obtained from the pinna identification and the adaptive HRTF obtained from the occlusion identification.

6. A head-related transfer function (HRTF) adaptive system, comprising: a memory configured to store computer-readable instructions; as well as A processor configured to: performing system identification when the computer readable instructions are executed, the system identification comprising pinna identification and occlusion identification; and performing system compensation based on the adaptive HRTF obtained from the system identification; wherein the processor is further configured to generate an HRTF rendering matrix based on the outputs of the pinna identification and the occlusion identification; The processor is further configured to perform the auricle recognition, and the auricle recognition includes: Obtaining an adaptive HRTF from the ear reference point ERP to the ear entry point EEP; performing curve fitting based on the adaptive HRTF to obtain a compensation curve of the HRTF; and multiplying the audio signal in the frequency domain FD by the compensation curve; The processor is further configured to perform the occlusion identification, the occlusion identification comprising: The audio signal received from the reference microphone and the audio signal received from the error microphone are input into the adaptive controller, and an adaptive HRTF is obtained. The system of claim 6 , wherein the compensation curve is an inverse function of the HRTF.

8. The system of claim 6, wherein the processor is further configured to implement the pinna identification through a feedback loop of an active noise canceller (ANC) comprising an adaptive controller.

9. The system of claim 6, wherein the processor is further configured to implement the occlusion identification via a feed-forward loop of an ANC comprising an adaptive controller.

10. The system of claim 6, wherein the processor is further configured to perform system compensation via a mixed output of the adaptive HRTF obtained from the pinna identification and the adaptive HRTF obtained from the occlusion identification.

11. A computer readable medium comprising instructions for executing the method according to any one of claims 1 to 5.

Images