Converting a binaural signal to a stereo audio signal

Abstract

Conversion of binaural signals into stereo audio signals. [Solution] An apparatus comprising means configured to acquire a binaural audio signal, acquire at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal, process the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band, and output the at least two audio signals for loudspeaker reproduction.

Description

[Technical field]

[0001] The present application relates to an apparatus and method for converting a binaural signal to a stereo audio signal, but not exclusively within a spatial audio signal environment. [Background technology]

[0002] Human perception of sound direction is based on binaural cues, including interbinaural time difference (ITD), interbinaural level difference (ILD), and spectral cues. Stereo signals for loudspeaker playback are typically generated using amplitude panning (e.g., VBAP, as discussed in "Virtual Sound Source Localization Using Vector Reference Amplitude Panning," Ville Pulkki, Audio Engineering Society, Journal of Audio Engineering Society, 1997), in which amplitude-panned sounds are reproduced on stereo loudspeakers and heard by a human listener.

[0003] Correspondingly, human perception of sound spaciousness and envelope is based on binaural cues related to interbinaural coherence (IC). Stereo signals are typically produced in a way that IC cues are produced in the human ear (e.g., using a reverberator) that create the perception of width or spaciousness (etc.) when the stereo signal is reproduced by stereo loudspeakers.

[0004] Binaural signals, on the other hand, are meant to be reproduced by headphones. Therefore, binaural cues (including ITD, ILD, IC, and spectral cues) need to be inherent in the audio signal itself. This can be achieved, for example, by recording spatial sound with microphones at the entrance of the ear canals of a real human or artificial head. Binaural sound can also be synthetically produced, for example, by applying appropriate head-related transfer functions (HRTFs) and reverberators to a multi-channel loudspeaker mix. When such binaural recordings, or binaural audio in general, are reproduced over headphones (possibly after headphone compensation), a realistic perception of spatial sound is achieved.

[0005] Immersive audio codecs have been implemented, one example of which is the Immersive Voice and Audio Services (IVAS) codec, which is designed for use over communication networks such as 3GPP 4G / 5G networks.

[0006] 3GPP 4G / 5G networks include use in immersive services such as immersive voice and audio for virtual reality (VR).

[0007] The input signal may be presented to the IVAS encoder in one of several supported formats (and in several allowed combinations of formats).

[0008] IVAS is proposed to use a binaural signal as input and have a conventional stereo audio output.

[0009] Because stereo audio signals are more suitable for loudspeaker playback, there is a need for an apparatus and method for efficiently converting binaural signals into conventional stereo audio signals. Summary of the Invention

[0010] According to a first aspect, there is provided an apparatus comprising means configured to obtain a binaural audio signal, obtain based on the binaural audio signal at least one directional parameter for at least one frequency band of the binaural audio signal, process the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by modifying inter-channel differences of the at least one frequency band of the binaural audio signal based on the at least one directional parameter of the at least one frequency band, and output the at least two audio signals for loudspeaker reproduction.

[0011] The inter-channel differences of at least one frequency band of the binaural audio signal may include at least one of at least one energy / amplitude difference for the channels of the binaural audio signal, at least one phase difference for the channels of the binaural audio signal, and at least one time difference for the channels of the binaural audio signal.

[0012] The means configured to process the binaural audio signals to generate at least two audio signals for loudspeaker reproduction may be further configured to apply a spectral adjustment to the processed at least one frequency band further based on the at least one directional parameter for the at least one frequency band.

[0013] The means configured to process the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of at least one directional parameter for at least one frequency band may be configured to generate an estimate of at least a portion of a covariance matrix for the at least one frequency band of the binaural audio signal, generate an energy estimate for the at least one frequency band of the binaural audio signal, generate at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band of the binaural audio signal, generate a mixing matrix for mixing the at least one frequency band of the binaural audio signal, and generate a left channel audio signal and a right channel audio signal from a combination of channels of the at least one frequency band of the binaural audio signal based on the mixing matrix.

[0014] The at least two audio signals for loudspeaker playback may include a left channel audio signal and a right channel audio signal.

[0015] The means configured to process the binaural audio signals to generate at least two audio signals for loudspeaker reproduction by modifying inter-channel differences of the at least one frequency band based on at least one directional parameter of the at least one frequency band may be further configured to generate, for the at least one frequency band, decorrelated audio signals based on the binaural audio signals, generate a further mixing matrix for the decorrelated audio signals, generate a further left channel audio signal and a further right channel audio signal from a combination of channels of the at least one frequency band of the decorrelated audio signals based on the further mixing matrix, combine the left channel audio signal and the further left channel audio signal to generate a combined left channel audio signal and combine the right channel audio signal and the further right channel audio signal to generate a combined right channel audio signal, wherein the at least two audio signals for loudspeaker reproduction comprise the combined left channel audio signal and the combined right channel audio signal.

[0016] The means configured to further apply a spectral adjustment to the processed at least one frequency band further based on the at least one directional parameter for the at least one frequency band may be configured to determine a binaural response and / or a long-term response estimate based on the directional parameter for the at least one frequency band and to compensate the determined binaural response and / or long-term response estimate from the processed at least one frequency band.

[0017] The binaural response and / or the long-term response may include at least one of: at least one energy / amplitude, at least one correlation for a channel of the binaural audio signal, at least one phase difference for a channel of the binaural audio signal, at least one time difference for a channel of the binaural audio signal.

[0018] The binaural response and / or long-term response may comprise a spectrum of the binaural audio signal, and the means configured to remove the determined binaural response and / or long-term response estimate from the processed at least one frequency band may be configured to obtain a filter and / or gain based on the estimated directional parameters and an averaged head-related transfer function corresponding to the at least one directional parameter, and to apply the filter and / or gain to the processed at least one frequency band.

[0019] The means configured to determine the binaural response and / or long-term response estimate based on the directional parameters of the at least one frequency band may be configured to generate a long-term equalization filter by comparing an average spectrum of the binaural signal with a predefined HRTF data set, and the means configured to remove the determined binaural response and / or long-term response estimate from the processed at least one frequency band may be configured to apply the long-term equalization filter to the processed at least one frequency band.

[0020] The means configured to obtain, based on the binaural audio signal, at least one directional parameter for at least one frequency band of the binaural audio signal may be configured to analyze the at least one frequency band of the binaural audio signal to determine the at least one directional parameter for the at least one frequency band.

[0021] The means configured to analyze at least one frequency band of the binaural audio signal to determine at least one directional parameter for the at least one frequency band may be further configured to estimate, for the at least one frequency band, a delay that maximizes correlation between channels of the binaural audio signal and to create the directional parameter based on the estimated delay.

[0022] The means may be further configured to obtain a direct-to-total energy ratio for at least one frequency band of the binaural audio signal based on the measured normalized correlation between the channels of the binaural audio signal.

[0023] The means configured to generate at least a portion of a target covariance matrix for the at least one frequency band of the binaural auditory signal based on the at least one directional parameter for the at least one frequency band may be further configured to generate the at least a portion of a target covariance matrix for the at least one frequency band of the binaural auditory signal further based on the direct-to-total energy ratio value for the at least one frequency band.

[0024] The means configured to determine a binaural response and / or a long-term response estimate based on the at least one directional parameter for the at least one frequency may be configured to determine the binaural response and / or the long-term response estimate based on a direct-to-total energy ratio for the at least one frequency band.

[0025] The means configured to obtain the binaural audio signal may be configured to perform one of: capturing the binaural audio signal with a dummy head; capturing the binaural audio signal at the entrance of a user's ear canal; rendering the binaural audio signal from a head-related transfer function; and rendering the binaural audio signal using a binaural room impulse response.

[0026] The means configured to output the at least two audio signals for loudspeaker reproduction may be configured to output the at least two audio signals for loudspeaker reproduction to stereo speakers.

[0027] According to a second aspect, there is provided a method comprising the steps of obtaining a binaural audio signal, obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal, processing the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band, and outputting the at least two audio signals for loudspeaker reproduction.

[0028] The inter-channel differences of at least one frequency band of the binaural audio signal may include at least one of at least one energy / amplitude difference for the channels of the binaural audio signal, at least one phase difference for the channels of the binaural audio signal, and at least one time difference for the channels of the binaural audio signal.

[0029] Processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction may further include applying a spectral adjustment to the processed at least one frequency band further based on at least one directional parameter for the at least one frequency band.

[0030] Processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band based on the at least one directional parameter for the at least one frequency band may include generating an estimate of at least a portion of a covariance matrix for the at least one frequency band of the binaural audio signal, generating at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal, generating at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band, generating a mixing matrix for mixing the at least one frequency band of the binaural audio signal, and generating a left channel audio signal and a right channel audio signal from a combination of channels of the at least one frequency band of the binaural audio signal based on the mixing matrix.

[0031] The at least two audio signals for loudspeaker playback may include a left channel audio signal and a right channel audio signal.

[0032] Processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of at least one frequency band based on at least one directional parameter for the at least one frequency band further comprises: generating, for the at least one frequency band, decorrelated audio signals based on the binaural audio signals; generating a further mixing matrix for the decorrelated audio signals; generating a further left channel audio signal and a further right channel audio signal from a combination of channels of the at least one frequency band of the decorrelated audio signals based on the further mixing matrix; combining the left channel audio signal and the further right channel audio signal to generate a combined left channel; and combining the right channel audio signal and the further right channel audio signal to generate a combined right channel, wherein the at least two audio signals for loudspeaker reproduction comprise the combined left channel audio signal and the combined right channel audio signal.

[0033] Applying a spectral adjustment to the processed at least one frequency band further based on the at least one directional parameter for the at least one frequency band may include determining a binaural response and / or a long-term response estimate based on the directional parameter for the at least one frequency band, and compensating the determined binaural response and / or long-term response estimate from the processed at least one frequency band.

[0034] The binaural response and / or the long term response may comprise at least one of at least one energy / amplitude, at least one correlation for a channel of the binaural audio signal, at least one phase difference for a channel of the binaural audio signal, at least one time difference for a channel of the binaural audio signal.

[0035] The binaural response and / or long-term response may comprise a spectrum of the binaural audio signal, and removing the determined binaural response and / or long-term response estimate from the processed at least one frequency band may comprise obtaining a filter and / or gain based on the estimated directional parameters and averaged head-related transfer functions corresponding to the at least one directional parameters, and applying the filter and / or gain to the processed at least one frequency band.

[0036] Determining the binaural response and / or long-term response estimate based on the directional parameters of the at least one frequency band may comprise generating a long-term equalization filter by comparing an average spectrum of the binaural signal with a predefined HRTF data set, and removing the determined binaural response and / or long-term response estimate from the processed at least one frequency band may comprise applying the long-term equalization filter to the processed at least one frequency band.

[0037] Obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal may comprise analysing the at least one frequency band of the binaural audio signal to determine the at least one directional parameter for the at least one frequency band.

[0038] Analyzing at least one frequency band of the binaural audio signal to determine at least one directional parameter for the at least one frequency band may include estimating a delay for the at least one frequency band that maximizes correlation between channels of the binaural audio signal, and creating the directional parameter based on the estimated delay.

[0039] The method may further include obtaining, for at least one frequency band of the binaural audio signal, a direct-to-total energy ratio value based on the measured normalized correlation between channels of the binaural audio signal.

[0040] Generating at least a portion of the target covariance matrix for at least one frequency band of the binaural audio signal based on at least one directional parameter for the at least one frequency band may further include generating at least a portion of the target covariance matrix for the at least one frequency band of the binaural audio signal based on a direct-to-total energy ratio value for the at least one frequency band.

[0041] Determining the binaural response and / or long-term response estimate based on the at least one directional parameter for the at least one frequency may include determining the binaural response and / or long-term response estimate based on a direct-to-total energy ratio value for the at least one frequency band.

[0042] Obtaining the binaural audio signal may include performing one of capturing the binaural audio signal with a dummy head, capturing the binaural audio signal at the entrance of a user's ear canal, rendering the binaural audio signal from a head-related transfer function, and rendering the binaural audio signal using a binaural room impulse response.

[0043] Outputting the at least two audio signals for loudspeaker reproduction may include outputting the at least two audio signals for loudspeaker reproduction to stereo loudspeakers.

[0044] According to a third aspect, there is provided an apparatus comprising at least one processor and at least one memory containing computer program code, the at least one memory and the computer program code causing the at least one processor to cause the apparatus to obtain a binaural audio signal, obtain based on the binaural audio signal at least one directional parameter for at least one frequency band of the binaural audio signal, process the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band, and output the at least two audio signals for loudspeaker reproduction.

[0045] The inter-channel differences of at least one frequency band of the binaural audio signal may include at least one of at least one energy / amplitude difference for the channels of the binaural audio signal, at least one phase difference for the channels of the binaural audio signal, and at least one time difference for the channels of the binaural audio signal.

[0046] The device that has processed the binaural audio signals to generate at least two audio signals for loudspeaker reproduction may be further adapted to apply a spectral adjustment to the processed at least one frequency band further based on at least one directional parameter for the at least one frequency band.

[0047] The apparatus for processing the binaural audio signal to generate at least two audio signals for the loudspeaker reproduction by correcting inter-channel differences of the at least one frequency band based on the at least one directional parameter for the at least one frequency band may generate an estimate of at least a portion of a covariance matrix for the at least one frequency band of the binaural audio signal, generate an energy estimate for the at least one frequency band of the binaural audio signal, generate at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band, generate a mixing matrix for mixing the at least one frequency band of the binaural audio signal, and generate a left channel audio signal and a right channel audio signal from a combination of channels of the at least one frequency band of the binaural audio signal based on the mixing matrix.

[0048] The at least two audio signals for loudspeaker playback may include a left channel audio signal and a right channel audio signal.

[0049] The apparatus for processing a binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of the at least one frequency band based on the at least one directional parameter for the at least one frequency band may further generate, based on the binaural audio signals, a decorrelated audio signal for loudspeaker reproduction, generate a further mixing matrix for the decorrelated audio signal, generate a further left channel audio signal and a further right channel audio signal from a combination of channels of the at least one frequency band of the decorrelated audio signal based on the further mixing matrix, combine the left channel audio signal with the further left channel audio signal to generate a combined left channel, and combine the right channel audio signal with the further right channel audio signal to generate a combined right channel. The at least two audio signals for loudspeaker reproduction include a combined left channel audio signal and a combined right channel audio signal.

[0050] The apparatus for further applying a spectral adjustment to the processed at least one frequency band further based on at least one directional parameter for the at least one frequency band may be configured to determine a binaural response and / or a long-term response estimate based on the directional parameter for the at least one frequency band, and to compensate the determined binaural response and / or long-term response estimate from the processed at least one frequency band.

[0051] The binaural response and / or the long term response may comprise at least one of at least one energy / amplitude, at least one correlation for a channel of the binaural audio signal, at least one phase difference for a channel of the binaural audio signal, at least one time difference for a channel of the binaural audio signal.

[0052] The binaural response and / or long-term response may comprise a spectrum of the binaural audio signal, and the apparatus for removing the determined binaural response and / or long-term response estimate from the at least one processed frequency band may obtain a filter and / or gain based on the estimated directional parameters and an averaged head-related transfer function corresponding to the at least one directional parameter, and apply the filter and / or gain to the at least one processed frequency band.

[0053] The apparatus adapted to determine the binaural response and / or long-term response estimate based on the directional parameters of the at least one frequency band may be adapted to generate a long-term equalization filter by comparing an average spectrum of the binaural signal with a predefined HRTF data set, and the apparatus adapted to remove the determined binaural response and / or long-term response estimate from the processed at least one frequency band may be adapted to apply the long-term equalization filter to the processed at least one frequency band.

[0054] An apparatus for obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on a binaural audio signal may analyze the at least one frequency band of the binaural audio signal to determine the at least one directional parameter for the at least one frequency band.

[0055] The apparatus for analyzing at least one frequency band of a binaural audio signal to determine at least one directional parameter for the at least one frequency band may further estimate a delay for the at least one frequency band that maximizes correlation between channels of the binaural audio signal and create the directional parameter based on the estimated delay.

[0056] The apparatus may be adapted to derive, for at least one frequency band of the binaural audio signal, a direct-to-total energy ratio value based on a measured normalized correlation between channels of the binaural audio signal.

[0057] An apparatus configured to generate at least a portion of a target covariance matrix for at least one frequency band of a binaural audio signal based on at least one directional parameter for the at least one frequency band may be further configured to further generate at least a portion of the target covariance matrix for the at least one frequency band of the binaural audio signal based on a direct-to-total energy ratio value for the at least one frequency band.

[0058] An apparatus adapted to determine a binaural response and / or a long-term response estimate based on at least one directional parameter for at least one frequency may be adapted to determine the binaural response and / or the long-term response estimate based on a direct-to-total energy ratio value for at least one frequency band.

[0059] The device for obtaining a binaural audio signal may perform one of capturing the binaural audio signal with a dummy head, capturing the binaural audio signal at the entrance of a user's ear canal, rendering the binaural audio signal from a head-related transfer function, and rendering the binaural audio signal using a binaural room impulse response.

[0060] The device for outputting at least two audio signals for loudspeaker reproduction may cause the at least two audio signals to be output to stereo loudspeakers for loudspeaker reproduction.

[0061] According to a fourth aspect, there is provided an apparatus comprising the steps of obtaining a circuit configured to obtain a binaural audio signal; obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal; processing circuitry configured to process the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band; and an output circuitry configured to output the at least two audio signals based on the at least one directional parameter for the at least one frequency band.

[0062] According to a fifth aspect, there is provided a computer program comprising instructions [or a computer readable medium comprising program instructions] for causing an apparatus to at least perform the following: obtaining a binaural audio signal; obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal; processing the binaural audio signal to generate at least two audio signals for reproduction on loudspeakers by correcting inter-channel differences for the at least one frequency band of the binaural audio signal based on the at least one directional parameter of the at least one frequency band; and outputting the at least two audio signals for reproduction on loudspeakers.

[0063] According to a sixth aspect, there is provided a non-transitory computer-readable medium comprising program instructions for at least: obtaining a binaural audio signal; obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal; processing the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of the at least one frequency band of the binaural audio signal based on the at least one directional parameter of the at least one frequency band; and outputting the at least two audio signals for loudspeaker reproduction.

[0064] According to a seventh aspect, there is provided an apparatus comprising means for obtaining a binaural audio signal; means for obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal; means for processing the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by modifying inter-channel differences of the at least one frequency band of the binaural audio signal based on the at least one directional parameter of the at least one frequency band; and means for outputting the at least two audio signals for loudspeaker reproduction.

[0065] According to an eighth aspect, there is provided a computer readable medium comprising program instructions for at least performing the following: obtaining a binaural audio signal; obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal; processing the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of the at least one frequency band of the binaural audio signal based on the at least one directional parameter of the at least one frequency band; and outputting the at least two audio signals for loudspeaker reproduction.

[0066] An apparatus comprising means for performing the operations of the methods described above.

[0067] An apparatus configured to perform the operations of the methods described above.

[0068] A computer program comprising program instructions for causing a computer to carry out the method described above.

[0069] A computer program product stored on the medium can cause an apparatus to perform the methods described herein.

[0070] The electronic device may comprise an apparatus as described herein.

[0071] The chipset may comprise an apparatus as described herein.

[0072] SUMMARY OF THE PRESENT APPLICATION The embodiments of the present application aim to address problems associated with the state of the art. [Brief description of the drawings]

[0073] For a better understanding of the present application, reference is made, by way of example, to the accompanying drawings, in which: [Figure 1]FIG. 1 illustrates a schematic of a system of apparatus suitable for implementing some embodiments. [Diagram 2] FIG. 2 illustrates a flow diagram of an exemplary apparatus operation according to some embodiments. [Diagram 3] FIG. 3 illustrates a schematic of an inter-channel difference modifier as shown in FIG. 1, according to some embodiments. [Figure 4] FIG. 4 illustrates a flow diagram of the operation of the exemplary inter-channel difference corrector shown in FIG. 3, according to some embodiments. [Diagram 5] FIG. 5 illustrates a schematic of a spectral whitener as shown in FIG. 1, according to some embodiments. [Figure 6] FIG. 6 illustrates a flow diagram of the operation of an example spectral whitener as shown in FIG. 5, according to some embodiments. [Figure 7] FIG. 7 shows an exemplary device suitable for implementing the apparatus shown in the previous figures. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0074] In the following, suitable devices and possible mechanisms for converting the binaural signal into a conventional stereo audio signal are described in further detail.

[0075] The concept discussed in further detail in the following embodiments is to generate a proper stereo audio signal from a binaural audio signal. In the following description, at least two audio signals are generated (which may include left and right channel audio signals, or may include front, middle, rear, top or bottom versions of left and right channels). The generated stereo audio signal can be reproduced on (stereo) loudspeakers. As a result, the binaural cues (ITD, ILD, IC, spectral cues) at the listener's ears generated by stereo speaker playback of the generated stereo audio signal are similar to the binaural cues when the binaural signal is played on headphones, and the spatial audio is perceived in the intended way. In other words, the present invention aims to prevent perceptual differences at the listener's ears depending on the output means. These differences include differences in sound direction, differences in sound width, differences in sound spread, differences in sound spectrum, etc.

[0076] With regard to spectral differences, binaural signals typically contain a characteristic spectrum caused by reflections from the human ears, head, torso, etc. The embodiments as discussed herein aim to generate a stereo audio signal based on the binaural signal, where this characteristic spectrum is compensated so that there is no additional binaural response in the signal when reproduced on (stereo) loudspeakers and heard by a human listener. As a result, the human listener does not receive a "double binaural spectrum" and the perception of the timbre is similar to the original timbre.

[0077] In terms of directional differences, a binaural signal is closer to a substantially dual mono signal with potential phase differences at lower frequencies, and therefore playback of such a signal on stereo loudspeakers produces an effect at lower frequencies similar to amplitude panning the sound to the center of the loudspeaker pair. The embodiments as discussed herein attempt to generate stereo audio signals that, when played on a stereo loudspeaker configuration, maintain the proper perception of width and sound source localization when compared to binaural audio signals when played through headphones.

[0078] The embodiments as discussed herein are configured to generate a suitable stereo audio signal from the binaural audio signal, thus eliminating the need to use the binaural signal when using stereo loudspeakers as a reproduction means, and thus preventing or reducing any spatial and timbre perception errors. As a result, the embodiments as discussed herein provide an improved perceived audio quality, since sound sources are not perceived from the wrong direction and timbre is not colored by binaural audio signals reproduced directly using stereo loudspeakers.

[0079] The concepts as discussed in the embodiments herein can be summarized as apparatus and methods relating to reproducing binaural signals using loudspeakers, where an apparatus and / or method is provided for converting the binaural signal into a "non-binaural" stereo signal suitable for stereo and multi-channel loudspeaker reproduction. Furthermore, as described in the embodiments herein, the conversion is performed by analyzing the direction of arrival (or more generally the directional parameters) in frequency bands from the binaural signal and modifying the binaural signal based on the analyzed direction, so that the inter-channel difference and spectrum match the expected characteristics of a "non-binaural" stereo signal.

[0080] The binaural signal can be any kind of binaural signal, such as a signal captured with a dummy head, a signal captured at the entrance of a real human ear canal, a signal rendered using a head-related transfer function, or a signal rendered using a binaural room impulse response, etc. Furthermore, the binaural signal may or may not include any type of headphone compensation (e.g., derived using a measured headphone transfer function).

[0081] Binaural signals are intended for listening over headphones, where they produce a natural perception of spatial sound (via natural ITDs, ILDs, and spectrum). As a result, sound sources can be perceived from the correct direction and with the correct timbre. In contrast, "non-binaural" stereo signals are intended for listening over loudspeakers (i.e., they are "traditional" stereo signals). When listened to over headphones, the reproduction does not resemble binaural sound in terms of ITDs, ILDs, or binaural spectrum, but instead these characteristics are formed when the "non-binaural" stereo signals are reproduced by loudspeakers and propagated to the listener's ears.

[0082] The direction of arrival can be analyzed by estimating the delay that maximizes the correlation between the (binaural) signals in the frequency band and creating a direction value based on that delay value. Based on the measured normalized correlation between the binaural left and right signals, the direct-to-total energy ratio value was estimated in the frequency band.

[0083] Inter-channel differences may in some embodiments be corrected by determining based on direction and ratio metadata at least a target energy / amplitude (and potentially correlation, phase / time difference) for loudspeaker playback, and correcting at least the energy / amplitude (and potentially correlation, phase / time difference) of the input binaural signals to match the corresponding target characteristics.

[0084] The spectrum in some embodiments may be modified by first obtaining a filter (or gain in a frequency band) based on the estimated direction of arrival and the averaged HRTF (of a set of HRTFs) corresponding to that direction. Furthermore, a long-term equalization filter can be applied by comparing the average spectrum of the binaural signal with a predefined HRTF data set (also with varying headphone compensation).

[0085] In some embodiments, the resulting "non-binaural" signals substantially eliminate or reduce the binaural features therein (intrinsic to the original binaural signals). In this way, binaural features are added by acoustic propagation from the loudspeakers to the listener's ears. Thus, the present invention can be used to achieve good audio quality (accurate, natural directional perception and non-loud timbre) for loudspeaker reproduction of binaural signals.

[0086] Referring to Figure 1, a block diagram of an apparatus suitable for implementing some embodiments is shown. As described below, the apparatus can be implemented in a mobile phone or a computer. Furthermore, it can be implemented as, for example, a stand-alone device or program, or it can be part of an audio codec, such as, for example, the IVAS codec.

[0087] The block diagram shows a binaural audio signal 100. The binaural audio signal 100 in this example is a time domain signal. However, in embodiments where the binaural audio signal 100 is a time-frequency domain signal, the use of a time-frequency transformer can be skipped or bypassed.

[0088] In one embodiment, the device comprises a time-frequency transformer 101. The time-frequency transformer 101 is configured to receive the binaural audio signals 100 and transform them into the time-frequency domain (time domain). Suitable transforms include, for example, the Short-Time Fourier Transform (STFT) and a complex-modulated Quadrature Mirror Filter (QMF) bank. The resulting time-frequency binaural audio signals 102 are S m It can be denoted as (b,n), where m is the channel index, b is the frequency bin index, and n is the time index.

[0089] The time-frequency binaural audio signal 102 may be forwarded to a direction analyzer 105 and an inter-channel difference corrector 103 .

[0090] In some embodiments, the device or transducer comprises a direction analyzer 105. The direction analyzer 105 is configured to receive the time-frequency domain binaural audio signal 102 and analyze the direction of arrival θ(k,n) and the direct-to-total energy ratio r(k,n) in the time-frequency domain, where k is a frequency band index.

[0091] The directional analysis is performed in frequency bands. The time-frequency transform has a certain frequency resolution, e.g. a 1024-point STFT results in 513 frequency bins from the DC frequency to the Nyquist frequency. These bins are grouped into frequency bands, e.g. 24 frequency bands that approximate the Bark frequency resolution.

[0092] The analysis can be done within these frequency bands. Each frequency band k has a lowest bin b low (k) and the highest bin b high (k).

[0093] For example, the analyzer may select a delay τ that maximizes the correlation between the two channels for each band k. kThis can be accomplished by creating time-shifted versions of the signal in one channel and correlating these with the other channel signal. S m The time shift of t time domain samples of (b,n) is

number

number

[0094] Delay τ k teeth,

number

[0095] Moreover, in some embodiments, the directional analyzer 105 is further configured to determine at least one corresponding energy ratio r(k,n), for example, using the correlation value c(k,n), e.g.,

number

number

[0096] The estimated direct-to-total energy ratio may also be forwarded to the inter-channel difference corrector 103 and the spectral whitener 107 .

[0097] In some embodiments, the converter comprises an inter-channel difference corrector 103. The inter-channel difference corrector 103 is configured to receive the time-frequency binaural audio signal 102 and the direction information 106 and the energy ratio information 108. Based on the analyzed direction and energy ratio, the inter-channel difference corrector 103 is configured to correct at least the inter-aural level differences (and potentially the phase and / or time difference and / or coherence) of the time-frequency binaural audio signal in the frequency band such that the processed output is suitable for loudspeaker reproduction of audio in the direction θ(k,n) and has an inter-channel level difference (and potentially the phase and / or time difference and / or coherence) with a direct-to-total energy ratio r(k,n).

[0098] The resulting time-frequency intermediate audio signal 104 is output from the inter-channel difference corrector and passed to a spectral whitener 107 .

[0099] The converter includes a spectral whitener 107 in some embodiments. The spectral whitener is configured to receive the time-frequency intermediate audio signals 104. Although the time-frequency intermediate audio signals 104 have appropriate directional cues for loudspeaker playback (e.g. level differences), they still have elements of the binaural spectrum included, which can be removed using the spectral whitener 107. Thus, the spectral whitener 107 is further configured to receive the directional information 106 and the direct-to-total energy ratio information 108. The spectral whitener 107 is configured to inverse frequency compensate the binaural spectrum, and the resulting time-frequency stereo audio signal 110 is output to an inverse time-to-frequency converter 111.

[0100] In some embodiments, the converter comprises an inverse time-to-frequency converter 111 configured to apply a time-frequency transformation, e.g. an inverse transformation corresponding to an inverse STFT, applied to the received time-frequency stereo audio signal 110, and then output a suitable (pulse code modulated) PCM stereo audio signal 112 that can be played on stereo loudspeakers.

[0101] With reference to FIG. 2, a flow diagram illustrating the operation of a converter such as that shown in FIG. 1 is shown.

[0102] Thus, for example, a first operation is that of receiving a binaural audio signal, as illustrated in FIG.

[0103] Then, as shown in FIG. 2 by step 203, the binaural audio signal is time-frequency transformed to generate a time-frequency binaural audio signal.

[0104] The time-frequency binaural audio signal can then be analyzed to determine the directions and energy ratios as shown in FIG. 2, by step 204 .

[0105] The time-frequency binaural audio signals may then be modified between channels based on the determined directions and energy ratios to generate time-frequency intermediate audio signals as shown in FIG. 2 by step 205 .

[0106] The time-frequency intermediate audio signal may then be spectrally whitened based on the determined direction and energy ratio to generate a time-frequency processed (stereo) audio signal, as shown in FIG. 2 by step 207.

[0107] The time-frequency processed (stereo) audio signal is then inverse time-frequency transformed to generate a stereo audio signal as shown in FIG.

[0108] Then, as shown in FIG. 2, the stereo audio signal can be output in stages in step 211.

[0109] 3, the inter-channel difference modifier 103 is shown in more detail. In some embodiments, the inter-channel difference modifier 103 comprises a covariance matrix estimator 301. The covariance matrix estimator 301 receives the time-frequency binaural audio signal 102,

number

number

[0110] The covariance matrix estimator 301 is C in (k, n) 300 estimated covariance matrix C in n(k, n) to the mixing matrix creator 307.

[0111] The covariance matrix estimator 301 also calculates the overall energy estimate E(k,n) by in The total energy estimate 302 may be provided to a target covariance matrix creator 305.

[0112] In the examples described herein, the input covariance matrix creation and the target covariance matrix creation encapsulate a set of inter-channel characteristics (energy difference, phase difference, correlation), all of which can be processed. However, in some embodiments, there may be at least a portion of the signal (e.g., some frequencies) where only the energy should be adjusted or modified. In such cases, it is not necessary to estimate the full covariance matrix. However, for simplicity, the full covariance matrix is ​​estimated here, and potentially unnecessary data (depending on the configuration) is not used in subsequent stages. A practical implementation in some embodiments is configured to estimate only the data or information required in subsequent stages.

[0113] In some embodiments, the inter-channel differential corrector 103 comprises a target covariance matrix creator 305. The target covariance matrix creator 305 is configured to receive the energy estimates 302 as well as the direction θ(k,n) 106 and direct-to-total energy ratio r(k,n) 108 parameters. The target covariance matrix creator 305 in some embodiments generates a target covariance matrix for the output loudspeaker signals. This can be performed in some embodiments by the following operations:

[0114] First, the matrix is ​​a panning gain

number

number

number

number

number

[0115] Next, the target covariance matrix C target The (k,n) 306 may be fed to a mixing matrix generator 307 .

[0116] In some embodiments, the inter-channel difference modifier 103 comprises a mixing matrix generator 307. The mixing matrix generator 307 is configured to receive the target covariance matrix 306 and the estimated covariance matrix 300 and generate a mixing matrix 308 that can be passed to a mixer 309.

[0117] In some embodiments, the mixing matrix creator 307 is configured to generate the mixing matrix according to the methods described in US20140233762A1 and “Optimized covariance domain framework for time-frequency processing of spatial audio”, Vilkamo, Juha, Tom Backstrom, and Achim Kuntz, Journal of Audio Engineering Society 61, no. 6 (2013): 403-411.

[0118] The method of the cited document involves a least squares optimized signal mixing technique for manipulating the covariance matrices of the signals while maintaining good audio quality. The method thus utilizes covariance matrix measures of the input signals and a target covariance matrix to provide a mixing matrix for performing such processing. The method also provides a means for optimally utilizing sufficient amounts of sound when there is not enough independent signal energy present at the input.

[0119] Thus, in some embodiments, the mixing matrix creator 307 is configured to generate a prototype matrix that determines how similar the output channels should be to the input channels (while satisfying the composition of the target covariance matrix). In the present context, the prototype matrix is

number

[0120] In some embodiments, the mixing matrix creator 307 is configured to compensate for the energy of the signal and not affect the phase or correlation between the channels. For example, at high frequencies this is the most robust option, where the phase / correlation information also has less perceptual relevance than at low frequencies. In this case the created mixing matrix is

number

[0121] In some embodiments, the inter-channel difference corrector 103 comprises a channel decorrelator 303. The channel decorrelator 303 is configured to receive the time-frequency binaural audio signal 102 and apply decorrelation to both channels s(b,n) to generate two incoherent versions (with respect to each other and with respect to the input) of the binaural input signal. The result is a decorrelated signal s d (b,n) is obtained. The decorrelation procedure may be a time-invariant phase scrambling procedure. Any decorrelator may be applied, and the choice of decorrelator may depend on the time-frequency transformation applied. The decorrelated signal 304 is then provided to a mixer 309 .

[0122] In some embodiments, the inter-channel difference modifier 103 comprises a mixer 309. The mixer 309 receives the time-frequency decorrelated audio signal 304, the time-frequency binaural audio signal 102 and a mixing matrix 308 and, for each frequency band k,

number

[0123] A mixing matrix is ​​for each band k, and the same mixing matrix can be applied to each bin b within that band. The mixing matrix (or alternatively, the covariance matrix before creating the mixing matrix) can be smoothed over time to reduce potential processing artifacts. The mixer 309 is then configured to output a time-frequency intermediate (loudspeaker) signal (without spectral whitening) 104.

[0124] The operation of the inter-channel difference corrector 103 is illustrated in a flow diagram as shown in FIG.

[0125] Receiving a time-frequency binaural audio signal is shown in step 401 of FIG.

[0126] After receiving the time-frequency binaural audio signal, both channels are decorrelated in step 403 as shown in FIG.

[0127] In addition from the time-frequency binaural audio signal, a covariance matrix and a (global) energy estimate are estimated, as shown in FIG. 4, according to step 405 .

[0128] The receipt of parametric parameters, such as direction and energy ratio, is indicated in FIG.

[0129] Following receipt of parametric parameters such as orientation and energy ratios and estimation of the covariance matrix, a target covariance matrix is ​​created by step 407 as shown in FIG.

[0130] Having created the target covariance matrix, a mixing matrix is ​​created as shown in FIG.

[0131] The time-frequency binaural audio signals and the time-frequency decorrelated audio signals are then mixed based on the created mixing matrix to generate time-frequency intermediate (loudspeaker) audio signals as shown in FIG. 4 by step 411.

[0132] Next, as shown in FIG. 4, a time-frequency intermediate (loudspeaker) audio signal is output via step 413.

[0133] Referring to FIG. 5, a block diagram of an example spectral whitener 107 is shown in accordance with some embodiments.

[0134] The spectral whitener 107 is configured to receive the time-frequency intermediate (loudspeaker) signal (without spectral whitening) LS(b,n) 104, the direction θ(k,n) 106, and the direct-to-total energy ratio r(k,n) 108.

[0135] In some embodiments, the spectral whitener 107 includes a binaural response estimator 503. The binaural response estimator 503 in some embodiments is configured to receive the direction 106 and the energy ratio 108, and the binaural response estimator 503 can then estimate an energy response of a typical binaural signal corresponding to the direction θ(k,n) and the energy ratio r(k,n). This energy response is common to binaural because the inter-channel difference has already been corrected in the inter-channel difference corrector 103.

[0136] The binaural response estimator 503 can be configured, for example, to first estimate an energy response to a direct sound based on the direction θ(k,n), e.g.

number

number

[0137] Next, we estimated the energy response to the ambient sound. The estimate is not based on any parameters and can be retrieved from a database. The estimate of the ambient sound energy response can be, for example, the averaged HRTF energy dataset.

number

[0138] The estimate of the binaural energy response is then

number

[0139] In some embodiments, the spectral whitener 107 includes a binaural response remover 501. The binaural response remover 501 receives a time-frequency intermediate (loudspeaker) signal (without spectral whitening) s′ LS (b, n) 104 and binaural energy response E bin (k,n) 504 as an input. The binaural response remover 501 first

number

number

[0140] Thus, in some embodiments, the processed intermediate signal s" LS (b,n) 502 may be forwarded to a long-term spectral estimator 505 and a long-term response remover 507 .

[0141] In some embodiments, the spectral whitener 107 is configured to convert the processed intermediate signal s" LS (b,n) 502, estimates the term spectrum of the intermediate signal and compares it to the expected average spectrum. Once the estimator finds a reliable deviation between the two, it estimates the estimated long-term response H lt (b,n) 506 is generated and sent to the long term response remover 507 .

[0142] In some embodiments, the spectral whitener 107 calculates the processed intermediate signal s" based on the estimated long-term response 506. LS (b,n) 502 and processes it to produce a suitable time-frequency stereo (loudspeaker) audio signal 110

number

[0143] If the deviation is not reliably detected, the estimated response H lt (b,n) may be set to 1 at all frequencies. Furthermore, in an embodiment, the long-term spectral estimator 505 and the long-term response remover 507 are optional and may be omitted, and the processed intermediate audio signal s" may be LS (b,n) 502 is passed directly as a time-frequency stereo audio signal 110 .

[0144] The output of the spectral whitener 107 is the time-frequency domain stereo signal s LS (b,n) and then transformed into a time domain signal as expressed in the context of FIG. 1, the result being suitable for loudspeaker reproduction.

[0145] The binaural channel differences are corrected to channel differences more suitable for loudspeaker reproduction, and the binaural spectrum is compensated.

[0146] With reference to FIG. 6, a flow diagram illustrating the operation of an example of the spectral whitener 107 is shown.

[0147] Thus, in step 601 an intermediate time-frequency audio signal is received as shown in FIG.

[0148] Additionally, parametric parameters such as direction and energy ratios are illustrated in FIG.

[0149] The binaural response is estimated by step 604 as shown in FIG.

[0150] The estimated binaural response is then removed from the time-frequency intermediate audio signal, by step 605, as shown in FIG.

[0151] Optionally, the long-term response is then estimated, via step 607, as shown in FIG.

[0152] Next, the estimated long-term response is optionally removed, as shown in FIG. 6, via step 609.

[0153] In the above-described embodiments, the binaural signal is completely converted to a non-binaural stereo signal. However, it may be desirable to convert only a portion of the binaural signal to a non-binaural stereo signal. For example, when the binaural to non-binaural conversion occurs, it is possible to render only those directions that are mapped between the stereo loudspeakers as non-binaural sounds, while the remaining (binaural) sounds are reproduced on the loudspeakers using a crosstalk cancelling scheme. Thus, in some embodiments, a portion of the binaural audio signal for a range of directions is converted to a stereo signal, while the remaining signal is passed through unconverted. This portion may also be a portion of the total energy of the binaural audio signal, or it may be a portion of the spectrum of the binaural audio signal (e.g., some frequency bands are converted and some frequency bands are passed through unprocessed).

[0154] 7, an exemplary electronic device that may be used as any of the device components of the system. The device may be any suitable electronic device or equipment. For example, in some embodiments, the device 1700 is a mobile device, a user device, a tablet computer, a computer, an audio playback device, etc.

[0155] In some embodiments, device 1700 includes at least one processor or central processing unit 1707. Processor 1707 may be configured to execute various program code, such as the methods described herein.

[0156] In some embodiments, the apparatus 1700 comprises a memory 1711. In some embodiments, at least one processor 1707 is coupled to the memory 1711. The memory 1711 may be any suitable storage means. In some embodiments, the memory 1711 includes program code sections for storing program code implementable on the processor 1707. Furthermore, in some embodiments, the memory 1711 may further comprise a stored data section for storing data, e.g., data that has been processed or is to be processed according to the embodiments described herein. The implemented program code stored in the program code section and the data stored in the stored data section may be retrieved by the processor 1707 whenever needed via the memory-processor coupling.

[0157] In some embodiments, the apparatus 1700 comprises a user interface 1705. The user interface 1705 may be coupled to a processor 1707 in some embodiments. In some embodiments, the processor 1707 may control the operation of the user interface 1705 and receive input from the user interface 1705. In some embodiments, the user interface 1705 may allow a user to input commands into the device 1700, for example, via a keypad. In some embodiments, the user interface 1705 may allow a user to obtain information from the device 1700. For example, the user interface 1705 may comprise a display configured to display information from the device 1700 to the user. The user interface 1705 may comprise a touch screen or touch interface in some embodiments that can both allow information to be input into the device 1700 and further display information to the user of the device 1700. In some embodiments, the user interface 1705 may be a user interface for communication.

[0158] In some embodiments, device 1700 comprises an input / output port 1709, which in some embodiments comprises a transceiver. The transceiver in such embodiments may be coupled to processor 1707 and configured to enable communication with other apparatus or electronic devices, for example, via a wireless communication network. The transceiver or any suitable transceiver or transmitter and / or receiver means may include: In some embodiments, it may be configured to communicate with other electronic devices or apparatus via wires or wired coupling.

[0159] The transceiver may communicate with the further device by any suitable known communication protocol, for example in some embodiments the transceiver may use a suitable Universal Mobile Telecommunications System (UMTS) protocol, a Wireless Local Area Network (WLAN) protocol such as IEEE 802.X, a suitable short range radio frequency communication protocol such as Bluetooth, or an Infrared Data Path (IRDA).

[0160] The transceiver input / output port 1709 may be configured to receive signals.

[0161] The input / output port 1709 may be coupled to any suitable audio output, for example a stereo speaker system.

[0162] In general, various embodiments of the present invention can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be shown and described as block diagrams, flow charts, or using any other pictorial representation, it should be well understood that these blocks, devices, systems, techniques, or methods described herein may be implemented in, by way of non-limiting examples, hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or any combination thereof.

[0163] The embodiments of the present invention may be realized by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Furthermore, in this regard, it should be noted that any block of the logic flow as illustrated may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on a physical medium, such as a memory chip, or a memory block implemented within a processor, a magnetic medium, such as a hard disk or floppy disk, and an optical medium, such as a DVD or data variants thereof.

[0164] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed and removable memory, etc. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), gate level circuits, and processors based on multi-core processor architectures.

[0165] Embodiments of the present invention can be implemented in a variety of components, such as integrated circuit modules. The design of integrated circuits is a highly automated process and is large scale. Complex and powerful software tools are available to convert logic level designs into complete semiconductor circuit designs that are ready to be etched and formed on semiconductor substrates.

[0166] Programs such as those available from Synopsys, Inc., Mountain View, California, and Cadence Design, Inc., San Jose, California, use well-established rules of design and a library of pre-stored design modules to automatically route conductors and identify the locations of components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standardized electronic format (e.g., Opus, GDSII, etc.) may be sent to a semiconductor manufacturing facility or "fab" for fabrication.

[0167] The foregoing description has provided a complete and informative description of exemplary embodiments of the present invention by way of illustrative and non-limiting examples. However, various modifications and adaptations will become apparent to those skilled in the art in view of the foregoing description upon perusal of the accompanying drawings and the appended claims. However, all such similar modifications of the teachings of the present invention will still fall within the scope of the present invention as defined in the appended claims.

Claims

1. 1. A method for converting a binaural signal into a stereo audio signal, the method comprising: obtaining a binaural audio signal; - obtaining at least one directional parameter of the at least one frequency band of the binaural audio signal based on an analysis of the at least one frequency band of the binaural audio signal; - obtaining at least one direct-to-total energy ratio value for the at least one frequency band of the binaural audio signal based on a measured normalized correlation between channels of the binaural audio signal; - processing the binaural audio signal to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences of the at least one frequency band of the binaural audio signal based on the at least one directional parameter and the at least one direct-to-total energy ratio value for the at least one frequency band; outputting at least two audio signals for loudspeaker playback; A method comprising:

2. The inter-channel difference of the at least one frequency band of the binaural audio signal is at least one energy / amplitude difference for a channel of the binaural audio signal; at least one phase difference for a channel of the binaural audio signal; and at least one time difference for a channel of the binaural audio signal; At least one of: The method of claim 1.

3. The step of processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction comprises: configured to further apply a spectral adjustment to the processed at least one frequency band further based on the at least one directional parameter for the at least one frequency band. The method of claim 1.

4. processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band based on the at least one directional parameter for the at least one frequency band, comprising: generating an estimate of at least a portion of a covariance matrix for the at least one frequency band of the binaural audio signal; generating an energy estimate for the at least one frequency band of the binaural audio signal; generating at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band; generating a mixing matrix for mixing the at least one frequency band of the binaural audio signal; configured to generate a left channel audio signal and a right channel audio signal from a combination of channels of the at least one frequency band of the binaural audio signal based on the mixing matrix. The method of claim 1.

5. The method of claim 4 , wherein the at least two audio signals for loudspeaker playback include the left channel audio signal and the right channel audio signal.

6. processing the binaural audio signals to generate at least two audio signals for loudspeaker reproduction by correcting inter-channel differences for the at least one frequency band based on the at least one directional parameter for the at least one frequency band, comprising: and generating, for at least one frequency band, a decorrelated audio signal based on the binaural audio signal; generating a further mixing matrix for the decorrelated audio signals; generating a further left channel audio signal and a further right channel audio signal from a combination of channels of the at least one frequency band of the uncorrelated audio signal based on the further mixing matrix; combining the left channel audio signal with the further left channel audio signal to generate a combined left channel; configured to combine the right channel audio signal with the further right channel audio signal to generate a combined right channel; Where: The at least two audio signals for loudspeaker reproduction include: the combined left channel audio signal; the combined right channel audio signal; Including, The method according to claim 4.

7. applying a spectral adjustment to the processed at least one frequency band further based on the at least one directional parameter for the at least one frequency band, comprising: determining a binaural response and / or a long-term response estimate based on the directional parameters for the at least one frequency band; and compensating the determined binaural response and / or long-term response estimate from the processed at least one frequency band. The method of claim 3 , wherein the method is configured to:

8. The binaural response and / or the long-term response may be At least one energy / amplitude, at least one correlation for a channel of the binaural audio signal; at least one phase difference for a channel of the binaural audio signal; at least one time difference for a channel of the binaural audio signal; at least one of The method according to claim 7.

9. the binaural response and / or the long-term response comprises a spectrum of the binaural audio signal; 4. The step of removing the determined binaural response and / or long-term response estimate from the processed at least one frequency band, comprising: obtaining filters and / or gains for applying filters and / or gains to the processed at least one frequency band based on the estimated directional parameters and an average head-related transfer function corresponding to the at least one directional parameter; applying said filter and / or gain to said processed at least one frequency band; The method according to claim 7.

10. The step of determining a binaural response and / or a long-term response estimate based on the directional parameters of the at least one frequency band is configured to generate a long-term equalization filter by comparing an average spectrum of the binaural signal with a predefined head-related transfer function (HRTF) data set; the step of removing the determined binaural response and / or long-term response estimate from the processed at least one frequency band is configured to apply the long-term equalization filter to the processed at least one frequency band. The method according to claim 7.

11. The step of obtaining at least one directional parameter for at least one frequency band of the binaural audio signal based on the binaural audio signal comprises: configured to analyze the at least one frequency band of the binaural audio signal to determine the at least one directional parameter for the at least one frequency band. The method of claim 1.

12. The step of analysing the at least one frequency band of the binaural audio signal to determine the at least one directional parameter for the at least one frequency band comprises: and estimating, for the at least one frequency band, a delay that maximizes correlation between channels of the binaural audio signal; configured to generate a directional parameter based on the estimated delay. The method of claim 11.

13. The step of generating at least a portion of a target covariance matrix for the at least one frequency band of the binaural audio signal based on the at least one directional parameter for the at least one frequency band comprises: and further configured to generate the at least a portion of the target covariance matrix for the at least one frequency band of the binaural audio signal further based on the direct-to-total energy ratio value for the at least one frequency band. The method of claim 1.

14. The step of configuring to determine a binaural response and / or a long-term response estimate based on the at least one directional parameter for the at least one frequency comprises: configured to determine the binaural response and / or long-term response estimate based on the direct-to-total energy ratio value for the at least one frequency band. The method of claim 1.

15. The step of obtaining a binaural audio signal includes: capturing the binaural audio signals with a dummy head; capturing the binaural audio signals at the entrance of a user's ear canal; - rendering the binaural audio signal from head-related transfer functions; and Rendering the binaural audio signal using a binaural room impulse response. configured to perform one of the following: The method of claim 1.

16. The step of outputting the at least two audio signals for loudspeaker playback comprises: configured to output the at least two audio signals for loudspeaker playback to a stereo loudspeaker. The method of claim 1.

17. Apparatus for implementing the method of any one of claims 1 to 16.

Images