APPARATUS AND METHOD FOR SYNTHESIZING A SPATIALLY EXTENDED SOUND SOURCE USING REFERENCE INFORMATION ELEMENTS.

Abstract

An apparatus for synthesizing a spatially extended sound source comprises: a spatial information interface (100) for receiving a spatial range indication indicating a limited spatial range for the spatially extended sound source within a maximum spatial range (600); a reference information provider (200) for providing one or more reference information elements in response to the limited spatial range; and an audio processor (300) for processing an audio signal representing the spatially extended sound source using one or more reference information elements.

Description

APPARATUS AND METHOD FOR SYNTHESIZING A SPATIALLY EXTENDED SOUND SOURCE USING REFERENCE INFORMATION ELEMENTS Field of Invention The present invention relates to audio signal processing and, in particular, to the reproduction of one or more spatially extended sound sources. Background of the Invention For several applications, sound sources must be reproduced through multiple speakers or headphones. These applications include 6-degree-of-freedom (6DoF) virtual, mixed, or augmented reality applications. The simplest way to reproduce sound sources across configurations is to render them as point sources. However, when the goal is to reproduce physical sound sources with a significant auditory spatial extent, this model is insufficient. Examples of these sound sources include a grand piano, a choir, or a waterfall, all of which have a certain size. The realistic reproduction of sound sources with spatial extension has become the goal of many sound reproduction methods. This includes binaural reproduction using headphones, as well as conventional reproduction using speaker configurations ranging from two speakers ("stereo") to multiple speakers arranged in a horizontal plane ("surround sound") and multiple speakers surrounding the listener in three dimensions ("3D audio"). The following provides a description of existing methods. The different methods are thus grouped into methods that consider the width of the source in 2D space, respectively, 3D space. Methods are described that relate to rendering SESS on a 2D surface oriented from a listener's viewpoint. This could, for example, be within a certain azimuth range at zero degrees of elevation (as is the case in conventional stereo / surround sound) or within certain azimuth and elevation ranges (as is the case in 3D audio or virtual reality (VR) with 3 degrees of freedom (3DoF) of user movement, i.e., head rotation along pitch / yaw / roll axes). Increasing the apparent width of an audio object that is panned between two or more speakers (generating a so-called ghost image or ghost source) can be achieved by decreasing the correlation of the participating channel signals [1, p.241 -257]. With decreasing correlation, the spread of the phantom source increases, until for correlation values ​​close to zero, the entire interval between the loudspeakers is covered. Decorrelated versions of a source signal are obtained by deriving and applying suitable decorrelation filters. Lauridsen [2] proposed adding / subtracting a delayed and scaled version of the source signal to itself in order to obtain two decorrelated versions of the signal. For example, Kendall proposed more complex approaches [3]. He iteratively derived paired decorrelating all-pass filters based on combinations of random number sequences. Faller et al. propose suitable decorrelation filters (“diffusers”) in [4,5]. Furthermore, Zotter et al. [6] derived filter pairs in which frequency-dependent amplitude or phase differences are used to achieve broadening of a phantom source. Alary et al.[7] proposed velvet noise-based decorrelation filters which were further optimized by Schlecht et al. [8]. In addition to reducing the correlation of the corresponding channel signals of the phantom source, the source width can also be increased by increasing the number of phantom sources attributed to an audio object. In [9], the source width is controlled by panning the same source signal to (slightly) different directions. The method was originally proposed to stabilize the perceived ghost source spread of VBAP-panned source signals

[10] as they move in the sound scene. This is advantageous since depending on the direction of a source, a rendered source is played by two or more speakers, which can result in unwanted alterations of the perceived source width. Virtual world DirAC

[11] is an extension of the traditional Directional Audio Coding (DirAC)

[12] approach for sound synthesis in virtual worlds. To render spatial extent, the directional sound components of a source are randomly panned within a certain interval around the original source direction, where the panning directions vary with time and frequency. A similar approach is pursued in

[13] , where spatial spreading is achieved by randomly distributing frequency bands of a source signal in different spatial directions. This is a method intended to produce a spatially distributed and immersive sound that comes equally from all directions rather than controlling an exact degree of spreading. Verrón et al. achieved spatial spreading of a source not by using panned correlated signals, but by synthesizing multiple incoherent versions of the source signal, distributing them evenly in a circle around the listener, and mixing them together

[14] . The number and gain of simultaneously active sources determines the intensity of the spreading effect. This method was implemented as a spatial extension to an ambient sound synthesizer. Methods are described for rendering extended sound sources in 3D space, i.e., volumetrically, as required for VR with 6DoF user motion. These 6 degrees of freedom include head rotation along the pitch / yaw / roll axes plus three directions of translational motion (x / y / z). Potard et al. extended the notion of source spread as a one-dimensional parameter of the source (i.e., its width between two speakers) by studying the perception of source shapes

[15] . They generated multiple incoherent point sources by applying (time-varying) decorrelation techniques to the original source signal and then placing the incoherent sources at different spatial locations and thus giving them a three-dimensional spread

[16] . In MPEG-4 Advanced AudíoBIFS

[17] , volumetric objects / shapes (pod, box, ellipsoid, and cylinder) can be filled with several equally distributed and decorrelated sound sources to evoke three-dimensional source spread. Recently, Schlecht et al.

[18] proposed an approach that projects the convex hull of the SESS geometry onto the listener position, allowing the SESS to be rendered at any position relative to the listener. Similar to MPEG4 Advanced AudioBIFS, multiple decorrelated point sources are placed within this projection. In order to increase and control source spread using Ambisonics, Schmele et al.

[19] proposed a mixture of reducing the Ambisonic order of an input signal, which inherently increases the apparent source width, and distributing decorrelated copies of the source signal around the listening space. Zotter et al., introduced another approach, where they adopted the principle proposed in [6] (i.e., deriving filter pairs that introduce frequency-dependent phase and magnitude differences to achieve source extension in stereo reproduction setups) for Ambisonics

[20] . A common disadvantage of panning-based approaches (e.g., [10, 9, 12, 11]) is their dependence on the listener's position. Even a small deviation from the optimal point causes the spatial image to collapse on the speaker closest to the listener. This drastically limits their application in the context of virtual reality and augmented reality (AR) where the listener is assumed to move freely. Moreover, the time-frequency bin distribution in DirAC-based approaches (e.g., [12, 11]) does not always guarantee proper rendering of the spatial extent of phantom sources. Moreover, it typically significantly degrades the timbre of the source signal. Decorrelation of source signals is usually achieved by one of the following methods: i) deriving filter pairs with complementary magnitude (e.g., [2]), or ii) using all-pass filters with constant magnitude but (randomly) disordered phase (e.g., [3, 16]). Furthermore, broadening of a source signal is obtained by randomly spatially distributing the time-frequency bins of the source signal (e.g.,

[13] ). All approaches have their own implications: Complementary filtering of a source signal according to i) typically leads to an altered perceived timbre of the uncorrelated signals. While all-pass filtering as in i) preserves the timbre of the source signal, the disordered phase disrupts the original phase relationships and, especially for transient signals, causes severe dispersion and diffusion artifacts. Spatial distribution of time-frequency bins was shown to be effective for some signals, but also alters the perceived timbre of the signal. It proved to be highly signal-dependent and introduces severe artifacts for impulse signals. Filling volumetric shapes with multiple uncorrelated versions of a source signal as proposed in Advanced AudioBIFS ([17,15,16]) assumes the availability of a large number of filters producing mutually uncorrelated output signals (typically, more than ten point sources are used per volumetric shape). However, finding these filters is not a trivial task and becomes more difficult the more these filters are needed. If the source signals are not completely uncorrelated and a listener moves around this shape, e.g., in a VR scenario, the individual source distances to the listener correspond to different delays of the source signals. Their superposition at the listener's ears will result in position-dependent comb filtering, which potentially introduces annoying and unstable coloration of the source signal.Furthermore, the application of many decorrelation filters means great computational complexity. Similar considerations apply to the approach described in

[18] , where several uncorrelated point sources are placed in the convex hull projection of the SESS geometry. While the authors do not mention anything about the required number of uncorrelated auxiliary sources, a potentially large number is needed to achieve a convincing source extension. This leads to the drawbacks already discussed in the previous paragraph. Controlling the source width using the Ambisonics-based technique described in

[19] by reducing the order of Ambisonics was shown to have an audible effect only for transitions from 2o to 1oo 0th order. These transitions are not only perceived as a widening source, but also frequently as a movement of the phantom source. While adding uncorrelated versions of the source signal might help stabilize the perception of the apparent source width, it also introduces comb filter effects, which alter the timbre of the phantom source. Brief Description of the Invention An object of the present invention is to provide an improved concept of synthesizing a spatially extended sound source. This object is achieved by an apparatus for synthesizing a spatially extended sound source of claim 1, a method for synthesizing a spatially extended sound source of claim 23 or a computer program of claim 24. The present invention is based on the finding that a reproduction of a spatially extended sound source can be achieved efficiently by using a spatial range indication indicating a limited spatial target range for a spatially extended sound source within a maximum spatial range. Based on the spatial range indication and, in particular, based on the limited spatial range, one or more track information elements or references are provided, and a processor processes the audio signal representing the spatially extended sound source using the one or more track elements. This process achieves highly efficient processing of the spatially extended sound source. For headphone playback, for example, only two binaural channels are required: one left binaural channel or one right binaural channel. For stereo playback, only two channels are also required.Therefore, in contrast to the synthesis of the spatially extended sound source using a considerable amount of peripheral sound sources which fill the actual volume or area of ​​the spatially extended sound source or, generally, which fill the limited spatial interval due to their individual location, this is not required according to the present invention, since the spatially extended sound source is not rendered using a considerable amount of individual sound sources placed within the volume, but rather the spatially extended sound source is rendered using two or, probably, three channels which have certain tracks between each other that would be obtained, when the high number of peripheral individual sound sources were received at two or three locations. Thus, in contrast to various existing methods that aim to realistically reproduce spatially extended sound sources (SESS), where these existing methods typically require a large number of decorrelated input signals, the present invention goes in a different direction. Generating these decorrelated input signals can be relatively expensive in terms of computational complexity. Previous existing methods can also impair the perceived sound quality through timbre differences or timbre diffusion. And finding a large number of mutually orthogonal decorrelators is generally not an easy problem to solve. Therefore, these prior procedures always result in a trade-off between the degree of mutual decorrelation and the introduced signal degradation, aside from the high computational resources required. In contrast, the present invention synthesizes a resulting low number of channels, such as the resulting left channel and the resulting right channel, for the spatially extended sound source using only two uncorrelated input signals. Preferably, the synthesis result is a left and right ear signal for headphone playback. However, for other types of playback scenarios, such as speaker rendering or active crosstalk reduction speaker rendering, the present invention may also be applied.Instead of placing many different uncorrelated sound signals at different locations within a volume for a spatially extended sound source, the audio signal for the spatially extended sound source consisting of one or more channels is processed using one or more track information elements derived from a track information provider in response to a limited spatial interval indication received from a spatial information interface. Preferred embodiments aim to efficiently synthesize the SESS for headphone playback. The synthesis is therefore based on the underlying model of describing a SESS by an (ideally) infinite number of densely spaced, decorrelated point sources distributed over the entire source spread range. The desired source spread range can be expressed as a function of azimuth and elevation angle, making the inventive method applicable to 3DoF applications. However, an extension to 6DoF applications is possible, by continuously projecting the SESS geometry in the direction towards the current listener position as described in

[18] . As a specific example, the desired source spread is described in the following in terms of azimuth and elevation angle range. Other preferred embodiments are based on the use of an inter-channel correlation value as track information, or additionally utilize an inter-channel phase difference, an inter-channel time difference, an inter-level difference, and a gain factor, or a pair of first and second gain factor information elements. Thus, the absolute channel levels can be determined by two gain factors or a single gain factor and the inter-channel level difference.Any audio filter functions instead of actual track elements or, in addition to actual track elements, may also be provided as track information elements from the track information provider to the audio processor such that the audio processor operates by synthesizing, for example, two output channels such as two binaural output channels or a pair of left and right output channels using an application of an actual track element and, in an olympic manner, filtering using a head-related transfer function for each channel as a track information element or using a head-related impulse response function as a track information element or using a binaural or room (non-binaural) impulse response function as a track information element.Typically, setting only one track element may be sufficient, but in more sophisticated embodiments, the audio processor may impose more than one signal element with or without filters on the audio signals. Therefore, when, in one embodiment, an inter-channel correlation value is provided as a track information element, and where the audio signal comprises a first audio channel and the second audio channel for the spatially extended sound source, or where the audio signal comprises a first audio channel and the second audio channel is derived from the first audio channel by a second channel processor implementing, for example, decorrelation processing or neural network processing or any other processing to derive a signal that can be considered as a decorrelated signal, the audio processor is configured to impose a correlation between the first audio channel and the second audio channel using the inter-channel correlation value and either in addition to or before or after this processing,Audio filter functions may also be applied to ultimately obtain the two output channels having the target inter-channel correlation indicated by the inter-channel correlation value and additionally having the other relationships indicated by the individual filter functions or the other actual track elements. The track information provider may be implemented as a lookup table comprising a memory or as a Gaussian mixture model or as a support vector machine or as a vector codebook, a multidimensional function fitting or some other device that efficiently provides the required tracks in response to a spatial interval indication. It is possible, for example, in the lookup table example, or in the vector codebook or multidimensional feature fitting example, or also in the GMM or SVM example, to already provide prior knowledge so that the main task of the spatial information interface is to actually find the matching candidate spatial interval that best matches, among all available candidate spatial intervals, the input spatial interval indication information. This information can be provided directly via a user or it can be calculated using information about the spatially extended sound source and using a listener position or listener orientation (as for example determined by a head tracker or this device) via some kind of projection calculation.The geometry or size of the object and the distance between the listener and the object may be sufficient to derive the opening angle and, therefore, the limited spatial range for rendering the sound source. In other embodiments, the spatial information interface is merely an input for receiving the limited spatial range and forwarding this data to the track information provider, when the data received by the interface is already in a format usable by the track information provider. Brief Description of the Figures Subsequently, the preferred embodiments of the present invention are analyzed with respect to the attached figures, in which: Figure 1a illustrates a preferred implementation of the apparatus for synthesizing the spatially extended sound source; Figure 1b illustrates another embodiment of the audio processor and track information provider; Figure 2 illustrates a preferred embodiment of a second channel processor included within the audio processor of Figure 1a; Figure 3 illustrates a preferred implementation of a device for performing ICC adjustment; Figure 4 illustrates a preferred embodiment of the present invention where the track information elements depend on actual track elements and filters; Figure 5 illustrates another embodiment that additionally relies on filters and an inter-channel correlation element; Figure 6 illustrates a schematic sector map illustrating a maximum spatial interval in a two-dimensional or three-dimensional situation and individual sectors or limited spatial intervals that can be used, for example, as candidate sectors; Figure 7 illustrates an implementation of the spatial information interface; Figure 8 illustrates another implementation of the spatial information interface that is based on projection calculation procedures; Figures 9a and 9b illustrate ways of performing the projection calculation and determining the spatial interval; Figure 10 illustrates another preferred implementation of the spatial information interface; Figure 11 illustrates a further implementation of the spatial information interface related to a decoder implementation; Figure 12 illustrates the calculation of a limited spatial range for a spherical spatially extended sound source; Figure 13 illustrates additional calculations of limited spatial intervals for a spatially extended ellipsoid sound source; Figure 14 illustrates a further calculation of a limited spatial range for a linear spatially extended sound source; Figure 15 illustrates a further illustration for the calculation of a limited spatial range for a spatially extended cuboid sound source; Figure 16 illustrates a further example for calculating the limited spatial range for a spherical spatially extended sound source; Figure 17 illustrates a spatially extended piano-shaped sound source with an approximate parametric ellipsoid shape; and Figure 18 illustrates points for defining the limited spatial range for rendering the spatially extended piano-shaped sound source. Detailed Description of the Invention Figure 1 illustrates a preferred implementation of an apparatus for synthesizing a spatially extended sound source. The apparatus comprises a spatial information interface 10 receiving a spatial range indication information input indicating a bounded spatial range for the spatially extended sound source within a maximum spatial range. The bounded spatial range is input to a track information provider 200 configured to provide one or more track information items in response to the bounded spatial range given by the spatial information interface 10.The track information element or the plurality of track information elements are provided to an audio processor 300 configured to process an audio signal representing the spatially extended sound source using the one or more track information elements provided by the track information provider 200. The audio signal for the spatially extended sound source may be a single channel, or may be a first audio channel and a second audio channel, or may be more than two audio channels. However, in order to have a low processing load, a small number of channels is preferred for the spatially extended sound source or, for the audio signal representing the spatially extended sound source.The audio signal is input to an audio signal interface 305 of the audio processor 300 and the audio processor 300 processes the input audio signal received by the audio signal interface or, when the number of input audio channels is less than required such as only one, the audio processor comprises a second channel processor 310 illustrated in Figure 2 comprising, for example, a de-correlator for generating a second audio channel S2 de-correlated with the first audio channel S which is also illustrated in Figure 2 as Si.The track information elements may be actual reference elements such as inter-channel correlation elements, inter-channel phase difference elements, inter-channel gain and level difference elements, gain factor elements G1, G2, together representing an inter-channel level difference and / or absolute amplitude or power or energy levels, for example, or the track information elements may also be actual filter functions such as head-related transfer functions with a number as required by the actual number of output channels to be synthesized in the synthesis signal. Thus, when the synthesis signal is to have two channels such as two binaural channels or two speaker channels, a head-related transfer function is required for each channel.Instead of head-related transfer functions, head-related impulse response (HRIR) functions or binaural or non-binaural room impulse response (B)RIR functions are required. As illustrated in Figure 1a, such a transfer function is required for each channel, and Figure 1a illustrates the implementation of having two channels with the indices indicating “1” and “2”. In one embodiment, the track information provider 200 is configured to provide, as a track information element, an inter-channel correlation value. The audio processor 300 is configured to actually receive, via the audio signal interface 305, a first audio channel and a second audio channel. However, when the audio signal interface 305 only receives a single channel, the optionally provided second channel processor generates, for example, by the method in FIG. 2, the second audio channel. The audio processor performs correlation processing to enforce a correlation between the first audio channel and the second audio channel using the inter-channel correlation value. Additionally, or alternatively, an additional track information element may be provided, such as an inter-channel phase difference element, an inter-channel time difference element, an inter-channel level difference and gain element, or a first gain factor and a second gain factor information element. The elements may also be interaural correlation (IACC) values, i.e., more specific inter-channel correlation values, or interaural phase difference (IAPD) elements, i.e., more specific phase difference values ​​between channels. In a preferred embodiment, audio processor 300 imposes correlation in response to the correlation track information element, before any ICPD, ICTD, or ICLD adjustments are made, or before HRTF or other transfer filter function processing is performed. However, depending on the case, the order may be set differently.In a preferred embodiment, the audio processor comprises a memory for storing information about different track information items related to different spatial interval indications. In this situation, the track information provider further comprises an output interface for retrieving, from the memory, the one or more track information items associated with the spatial interval indication entry in the corresponding memory. This lookup table 210 is illustrated, for example, in Figure 1b, 4 or 5, where the lookup table comprises a memory and an output interface for generating the corresponding track information items.In particular, the memory may not only store IACC, IAPD or Gi and Gr values ​​as illustrated in Figure 1b, but the memory within the lookup table may also store filter functions as illustrated in block 220 of Figure 4 and Figure 5 which is indicated as “select HRTF”. In this embodiment, although illustrated separately in Figure 4 and Figure 5, blocks 210, 220 may comprise the same memory where, in association with the corresponding spatial interval indication indicated as azimuth angles and elevation angles, corresponding track information elements such as IACC and optionally IAPD and transfer functions for filters such as HRTFi for the left output channel and HRTFr for the right output channel are stored, where the left and right output channels are indicated as Si and Sr in Figure 4 or Figure 5 or Figure 1b. The memory used by the lookup table 210 or the function selection block 220 may also utilize a storage device where, based on given sector codes or sector angles or sector angle ranges, corresponding parameters are available. Alternatively, the memory may store a vector codebook, a multidimensional function fitting routine, or a Gaussian mixture model (GMM) or a support vector machine (SVM), as the case may be. Given a desired source extension interval, a SESS is synthesized using two uncorrelated input signals. These input signals are processed in such a way that the perceptually important auditory cues are correctly reproduced. This includes the following interaural cues: interaural cross-correlation (IACC), interaural phase differences (IAPD), and interaural level differences (IALD). In addition, monaural spectral cues are reproduced. These are primarily important for localizing the sound source in the vertical plane. While IAPDs and IALDs are also primarily important for localization purposes, the IACC is known to be a crucial cue for the perception of source width in the horizontal plane. During runtime, the target values ​​for these cues are retrieved from a precomputed storage. In the following, a lookup table is used for this purpose.However, any other means of storing multidimensional data could be used, for example, a vector codebook or a multidimensional function fit. Apart from the source extension interval considered, all cues depend only on the Head-Related Transfer Function (HRTF) data set used. A derivation of the different auditory cues is given later. A general block diagram of the proposed method is shown in Figure 1 b. [Φ1?Φ2] describes the desired source extent in terms of azimuth angle range. 02] is the desired source extent in terms of elevation angle range. (ω) and S2(w) denote two uncorrelated input signals, with ω describing the frequency index. For ^(ω) and 52(ω) therefore the following equation holds: E{SiM · S»} = 0. (1) Furthermore, both input signals are required to have the same power spectral density. Alternatively, it is possible to give only one input signal, 5(ω). The second input signal is generated internally using a decorrelator as illustrated in Figure 2. Given ^(ω) and 5Γ(ω), the extended sound source is synthesized by successively adjusting the interchannel coherence (ICC), interchannel phase differences (ICPD), and interchannel level differences (ICLD) to match the corresponding interaural tracks. The quantities required for these processing steps are read from the precomputed lookup table. The resulting left and right channel signals, SL(ω) and Sr(ω), can be played back through headphones and resemble the SESS. It should be noted that the ICC adjustment must be performed first; however, the ICPD and ICLD adjustment blocks can be swapped.Instead of IAPD, the corresponding interaural time differences (IATD) could also be reproduced. However, IAPD is only considered in addition in the following. In the ICC adjustment block, the cross-correlation between both input signals is adjusted to a desired value |IACC(w)| using the following formulas

[21] : 5ι(ω) = Ηα(ω) ^(ω) + Ηβ(ω) S2(w), (2) ί2(ω) = Η Μ 52(ω) + ^(ω) ^(ω), (3) Ηβ^ω) = Ha(W)J|(l-71-|lACC(W)|2), (4) Ηα(ω) = ^Ι-Ηβ(ω). (5) Applying these formulas results in the desired cross-correlation, provided the input signals (ω) and 52(ω) are completely uncorrelated. Furthermore, their power spectral densities must be identical. The corresponding block diagram is shown in Figure 3. The ICPD adjustment block is described by the following formulas: where describes the left ear gain and Gr(u) describes the right ear gain. This results in the desired ICLD provided that S[(w) and 52(ω) have the same power spectral density. Since the left and right ear gain are used directly, monaural spectral cues are reproduced in addition to the IALD. In order to further simplify the method analyzed above, two simplification options are described. As mentioned above, the main interaural cue influencing the perceived spatial extent (in the horizontal plane) is the IACC. Therefore, it would be conceivable not to use precalculated IAPD and / or IALD values, but to adjust them directly via the HRTF. For this purpose, the HRTF corresponding to a representative position of the desired source extent range is used. This position, the average of the desired azimuth / elevation range, is chosen here without loss of generality. A description of both options is given below. The first option involves using precalculated IACC and IAPD values. However, the ICLD is adjusted using the HRTF corresponding to the center of the source extension interval. Figure 4 shows a block diagram of the first option. These are now calculated using the following formulas: Qznzon / zznz / q / υιλι with Φ = (Φ! + Φ2) / 2 and θ = (θ! + 02) / 2 describing the location of an HRTF representing an average of the desired azimuth / elevation interval. The main advantages of the first option include: • No spectral shaping / coloring as source extent increases compared to a point source in the center of the source extent interval. • Lower memory requirements compared to full memory, since yGr(w) do not have to be stored in the lookup table. More flexible to changes in the HRTF dataset during runtime compared to the full method, since only the resulting ICC and ICPD, but not the ICLD, depend on the HRTF dataset used during precomputation. The main disadvantage of this simplified version is that it will fail whenever there are drastic changes in the IALD compared to the unextended source. In this case, the IALD will not be reproduced accurately enough. This is the case, for example, when the source is not centered around the 0 o'clock azimuth, and at the same time, the source extension in the horizontal direction becomes too large. The second option involves using only precalculated IACC values. The ICPD and ICLD are adjusted using the HRTF corresponding to the center of the source extension interval. A block diagram of the second option is shown in Figure 5. 5;(ω) and 5Γ(ω) are now calculated using the following formulas: (12)uncle / In contrast to the first option, the phase and magnitude of the HRTF are now used instead of just the magnitude. This allows for adjusting not only the CIPD but also the CIPD. The main advantages of the second option include: • As for the first option, no spectral shaping / coloring occurs when the source extent increases compared to a point source in the center of the source extent interval. • Even lower memory requirements than for the first option, since <7Γ(ω) and no IAPD need be stored in the lookup table. • Compared to the first option, it is even more flexible to changes in the HRTF dataset during runtime. Only the resulting ICC depends on the HRTF dataset used during precalculation. • Efficient integration into existing binaural rendering systems is possible, since simply two different inputs, ^(ω) and S2(w), have to be used for the generation of left and right ear signals. Regarding the first option, this simplified version will fail whenever there are drastic changes in the IAPD compared to the non-extended source. Furthermore, the changes in IAPD should not be too large compared to the non-extended source. However, since the IAPD of the extended source will be rather close to the IAPD of a point source in the center of the source extension interval, the latter is not expected to be a major problem. Figure 6 illustrates an example of a schematic sector map. In particular, a schematic sector map is illustrated at 600 and the schematic sector map 600 illustrates the maximum spatial range. When considering that the schematic sector map is a two-dimensional illustration of a three-dimensional surface of a sphere, which is intended to show the azimuth and elevation angle ranges of 0° to 360° for the azimuth angle and -90° to +90° for the elevation angle, it becomes clear that, when the schematic sector map is wrapped on a sphere, and the listener position is placed within the center of the sphere, all of the individual sectors illustrated exemplarily by some instances, i.e., S1 to S24, can subdivide an entire spherical surface into sectors.Thus, for example, sector S3 extends, with respect to the azimuth angle range of Φι = 60° up to Φ2 at 90°, when the notation of Figure 1b, Figure 4, Figure 5 is applied. Sector S3 extends for example within the elevation angle range between -30° and 0°. However, the schematic sector map 600 can also be used when the listener is not positioned within the center of the sphere, but rather at a specific position relative to the sphere. In this case, only certain sectors of the sphere are visible, but it is not necessary that certain elements of track information be available for all sectors of the sphere. It is only necessary that, for some (required) sectors, certain elements of track information be available, which are preferably precalculated as discussed below or, alternatively, obtained by measurements. Alternatively, the schematic sector map can be viewed as a two-dimensional maximum interval where a spatially extended sound source can be located. In this situation, the horizontal distance extends between 0% and 100%, and the vertical distance extends between 0% and 100%. The actual vertical distance or extent and the actual horizontal distance or extent can be mapped to absolute distances or extents using a given absolute scaling factor. When, for example, the scaling factor is 10 meters, 25% would correspond to 2.5 meters in the horizontal direction. In the vertical direction, the scaling factors can be the same as or different from the scaling factor in the horizontal direction.Thus, for the horizontal / vertical distance / extent example, sector S5 would extend, with respect to the horizontal dimension, between 33% and 42% of the (maximum) scale modification factor, and sector S5 would extend, within the vertical interval, between 33% and 50% of the vertical scale modification factor. Thus, a spherical or non-spherical maximum spatial interval can be subdivided into limited sectors or spatial intervals S1 to S24, for example. In order to efficiently adapt the grid to human hearing perception, it is preferred to have a low resolution within the vertical or elevation direction and a higher resolution within the horizontal or azimuth direction. For example, only sectors of a sphere that cover the entire elevation range can be used, which would mean that only a line of sectors extending, for example, from S1 to S12 is available as different sectors or limited spatial intervals where the horizontal dimensions are given by the determined angular values ​​and the vertical dimension extends from -90° to +90° for each sector.Naturally, other sectorization techniques are also available, for example, we have in the example of Figure 6, 24 sectors where sectors S1 to S12 cover, for each sector, the entire elevation or vertical interval between -90° and 0° between 0% and 50%, where the other sectors S13 to S24 cover the upper hemisphere between elevation angles of 0° to 90° or cover the upper half of the horizon extending between 50% and 100%. Figure 7 illustrates a preferred implementation of a spatial information interface 10 of Figure 1a. In particular, the spatial information interface comprises an actual receiving (user) interface for receiving the spatial interval indication. The spatial interval indication may be input by the user himself or may be derived from head tracker information in case a virtual reality or augmented matcher 30 matches the actually received limited spatial interval with available candidate spatial intervals that are known from the track information provider 200 in order to find a matching candidate spatial interval that is closest to the input actually limited spatial interval. Based on this matching candidate spatial interval, the track information provider 200 of Figure 1a supplies the one or more track information elements such as inter-channel data or filter functions.The candidate matching spatial interval or the limited spatial interval may comprise a pair of azimuth angles or a pair of elevation angles or both as illustrated, for example, in Figure 1b, which shows an azimuth interval and an elevation interval for a sector. Alternatively, as illustrated in Figure 6, the bounded spatial range may be bounded by horizontal distance information, vertical distance information, or both vertical and horizontal distance information. When the maximum spatial range is swept in two dimensions, not only a single vertical or horizontal distance is sufficient, but a pair of vertical and horizontal distances is necessary, as illustrated with respect to slice S5. Again alternatively, the bounded spatial range information may comprise a code identifying the bounded spatial range as a specific slice of the maximum spatial range where the maximum spatial range comprises a plurality of different slices.This code is, for example, given by the indications S1 to S24, since each code is associated only with a certain two-dimensional or three-dimensional geometric sector in the schematic sector map 600. Figure 8 illustrates a further implementation of a spatial information interface consisting, again, of the user receiving interface 100 but now additionally consisting of a projection calculator 120 and a downstream connected spatial interval determiner 140. The user receiving interface 100 exemplarily receives the listener position where the listener position comprises the actual location of the user in a certain environment and / or the orientation of the user at the certain location. Thus, a listener position may be related to the actual location or the actual orientation or both the actual location of the listener and the actual orientation of the listener. Based on this data, a projection calculator 120 calculates, using information about the spatially extended sound source, so-called envelope projection data.The SESS information may comprise the geometry of the spatially extended sound source and / or the position of the spatially extended sound source and / or the orientation of the spatially extended sound source, etc. Based on the envelope projection data, the spatial range determiner 140 determines the bounded spatial range in one of the alternatives illustrated in Figure 6, or as discussed with respect to Figures 10, 11 or Figure 12 to Figure 18, where the bounded spatial range is given by two or more feature points illustrated in the examples between Figure 12 and Figure 18, where the set of feature points always defines a certain bounded spatial range of a complete spatial range. Figure 9a and Figure 9b illustrate different ways of calculating the envelope projection data output by block 120 of Figure 8. In the embodiment of Figure 9a, the spatial information interface is configured to calculate the envelope of the spatially extended sound source using, as the information about the spatially extended sound source, the geometry of the spatially extended sound source as indicated by block 121. The envelope of the spatially extended sound source is projected 122 towards the listener using the listener position to obtain the projection of the two-dimensional or three-dimensional envelope on a projection plane.Alternatively, as illustrated in Figure 9b, the spatially extended sound source and, in particular, the spatially extended sound source geometry as defined by the spatially extended sound source geometry information is projected in a direction toward the listener position illustrated in block 123, and the envelope of a projected geometry is calculated as indicated in block 124 to obtain the projection of the two-dimensional or three-dimensional envelope on the projection plane. The limited spatial range represents the vertical / horizontal or azimuth / elevation extent of the projected envelope in the embodiment of Figure 9a or of the envelope of the projected geometry as obtained by the implementation of Figure 9b. Figure 10 illustrates a preferred implementation of the spatial information interface 10. It comprises a listener position interface 100 which is also illustrated in Figure 8 as the user receiving interface. In addition, the position and geometry of the spatially extended sound source are input as also illustrated in Figure 8. A projector 120 and calculator 140 are provided for calculating the limited spatial range. Figure 11 illustrates a preferred implementation of a spatial information interface comprising an interface 100, a projector 120, and a limited spatial range location calculator 140. The interface 100 is configured to receive a listener position. The projector 120 is configured to calculate a projection of a two- or three-dimensional envelope associated with the spatially extended sound source onto a projection plane using the listener position as received by the interface 100 and additionally using information about the geometry of the spatially extended sound source, and additionally using information about the position of the spatially extended sound source in space.Preferably, the defined position of the spatially extended sound source in space and, additionally, the geometry of the spatially extended sound source in space are received by a bit stream arriving at a bit stream demultiplexer or scene analyzer 180. The bit stream demultiplexer 180 extracts, from the bit stream, information about the geometry of the spatially extended sound source and provides this information to the projector. The bit stream demultiplexer also extracts the position of the spatially extended sound source from the bit stream and returns this information to the projector. Preferably, the bitstream also comprises the audio signal for the SESS having one or two different audio signals, and preferably the bitstream demultiplexer also extracts, from the bitstream, a compressed representation of the one or more audio signals, and the signals are decompressed / decoded by a decoder such as an audio decoder 190. The one or more decoded signals are finally forwarded to the audio processor 300 of Figure 1a, for example, and the processor renders the at least two sound sources in line with the track elements provided by the track information provider 200 of Figure 1a. Although Figure 11 illustrates a bitstream-related playback apparatus having a bitstream demultiplexer 180 and an audio decoder 190, playback may also take place in a situation other than an encoder / decoder scenario. For example, the defined position and geometry in space may already exist in the playback apparatus such as in a virtual reality or augmented reality scene, where data is generated on-site and consumed on-site. The bitstream demultiplexer 180 and audio decoder 190 are not actually needed, and the information of the spatially extended sound source geometry and the spatially extended sound source position are available without any extraction from a bitstream. Subsequently, further preferred embodiments of the present invention are discussed. These embodiments relate to rendering spatially extended sound sources in 6DoF VR / AR (virtual reality / augmented reality). Preferred embodiments of the invention are directed to a method, apparatus, or computer program designed to enhance the reproduction of spatially extended sound sources (SESS). In particular, embodiments of the inventive method or apparatus account for the time-varying relative position between the spatially extended sound source and the virtual listener position. In other words, embodiments of the inventive method or apparatus allow the width of the auditory source to match the spatial extent of the represented sound object at any position relative to the listener. As such, one embodiment of the inventive method or apparatus is particularly applicable to 6-degrees-of-freedom (6DoF) virtual, mixed, and augmented reality applications where the spatially extended sound source complements traditionally employed point sources. The embodiment of the inventive method or apparatus renders a spatially extended sound source using a limited spatial range. The limited spatial range depends on the listener's position relative to the spatially extended sound source. Figure 1a illustrates the overview block diagram of a spatially extended sound source renderer according to the embodiment of the inventive method or apparatus. The key components of the block diagram are: Qznzon / zznz / q / υιλι 1. Listener Position: This block provides the current position of the listener, as measured, for example, by a virtual reality tracking system. The block can be implemented as a detector 100 for detecting or an interface 100 for receiving the listener position. 2. Position and geometry of the spatially extended sound source: This block provides the position and geometry data of the spatially extended sound source to be rendered, for example, as part of a virtual reality scene rendering. 3. Projection and Convex Hull Calculation: This block 120 calculates the convex hull of the spatially extended sound source geometry and then projects it toward the listener position (e.g., “image plane,” see below). Alternatively, the same function can be achieved by first projecting the geometry toward the listener position and then calculating its convex hull. 4. Location of the bounded spatial interval determination: This block 140 calculates the location of the bounded spatial interval from the convex hull projection data calculated by the previous block. This calculation may also consider the listener position and, therefore, the proximity / distance of the listener (see below). The outputs are, for example, point locations that collectively define the bounded spatial interval. Figure 10 illustrates a block diagram overview of one embodiment of the inventive method or apparatus. The dashed lines indicate the transmission of metadata such as geometry and positions. The locations of the points collectively defining the limited spatial interval depend on the geometry, in particular the spatial extent, of the spatially extended sound source and the listener's relative position with respect to the spatially extended sound source. In particular, the points defining the limited spatial interval can be located on the projection of the convex hull of the spatially extended sound source onto a projection plane. The projection plane can be an image plane, i.e., a plane perpendicular to the line of sight from the listener to the spatially extended sound source, or a spherical surface around the listener's head. The projection plane is located an arbitrarily small distance from the center of the listener's head.Alternatively, the projection convex hull of the spatially extended sound source can be calculated from azimuth and elevation angles, which are a subset of the spherical coordinates with respect to the listener's head perspective. In the illustrative examples below, the projection plane is preferred due to its more intuitive nature. In the implementation of the projected convex hull calculation, the angular representation is preferred due to its simpler formalization and lower computational complexity. Both the convex hull projection of the spatially extended sound source and the convex hull of the projected spatially extended sound source geometry are identical; that is, the convex hull calculation and the projection onto an image plane can be used in either order. When the listener's position relative to the spatially extended sound source changes, the projection of the spatially extended sound source onto the projection plane changes accordingly. In turn, the locations of the points defining the limited spatial interval change accordingly. The points will preferably be chosen so that they change smoothly for the continuous motion of the spatially extended sound source and the listener. The projected convex hull changes when the geometry of the spatially extended sound source changes. This includes rotating the geometry of the spatially extended sound source in 3D space, which alters the projected convex hull.The rotation of the geometry is equal to an angular displacement of the listener's position relative to the spatially extended sound source and is referred to inclusively as the relative position of the listener and the spatially extended sound source. For example, a circular motion of the listener around a spherical spatially extended sound source is represented by rotating the points defining the limited spatial interval around the center of gravity. Similarly, rotating the spatially extended sound source with a stationary listener produces the same shift in the points defining the limited spatial interval. The spatial extent generated by the inventive method or apparatus is inherently reproduced correctly for any distance between the spatially extended sound source and the listener. Naturally, as the user approaches the spatially extended sound source, the opening angle between the points defining the limited spatial interval change increases as appropriate for modeling physical reality. Therefore, the angular placement of the points defining the bounded spatial interval is uniquely determined by the location on the convex hull projected onto the projection plane. To specify the geometric shape / convex hull of the spatially extended sound source, an approximation is used (and possibly passed to the renderer core or renderer) that includes a simplified 1D shape, e.g., line, curve; 2D shape, e.g., ellipse, rectangle, polygons; or 3D shape, e.g., ellipsoid, cuboid, and polyhedra. The geometry of the spatially extended sound source or the corresponding approximate shape, respectively, can be described in a variety of ways, including: • Parametric description, that is, a formalization of the geometry through a mathematical expression that accepts additional parameters. For example, a 3D ellipsoid shape can be described by an implicit function in the Cartesian coordinate system, with additional parameters including the extension of the principal axes in the three directions. Other parameters may include 3D rotation and deformation functions of the ellipsoid surface. • Polygonal description, that is, a collection of primitive geometric shapes such as lines, triangles, squares, tetrahedrons, and cuboids. Polygons, prisms, and polyhedra can be concatenated into larger, more complex geometries. In certain application scenarios, the focus is on the compact and interoperable transmission / storage of 6DoF VR / AR content. In this case, the entire chain consists of three steps: 1. Authoring / encoding the desired spatially extended sound sources into a bit stream 2. Transmission / storage of the generated bitstream. According to the presented invention, the bitstream contains, among other elements, the description of the spatially extended sound source geometries (parametric or polygonal) and the associated source base signal(s), such as a monophonic or stereophonic piano recording. The waveforms can be compressed using perceptual audio coding algorithms, such as Advanced Audio Coding (AAC) MP3 or MPEG-2 / 4. 3. Decoding / rendering the spatially extended sound sources based on the transmitted bitstream as described above. Subsequently, several practical implementation examples are presented. These include a spherical spatially extended sound source, an ellipsoid spatially extended sound source, a linear spatially extended sound source, a cuboid spatially extended sound source, limited distance-dependent spatial intervals, and / or a piano-shaped spatially extended sound source or a spatially extended sound source shaped like any other musical instrument. As described in the embodiments of the inventive method or apparatus above, various methods can be applied to determine the location of the points defining the limited spatial range. The following practical examples demonstrate some isolated methods in specific cases. In a complete implementation of the embodiment of the inventive method or apparatus, the various methods can be combined as appropriate, taking into account computational complexity, application purpose, audio quality, and ease of implementation. The spatially extended sound source geometry is indicated as a surface mesh. It should be noted that the mesh display does not imply that the spatially extended sound source geometry is described by a polygonal method, since in fact the spatially extended sound source geometry can be generated from a parametric specification. The listener position is indicated by a blue triangle. In the following examples, the image plane is chosen as the projection plane and is represented as a transparent gray plane indicating a finite subset of the projection plane. The projected geometry of the spatially extended sound source on the projection plane is illustrated with the same surface mesh. The points defining the limited spatial range in the projected convex hull are illustrated as crosses on the projection plane.The backprojected points defining the limited spatial interval in the spatially extended sound source geometry are illustrated as dots. The corresponding points defining the limited spatial interval in the projected convex hull and the backprojected points defining the limited spatial interval in the spatially extended sound source geometry are connected by lines to aid in identifying the visual correspondence. The positions of all objects involved are represented in a Cartesian coordinate system with units of meters. The choice of the coordinate system represented does not imply that the calculations involved are performed with Cartesian coordinates. The first example in Figure 12 considers a spherical spatially extended sound source. The spherical spatially extended sound source has a fixed size and a fixed position relative to the listener. Three different sets of three, five, and eight points are chosen that define the bounded spatial interval on the projected convex hull. The three sets of points defining the bounded spatial interval are chosen at uniform distances from each other on the convex hull curve. The offset positions of the points defining the bounded spatial interval on the convex hull curve are deliberately chosen so that the horizontal extent of the spatially extended sound source geometry is well represented.Figure 12 illustrates a spherical spatially extended sound source with different numbers (i.e., 3 (top), 5 (center), and 8 (bottom)) of points defining the limited spatial range uniformly distributed in the convex hull. The following example in Figure 13 considers a spatially extended ellipsoid sound source. The spatially extended ellipsoid sound source has a fixed shape, position, and rotation in 3D space. Four points are chosen that define the limited spatial range in this example. Three different methods for determining the location of the points that define the limited spatial range are exemplified: a) Two points defining the bounded spatial interval are placed at the two horizontal endpoints, and two points defining the bounded spatial interval are placed at the two vertical endpoints. Whereas, extreme point positioning is simple and often appropriate. This example shows that this method can provide point locations that are relatively close to each other. b) The four points defining the bounded spatial interval are uniformly distributed on the projected convex hull. The offset of the points defining the bounded spatial interval location is chosen so that the upper point location coincides with the upper point location in a). c) The four points defining the bounded spatial interval are uniformly distributed on a shrunken projected convex hull. The offset location of the point locations is equal to the offset location chosen in b). The shrinking operation of the projected convex hull is performed toward the centroid of the projected convex hull with a direction-independent stretch factor. Thus, Figure 13 illustrates a spatially extended ellipsoid sound source with four points defining the bounded spatial interval under three different methods for determining the location of the points defining the bounded spatial interval: a / top) horizontal and vertical extreme points, b / middle) points uniformly distributed on the convex hull, c / bottom) points uniformly distributed on a shrunken convex hull. The following example in Figure 14 considers a linear spatially extended sound source. While the previous examples considered volumetric spatially extended sound source geometry, this example demonstrates that the spatially extended sound source geometry may well be chosen as a one-dimensional object within 3D space. Subfigure a) illustrates two points defining the bounded spatial interval placed at the endpoints of the finite linear spatially extended sound source geometry, b) Two points defining the bounded spatial interval are placed at the endpoints of the finite linear spatially extended sound source geometry and an additional point is placed in the middle of the line.As described in embodiments of the inventive method or apparatus, placing additional points within the spatially extended sound source geometry can help fill large gaps in large spatially extended sound source geometries. c) The same linear spatially extended sound source geometry is considered as in a) and b), however, the relative angle toward the listener is altered such that the projected length of the linear geometry is considerably smaller. As described in embodiments of the inventive method or apparatus above, the reduced size of the projected convex hull can be represented by a reduced number of points defining the bounded spatial range, in this particular example, by a single point located at the center of the linear geometry. Thus, Figure 14 illustrates a linear spatially extended sound source with three different methods of distributing the location of the points defining the limited spatial interval: a / top) two extreme points on the projected convex hull; b / middle) two extreme points on the projected convex hull with one additional point at the center of the line; c / bottom) one or two points defining the limited spatial interval at the center of the convex hull as the projected convex hull of the rotated line is too small to allow more than one or two points. The following example in Figure 15 considers a cuboid spatially extended sound source. The cuboid spatially extended sound source has fixed size and fixed location, however, the relative position of the listener changes. Subfigures a) and b) illustrate different methods for placing four points defining the bounded spatial interval on the projected convex hull. The backprojected point locations are uniquely determined by the choice on the projected convex hull, c) illustrates four points defining the bounded spatial interval that do not have well-separated backprojection locations. Instead, the distances of the point locations are chosen equal to the centroid distance of the spatially extended sound source geometry. Thus, Figure 15 illustrates a cuboid spatially extended sound source with three different methods for distributing the points defining the bounded spatial range: a / top) two points defining the bounded spatial range on the horizontal axis and two points defining the bounded spatial range on the vertical axis; b / middle) two points defining the bounded spatial range at the horizontal endpoints of the projected convex hull and two points defining the bounded spatial range at the vertical endpoints of the projected convex hull; c / bottom) backprojection point distances are chosen to be equal to the centroid distance of the spatially extended sound source geometry. The following example in Figure 16 considers a spatially extended spherical sound source of fixed size and shape, but at three different distances from the listening position. The points defining the limited spatial range are uniformly distributed on the convex hull curve.The number of points defining the bounded spatial interval is dynamically determined from the length of the convex hull curve and the minimum distance between possible point locations, a) The spherical spatially extended sound source is located at a close distance such that four points defining the bounded spatial interval are chosen on the projected convex hull, b) The spherical spatially extended sound source is located at a medium distance such that three points defining the bounded spatial interval are chosen on the projected convex hull, a) The spherical spatially extended sound source is located at a far distance such that only two points defining the bounded spatial interval are chosen on the projected convex hull.As described in embodiments of the inventive method or apparatus above, the number of points defining the limited spatial range may also be determined from the extent represented in spherical angular coordinates. Thus, Figure 16 illustrates a spherical spatially extended sound source of the same size but at different distances: a / top) near distance with four points defining the bounded spatial range uniformly distributed on the projected convex hull; b / middle) middle distance with three points defining the bounded spatial range uniformly distributed on the projected convex hull; c / bottom) far distance with two points defining the bounded spatial range uniformly distributed on the projected convex hull. The final example in Figures 17 and 18 considers a spatially extended sound source in the form of a piano placed within a virtual world. The user wears a head-mounted display (HMD) and headphones. The user is presented with a virtual reality scene consisting of an open-world space and a vertical 3D upright piano model on the floor within the free-movement area (see Figure 17). The open-world space is a spherical static image projected onto a sphere surrounding the user. In this particular case, the open-world space represents a blue sky with white clouds. The user is able to walk around and see and hear the piano from various angles.In this scene, the piano is rendered using cues that represent a single point source placed at the center of gravity or that represent a spatially extended sound source with three points defining the limited spatial range in the projected convex hull (see, Figure 18). To simplify the point computation, the piano geometry is abstracted to an ellipsoid shape with similar dimensions, see Fig. 17. Two surrogate points are placed at extreme left and right points on the equatorial line, while the third surrogate point remains at the north pole, see Fig. 18. This arrangement ensures the appropriate horizontal font width from all angles at a greatly reduced computational cost. Thus, Figure 17 illustrates a spatially extended piano-shaped sound source with an approximate parametric ellipsoid shape, and Figure 18 illustrates a spatially extended piano-shaped sound source with three points defining the bounded spatial range distributed across the vertical end points of the projected convex hull and the vertical top position of the projected convex hull. It is noted that for ease of visualization, the points defining the bounded spatial range are placed on a stretched projected convex hull. The application of the described technology can be as part of a 6DoF VR / AR Audio standard. In this context, one has the classic encoder / bitstream / decoder (+renderer) scenario: • In the encoder, the shape of the spatially extended sound source would be encoded as secondary information together with the 'base' waveforms of the spatially extended sound source which may be • a monaural signal, or • a stereo signal (preferably sufficiently uncorrelated), or • even more recorded signals (also preferably sufficiently uncorrelated) characterizing the spatially extended sound source. These waveforms could be encoded with Qznzon / zznz / q / υιλι low bit rate. • In the decoder / renderer, the spatially extended sound source shape and corresponding waveforms are recovered from the bitstream and used to render the spatially extended sound source as described above. Depending on the embodiments used, and as alternatives to the embodiments described, it will be noted that the interface can be implemented as an actual tracker or detector for detecting a listener position. However, the listener position will typically be received from an external tracker device and fed into the playback device via the interface. However, the interface can represent a data input for the output data of an external tracker, or it can also represent the tracker itself. As outlined, the bitstream generator can be implemented to generate a bitstream with only one audio signal for the spatially extended sound source, with the remaining audio signals being generated on the decoder or playback side via decorrelation. When only a single signal exists, and when the entire space is to be filled equally with this single signal, no location information is required. However, in this situation, it can be useful to have at least additional information about the geometry of the spatially extended sound source. Depending on the implementation, it is preferred to use, within the track information provider 200 of Figures 1a, 1b, 4, 5, some form of pre-computed data to have the correct track information elements for a given environment. This pre-computed data, i.e., the set of values ​​for each sector such as from the sector map 600 of Figure 6, can be measured and stored such that the data within, for example, the lookup table 210 and the selected HRTF blocks 220 are determined empirically. In another embodiment, this data can be pre-computed, or the data can be derived in a mixed empirical and pre-computation method. The preferred embodiment for calculating this data is given below. During lookup table generation, the IACC, IAPD, and IALD values ​​required for SESS synthesis, as described above, are precalculated for a number of source extension intervals. As mentioned above, as an underlying model, the SESS is described by an infinite number of uncorrelated point sources distributed across the entire source spread interval. This model is approximated here by placing a correlated uncorrelated point source at each HRTF dataset position within the desired source spread interval. By convolving these signals with the corresponding HRTF, the resulting left and right ear signals (Υ^ω), respectively Υτ(ω), can be determined. From these, the IACC, IAPD, and IALD values ​​can be obtained. A derivation of the corresponding expressions is given below. Given N uncorrelated signals ^(ω) with equal power spectral density: with (14) rznzon / zznz / q / υιλι where N equals the number of HRTF dataset points within the desired source spread interval. Therefore, each of these N input signals is placed at a different HRTF dataset position, with him; = / / i.....a.(16)HKH-VU / / ) = Λ„·^'·. / / 1.....;V Note: Aln, Arn, Φί η, and Al n, in general depend on ω. However, this dependence is omitted here for notational simplicity. Using equations (16), (17), the left and right ear signals, Υι(ω) respectively(ω), can be expressed as follows: TO' (18) TO' To determine the IACC, IALD and IAPD, the first expressions for Ε{K,(ω) / / (ω)}, Ε{|ν;(ω)|2} and E{|ΥΓ(ω)|2} are derived: ( \ .V ε{ΕΑ·) r;(^)} e .i,„ · c'- · s„w · .i,,„ < .....· sm η = 1 you = 1 (20) Using equation (20) to (22), the following expressions can be determined for ΙΑΟΟ(ω), IALD (ω) and ΙΑΡϋ(ω): IACC(cv) \ / Σ„=ι „ Σ,„=ι 4 / ,,, (23) IALD (te) = 10 log WW) Ε{|4Σ)|2} V 42n] 4-^11=1 l.η = 101ogln, 4„=!(24)iapdm = z(E{wjk'%)}) = z (IACCM) The left and right ear gain, όζ(ω) respectively Gr(w), are determined by normalizing Ε{|Τζ(ω)|2respectively E{|Yr(ω) |2by the number of sources as well as the source energy: As you can see, all resulting expressions depend solely on the chosen HRTF dataset and no longer depend on the input signals. To reduce computational complexity during lookup table generation, one option is to not consider every position in the available HRTF dataset. In this case, a desired spacing is defined. While this procedure reduces computational complexity during precalculation, it will also lead to a degradation of the solution to some extent. Preferred embodiments of the present invention provide significant advantages compared to the prior art. Since the proposed method requires only two uncorrelated input signals, several advantages arise compared to the current state of the art that requires a larger number of uncorrelated input signals: • The proposed method exhibits lower computational complexity, since only one decorrelator needs to be applied. Furthermore, only two input signals need to be filtered. • Since pairwise decorrelation is typically greater when fewer decorrelated signals are generated (while allowing for the same amount of signal degradation), more accurate reproduction of auditory cues is expected. • Similarly, more signal degradations are expected to achieve the same amount of pairwise decorrelation and therefore the same accuracy of the reproduced auditory cues. Subsequently, several interesting features of embodiments of the present invention are summarized. 1. Only two uncorrelated input signals are needed (or one input signal plus a decorrelator). 2. [Frequency-selective] tuning of the binaural tracks of these input signals to efficiently achieve binaural output signals for the spatially extended sound source (instead of modeling many individual point sources covering the area / volume of the SESS) (a ) The input ICCs are always tuned. b) ICPD / ICTD and ICLD can either be adjusted in a specific processing step or can be introduced into the signals when using HRIR / HRTF processing with these features. 3. The target [frequency-selective] binaural tracks are determined from a pre-computed storage (lookup table or other multidimensional data storage medium such as a vector codebook or multidimensional function fitting, GMM, SVM) as a function of the spatial interval to be filled (specific example: azimuth interval, elevation interval) (a) The target IACCs are always stored and retrieved / used for synthesis. (b) The target IAPD / IATD and IALD can be stored and retrieved / used for synthesis or replaced when using HRIR / HRTF processing. A preferred implementation of the present invention may be as part of an MPEG-I Audio 6 DoF VR / AR (virtual reality / augmented reality) standard. In this context, one has an encoder / bitstream / decoder (plus renderer) application scenario. In the encoder, the shape of the spatially extended sound source or the multiple spatially extended sound sources would be encoded as side information together with the (one or more) spatial waveforms of the spatially extended sound source. These waveforms representing the signal input at block 300, i.e. the audio signal for the spatially extended sound source could be low bit rate encoded by means of an AAC, EVS or any other encoder.In the decoder / renderer, where an application is illustrated, for example, in Figure 11 as comprising a bitstream demultiplexer (parser 180 and an audio decoder 190), the SESS shape and corresponding waveforms are recovered from the bitstream and used to render the SESS. The methods illustrated with respect to the present invention provide a high quality, yet low complexity decoder / renderer. Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block, element, or feature of a corresponding apparatus. Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be realized using a digital storage medium, e.g., a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM, or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, so that one of the methods described herein is carried out. In general, embodiments of the present invention may be implemented as a computer program product with program code, the program code being operative to perform one of the methods, when the computer program product is executed on a computer. The program code may, for example, be stored on a machine-readable medium. Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine-readable carrier or a non-transitory storage medium. In other words, one embodiment of the inventive method is, therefore, a computer program having program code for performing one of the methods described herein, when the computer program is executed on a computer. A further embodiment of the inventive methods is therefore a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. A further embodiment of the inventive method is therefore a data stream or sequence of signals representing the computer program for performing one of the methods described herein. The data stream or sequence of signals may, for example, be configured to be transferred via a data communication connection, e.g., via the Internet. A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein. One embodiment further comprises a computer having installed thereon the computer program for performing one of the methods described herein. In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to implement some or all of the functionalities of the methods described herein. In some embodiments, a field-programmable gate array may cooperate with a microprocessor to implement one of the methods described herein. In general, the methods are preferably implemented by any hardware apparatus. The embodiments described above are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended to be limited only by the scope of the patent claims herein and not by the specific details presented by way of description and explanation of the embodiments herein. References [1] J. Blauert, Spatial Hearing: Psychophysics of Human Sound Localization, 3rd ed. Cambridge, Mass: MIT Press, 2001. [2] H. Lauridsen, “Experiments Concerning Different Kinds of Room-Acoustics Recording,” Ingenioren, 1954. [3] G. Kendall, “The Decorrelation of Audio Signáis and Its Impact on Spatial Imagery,” Computer Musió Journal, vol. 19, no.4, pp. 71-87, 1995. [4] C. Faller y F. Baumgarte, “Binaural cue coding-Part II: Schemes and applications,” IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, pp. 520-531, Nov. 2003. [5] F. Baumgarte y C. Faller, “Binaural cue coding-Part I: Psychoacoustic fundamentáis and design principies,” IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, pp. 509-519, Nov. 2003. [6] F. Zotter y M. Frank, “Efficient Phantom Source Widening,” Archives ofAcoustics, vol. 38, pp. 27-37, Mar. 2013. [7] B. Alary, A. Politis, y V. Valimáki, “Velvet-noise decorrelator,” Proc. DAFx-17, Edimburgo, Reino Unido, pp. 405-411, 2017. [8] S. Schlecht, B. Alary, V. Valima'ki, y E. Habets, “Optimized velvet-noise decorrelator,” Sep. 2018. [9] V. Pulkki, “Uniform spreading of amplitude panned virtual sources,” Proceedings of the 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. WASPAA’99 (Cat. No. 99TH8452), págs. 187-190,1999.

[10] ----, “Virtual Sound Source Positioning Using Vector Base Amplitude Panning,” Journal ofthe Audio Engineering Society, vol. 45, no. 6, págs. 456 a 466, junio de 1997.

[11] V. Pulkki, M.-V. Laitinen, y C. Erkut, “Efficient Spatial Sound Synthesis for Virtual Worlds”. Audio Engineering Society, febrero de 2009.

[12] V. Pulkki, “Spatial Sound Reproduction with Directional Audio Coding,” Journal ofthe Audio Engineering Society, vol. 55, no. 6, págs. 503 a 516, junio de 2007.

[13] T. Pihlajam aki, O. Santala, y V. Pulkki, “Synthesis of Spatially Extended Virtual Source with Time-Frequency Decomposition of Mono Signáis,” Journal ofthe Audio Engineering Society, vol. 62, no. 7 / 8, págs. 467 a 484, agosto de 2014.

[14] C. Verrón, M. Aramaki, R. Kronland-Martinet, y G. Pallone, “A 3-D Immersive Synthesizer for Environmental Sounds,” Audio, Speech, andLanguage Processing, IEEE Transactions on, vol. 18, págs. 1550-1561, Sep. 2010.

[15] G. Potard e I. Burnett, “Astudy on sound source apparent shape and wideeness,” pp. 6-9, agosto de 2003.

[16] —, “Decorrelation techniques for the rendering of apparent sound source width in 3D audio displays”, enero de 2004, págs. 280-208.

[17] J. Schmidt y E. F. Schroeder, “New and Advanced Features for Audio Presentation in the MPEG-4 Standard”. Audio Engineering Society, mayo de 2004.

[18] S. Schlecht, A. Adami, E. Habets y J. Herre, “Apparatus and Method for Reproducing a Spatially Extended Sound Source or Apparatus and Method for Generating a Bitstream from a Spatially Extended Sound Source”, Solicitud de patente PCT / EP2019 / 085 733.

[19] T. Schmele y U. Sayin, “Controlling the Apparent Source Size in Ambisonics Using Decorrelation Filters.” Audio Engineering Society, julio de 2018.

[20] F. Zotter, M. Frank, M. Kronlachner, y J.-W. Choi, “Efficient Phantom Source Widening and Diffuseness in Ambisonics”, enero de 2014.

[21] C. ΒοΓβ, “An Improved Parametric Model for the Design of Virtual Acoustics and its Applications,” tesis doctoral, RuhrUniversit” en Bochum, enero de 2011.

Claims

CLAIMS 1. An apparatus for synthesizing a spatially extended sound source, comprising: a spatial information interface for receiving a spatial range indication indicating a limited spatial range for the spatially extended sound source within a maximum spatial range; a track information provider for providing one or more track information elements in response to the limited spatial range; and an audio processor for processing an audio signal representing the spatially extended sound source using the one or more track information elements.

2. Apparatus according to claim 1, wherein the track information provider is configured to provide, as a track information element, an interchannel correlation value, wherein the audio signal comprises a first audio channel and a second audio channel for the spatially extended sound source, or wherein the audio signal comprises a first audio channel and a second audio channel is derived from the first audio channel by means of a second channel processor, and wherein the audio processor is configured to impose a correlation between the first audio channel and the second audio channel using the interchannel correlation value.

3. Apparatus according to claim 1 or 2, wherein the track information provider is configured to provide, as an additional track information element, at least one of an interchannel phase difference element, an interchannel time difference element, an interchannel level difference, and a gain element, and a first gain and a second gain information element, wherein the audio signal comprises a first audio channel and a second audio channel for the spatially extended sound source, or wherein the audio signal comprises a first audio channel and a second audio channel derived from the first audio channel by means of a second channel processor, and wherein the audio processor is configured to impose an interchannel phase difference,an interchannel time difference or an interchannel level difference or absolute levels of the first audio channel and the second audio channel using at least one of the interchannel phase difference element, the interchannel time difference element, the interchannel level difference and a gain element, and the first and second gain elements.

4. Apparatus according to claim 1 or 2, wherein the audio processor is configured to impose a correlation between the first channel and the second channel and, after the correlation has been determined, to impose the interchannel phase difference, interchannel time difference, or interchannel level difference or the absolute levels of the first channel and the second channel, or wherein the second channel processor comprises a decorrelation filter or a neural network processor for deriving, from the first audio channel, the second audio channel such that the second audio channel is decorrelated from the first audio channel.

5. Apparatus according to claim 1 or 2, wherein the track information provider comprises a filter function provider for providing audio filter functions to one or more track information elements in response to the limited spatial range, and wherein the audio signal comprises a first audio channel and a second audio channel for the spatially extended sound source, or wherein the audio signal comprises a first audio channel and a second audio channel derived from the first audio channel by means of a second channel processor, and wherein the audio processor comprises a filter applicator for applying the audio filter functions to the first audio channel and the second audio channel.

6. Apparatus according to claim 5, wherein the audio filter functions comprise, for each of the first and second audio channels, a head-related transfer function, a head-related impulse response, a room binaural impulse response, or a room impulse response, or wherein the second channel processor comprises a decorrelation filter or a neural network processor for deriving, from the first audio channel, the second audio channel such that the second audio channel is decorrelated from the first audio channel.

7. Apparatus according to claim 5 or claim 6, wherein the track information provider is configured to provide, as a track information element, an interchannel correlation value, wherein the audio signal comprises a first audio channel and a second audio channel for the spatially extended sound source, or wherein the audio signal comprises a first audio channel and a second audio channel is derived from the first audio channel by means of a second channel processor, and wherein the audio processor is configured to impose a correlation between the first audio channel and the second audio channel using the interchannel correlation value, and wherein the filter applicator is configured to apply audio filter functions to a result of the correlation determination performed by the audio processor in response to the interchannel correlation value.

8. Apparatus according to any of the preceding claims, wherein the track information provider comprises at least one memory for storing information about different track information elements in relation to different limited spatial intervals, and an output interface for retrieving, using the memory, the one or more track information elements associated with the limited spatial interval.

9. Apparatus according to claim 8, wherein the memory comprises at least one of a lookup table, a vector codebook, a multidimensional function fit, a Gaussian mixture model (GMM), and a support vector machine (SVM), and wherein the output interface is configured to retrieve one or more clue information items by looking up the lookup table, using the vector codebook, applying the multidimensional function fit, or using the GMM or the SVM.

10. Apparatus according to any of the preceding claims, wherein the track information provider is configured to store information about one or more track information elements associated with a set of spaced candidate spatial intervals, the set of spaced limited spatial intervals covering the maximum spatial interval, wherein the track information provider is configured to match (30) the candidate limited spatial interval defining a candidate spatial interval that is closest to a specific limited spatial interval defined by the limited spatial interval and to provide one or more track information elements associated with the matching candidate limited spatial interval, or wherein the limited spatial interval comprises at least one of a pair of azimuth angles, a pair of elevation angles, information about a horizontal distance,information about a vertical distance, information about a total distance and a pair of azimuth angles and a pair of elevation angles, or wherein the spatial interval indication comprises a code (S3, S5) that identifies the limited spatial interval as a specific sector of the maximum spatial interval, wherein the maximum spatial interval comprises a plurality of different sectors.

11. Apparatus according to claim 10, wherein a sector of the plurality of different sectors has a first extension in an azimuthal or horizontal direction and a second extension in an elevation or vertical direction, wherein the second extension in an elevation or vertical direction of a sector is greater than the first extension, or wherein the second extension covers a maximum vertical direction or elevation range.

12. Apparatus according to claim 10 or 11, wherein the plurality of different sectors is defined such that a distance between centers of adjacent sectors in the azimuthal or horizontal direction is greater than 5 degrees or even greater than or equal to 10 degrees.

13. Apparatus according to any of the preceding claims, wherein the audio processor is configured to generate, from the audio signal, a first processed channel and a second processed channel for binaural rendering or loudspeaker rendering or active crosstalk reduction loudspeaker rendering.

14. Apparatus according to any of the preceding claims, wherein the track information provider is configured to provide one or more interchannel track values ​​as the one or more track information elements, wherein the audio processor is configured to generate (320, 330, 340, 350), from the audio signal, a first processed channel and a second processed channel such that the first processed channel and the second processed channel have one or more interchannel tracks as controlled by the one or more interchannel track values.

15. Apparatus according to claim 14, wherein the track information provider is configured to provide one or more inter-channel correlation track values ​​as the one or more track information elements, wherein the audio processor is configured to generate, from the audio signal, a first processed channel and a second processed channel such that the first processed channel and the second processed channel have an inter-channel correlation value as controlled by the one or more inter-channel correlation track values.

16. Apparatus according to any of the preceding claims, wherein the track information provider is configured to provide one or more track information elements for a plurality of frequency bands in response to the limited spatial interval that is identical for the plurality of frequency bands, wherein the track information elements for different bands are different from each other.

17. Apparatus according to any of the preceding claims, wherein the track information provider is configured to provide one or more track information elements for a plurality of different frequency bands, and wherein the audio processor is configured to process the audio signal in a spectral domain, wherein a track information element for a band is applied to a plurality of spectral values ​​of the audio signal in the band.

18. Apparatus according to any one of the preceding claims, wherein the audio processor is configured to receive a first audio channel and a second audio channel as the audio signal representing the spatially extended sound source, or wherein the audio processor is configured to receive a first audio channel as the audio signal representing the spatially extended sound source and to derive the second audio channel by means of a second channel processor, wherein the first audio channel and the second audio channel are decorrelated with each other by a certain degree of decorrelation, wherein the track information provider is configured to provide an interchannel correlation value as the one or more track information elements,and where the audio processor is configured to decrease the degree of correlation between the first channel and the second channel to the value indicated by one or more interchannel correlation signals provided by the track information provider.

19. Apparatus according to any of the preceding claims, further comprising an audio signal interface for receiving the audio signal representing the spatially extended sound source, wherein the audio signal comprises only a first audio channel or comprises only a first audio channel and a second audio channel, or the audio signal comprises no more than two audio channels.

20. Apparatus according to any one of the preceding claims, wherein the spatial information interface is configured to receive a listener position as the spatial interval indication, to calculate a projection of a two-dimensional or three-dimensional envelope associated with the spatially extended sound source onto a projection plane using, as the spatial interval indication, the listener position and information about the spatially extended sound source such as a geometry or a position of the spatially extended sound source, or to calculate a two-dimensional or three-dimensional envelope of a projection of a geometry of the spatially extended sound source onto a projection plane using, as the spatial interval indication, the listener position and information about the spatially extended sound source such as a geometry or a position of the spatially extended sound source.and to determine the limited spatial range from envelope projection data.

21. Apparatus according to claim 20, wherein the spatial information interface is configured to calculate the spatially extended sound source envelope using information about the spatially extended sound source, the geometry of the spatially extended sound source, and to project the envelope in a direction towards the listener using the listener's position to obtain the projection of the two-dimensional or three-dimensional envelope onto the projection plane, or to project the geometry of the spatially extended sound source as defined by information about the geometry of the spatially extended sound source in a direction towards the listener's position and to calculate the envelope of a projected geometry to obtain the projection of the two-dimensional or three-dimensional envelope onto the projection plane.

22. Apparatus according to claim 20 or claim 21, wherein the spatial information interface is configured to determine the limited spatial interval such that an edge of a sector defined by the limited spatial interval is located to the right of the projection plane with respect to the listener and / or to the left of the projection plane with respect to the listener and / or at the top of the projection plane with respect to the listener and / or at the bottom of the projection plane with respect to the listener or coincides, for example, within a tolerance of + / -10% with one of a right edge, a left edge, an upper edge and a lower edge of the projection plane with respect to the listener.

23. A method for synthesizing a spatially extended sound source, the method comprising: receiving a spatial range indication indicating a limited spatial range for the spatially extended sound source within a maximum spatial range; providing one or more track information elements in response to the limited spatial range; and processing an audio signal representing the spatially extended sound source using the one or more track information elements.

24. Computer-readable means for synthesizing a spatially extended sound source, comprising the method of claim 23.

Images