Apparatus, method, or computer program for processing audio scenes encoded with bandwidth extension

Abstract

An apparatus for processing an audio scene (130) representing a sound field, the audio scene including information about a transport signal (122) and a parameter set (112; 114). The apparatus comprises: an output interface (120) for generating a processed audio scene (124) using the information about the parameter set (112; 114) and the transport signal (122), the output interface (120) being configured to generate a raw representation of two or more channels using the parameter set (112; 114) and the transport signal (112; 144), a multi-channel enhancer (990) for generating an augmented representation of the two or more channels using the transport signal (122), and a signal combiner (940) for combining the raw representation of the two or more channels and the augmented representation of the two or more channels to obtain the processed audio scene (124).

Description

[Technical field]

[0001] The present invention relates to audio processing, and in particular to the processing of encoded audio scenes for the purpose of generating processed audio scenes for rendering, storage or transmission. [Background technology]

[0002] Traditionally, audio applications providing a means for user communication such as telephone or videoconferencing have been largely limited to mono recording and playback. However, in recent years, the emergence of new immersive VR / AR technologies has also increased interest in spatial rendering of communication scenarios. To meet this interest, a new 3GPP (registered trademark, hereinafter the same) audio standard called Immersive Voice and Audio Services (IVAS) is currently under development. Based on the recently released Enhanced Voice Services (EVS) standard, IVAS provides multi-channel and VR enhancements that can render immersive audio scenes, for example for spatial videoconferencing, while still meeting the low-latency requirements of smooth audio communication. This continuing need to keep the overall delay of the codec to a minimum without sacrificing playback quality provides the motivation for the work described below.

[0003] Coding scene-based audio (SBA) material - such as third-order Ambisonics content - at low bitrates (e.g., below 32 kbps) by systems using parametric audio coding - such as directional audio coding (DirAC) [1] [2] - allows direct coding of only a single (transport) channel, while restoring spatial information via side parameters at the decoder in the filterbank domain. If the speaker configuration at the decoder allows only stereo reproduction, a full reconstruction of the 3D audio scene is not required. Since higher bitrate coding of two or more transport channels is possible, in those cases the stereophonic reproduction of the scene can be directly extracted and reproduced (e.g., due to an additional filterbank analysis / synthesis, such as a complex-valued low-delay filterbank (CLDFB)) without any parametric spatial upmix (skipping the spatial renderer entirely) and the extra delay that comes with it. However, in the case of low rates with only one transport channel, this is not possible. Thus, for DirAC, until now, a FOA (first-order Ambisonics) upmix with the following L / R conversion was required for the stereo output: This is problematic because in this case the overall delay is larger than other possible stereo output configurations in the system and it is desirable to have all stereo output configurations aligned.

[0004] High-latency DirAC stereo rendering example FIG. 12 shows an example block diagram of a conventional decoder process for high-delay DirAC stereo upmix.

[0005] For example, in an encoder not shown, a single downmix channel is derived via spatial downmix in a DirAC encoder process and then encoded by a core coder such as Enhanced Voice Services (EVS) [3].

[0006] In a decoder, for example using the conventional DirAC upmix process depicted in FIG. 12, one available transport channel is first decoded from the bitstream 1212 by using a mono or IVAS mono decoder 1210, resulting in a time-domain signal that can be viewed as a decoded mono downmix 1214 of the original audio scene.

[0007] The decoded mono signal 1214 is input to a CLDFB 1220 in order to analyze the delay-inducing signal 1214 (transform the signal into the frequency domain). The significantly delayed output signal 1222 is input to a DirAC renderer 1230, which processes the delayed output signal 1222 and uses the transmitted side information, i.e., DirAC side parameters 1213, to convert the signal 1222 into a FOA representation, i.e., a FOA upmix 1232 of the original scene with spatial information restored from the DirAC side parameters 1213.

[0008] The transmitted parameters 1213 may include directivity angles, e.g., one azimuth value for the horizontal plane and one elevation angle for the vertical plane, as well as one diffuseness value per frequency band to perceptually describe the entire 3D audio scene. Due to the per-band processing of the DirAC stereo upmix, the parameters 1213 are transmitted multiple times per frame, i.e., one set per frequency band. Furthermore, each set comprises multiple directivity parameters for individual subframes within the entire frame (e.g., of 20 ms length) to increase the time resolution.

[0009] The result of the DirAC renderer 1230 can be, for example, a complete 3D scene in FOA format, i.e., FOA upmix 1232, which can be converted using a matrix transform 1240 into an L / R signal 1242 suitable for playback on a stereo speaker setup. In other words, the L / R signal 1242 can be input to stereo speakers or can be input to a CLDFB synthesis 1250 using predefined channel weights. The CLDFB synthesis 1250 converts the two input frequency domain output channels (L / R signal 1242) into the time domain, resulting in a stereo playable output signal 1252.

[0010] Alternatively, it is possible to use the same DirAC stereo upmix to directly generate a rendering of the stereo output configuration, which avoids the intermediate step of generating the FOA signal. This reduces the algorithmic complexity of the framework, a potential complication. Nevertheless, both approaches require the use of an additional filter bank after the core encoding, resulting in an additional delay of 5 ms. Further examples of DirAC rendering can be found in [2].

[0011] The DirAC stereo upmix approach is rather suboptimal in terms of both delay and complexity. By using the CLDFB filter bank, the output is significantly delayed (an additional 5 ms in the DirAC example) and therefore has the same overall delay as a full SBA upmix (compared to the delay of a stereo output configuration where no additional step of rendering is required). It is also a reasonable assumption that doing a full SBA upmix to generate a stereo signal is not ideal in terms of system complexity. Summary of the Invention [Problem to be solved by the invention]

[0012] It is an object of the present invention to provide an improved concept for processing encoded audio scenes. [Means for solving the problem]

[0013] This object is achieved by an apparatus for processing encoded audio scenes according to claim 1, a method for processing encoded audio scenes according to claim 32 or a computer program according to claim 33.

[0014] According to a first aspect relating to parameter transformation, the invention is based on the discovery that an improved concept for processing an encoded audio scene is obtained by transforming given parameters in the encoded audio scene related to a virtual listener position into transformed parameters related to a channel representation of a given output format. This procedure offers high flexibility in processing and finally rendering the processed audio scene in a channel-based environment.

[0015] An embodiment according to a first aspect of the invention comprises an apparatus for processing an encoded audio scene representing a sound field associated with a virtual listener position, the encoded audio scene comprising information about a transport signal, e.g. a core encoded audio signal, and a first set of parameters associated with the virtual listener position. The apparatus comprises a parameter converter for converting the first set of parameters, e.g. Directional Audio Coding (DirAC) side parameters in B-format or First Order Ambisonics (FOA) format, into a second set of parameters, e.g. stereo parameters associated with a channel representation comprising two or more channels for reproduction at a given spatial position of the two or more channels, and an output interface for generating a processed audio scene using the second set of parameters and the information about the transport signal.

[0016] In an embodiment, a short-time Fourier transform (STFT) filter bank is used for the upmix rather than a directional audio coding (DirAC) renderer. It is therefore possible to upmix one downmix channel (contained in the bitstream) to a stereo output without additional overall delay. The use of a window with a very short overlap for the analysis in the decoder allows the upmix to stay within the overall delay required for communication codecs or for the upcoming Immersive Voice and Audio Services (IVAS). This value can be, for example, 32 ms. In such an embodiment, any post-processing for the purpose of bandwidth extension can be avoided since such processing can be done in parallel with the parameter conversion or mapping.

[0017] By mapping listener-specific parameters of the low-band (LB) signal to a channel-specific stereo parameter set for the low-band, low-latency upmixing of the low-band in the DFT domain can be achieved. For the high-band, a single stereo parameter set allows the upmix to be performed in the high-band in the time domain, preferably in parallel with the spectral analysis, spectral upmixing and spectral synthesis of the low-band.

[0018] Exemplarily, the parameter converter is configured to use a single-sided gain parameter for panning and a residual prediction parameter that is closely related to stereo width and also to a diffuseness parameter used in directional audio coding (DirAC).

[0019] This "DFT-stereo" approach, in an embodiment, allows the IVAS codec to stay within the same overall latency as EVS, specifically 32 ms, when processing the encoded audio scene (scene-based audio) to obtain a stereo output. The complexity of the parametric stereo upmix is ​​reduced by implementing simple processing via DFT-stereo instead of spatial DirAC rendering.

[0020] The invention is based on the discovery that a second aspect relating to bandwidth extension results in an improved concept for processing encoded audio scenes.

[0021] An embodiment according to a second aspect of the present invention comprises an apparatus for processing an audio scene representing a sound field, the audio scene comprising information relating to a transport signal and a parameter set, the apparatus further comprising an output interface for generating a processed audio scene using the parameter set and the information relating to the transport signal, the output interface being configured to generate a raw representation of two or more channels using the parameter set and the transport signal, a multi-channel enhancer for generating an augmented representation of the two or more channels using the transport signal, and a signal combiner for combining the raw representation of the two or more channels and the augmented representation of the two or more channels to obtain a processed audio scene.

[0022] Generating a raw representation of two or more channels on the one hand and an augmented representation of two or more channels separately on the other hand allows for great flexibility in choosing the algorithms for the raw and augmented representations. The final combination is already performed for each of the one or more output channels, i.e. in the multi-channel output domain rather than in the lower channel input or encoded scene domain. Thus, following the combination, the two or more channels can be composited and used for further procedures such as rendering, transmission, or storage.

[0023] In an embodiment, some of the core processing, such as the bandwidth extension (BWE) of the Algebraic Code Excited Linear Prediction (ACELP) audio coder for the extended representation, can be performed in parallel with the DFT-stereo processing for the raw representation. Thus, the delays caused by both algorithms are not accumulated, and only a given delay caused by one algorithm is the final delay. In an embodiment, only a transport signal, e.g., a low-band (LB) signal (channel), is input to an output interface, e.g., a DFT-stereo processing, while the high-band (HB) is separately upmixed in the time domain, e.g., using a multi-channel enhancer, so that the stereo decoding can be processed within a target time window of 32 ms. For example, a linear time-domain upmix of the entire high-band is obtained without significant delay, e.g., by using wide-band panning, based on the mapped side gains from the parameter converter.

[0024] In an embodiment, the delay reduction in DFT-stereo may not come entirely from the difference in the overlap of the two transforms, e.g., the 5 ms transform delay caused by CLDFB and the 3,125 ms transform delay caused by STFT. Instead, DFT-stereo exploits the fact that the last 3,25 ms from the EVS coder target delay of 32 ms essentially comes from the ACELP BWE. Everything else (the remaining milliseconds to reach the EVS coder target delay) is simply artificially delayed to finally achieve alignment of the two transformed signals (HB stereo upmix signal and HB filling signal with LB stereo core signal) again. Thus, to avoid additional delay in DFT-stereo, only all other components of the encoder are transformed, e.g., within a very short DFT window overlap, while the ACELP BWE, e.g., using a multi-channel enhancer, is mixed almost without delay in the time domain.

[0025] According to a third aspect relating to parameter smoothing, the invention is based on the discovery that an improved concept for processing encoded audio scenes is obtained by performing parameter smoothing with respect to time according to a smoothing rule. Thus, a processed audio scene obtained by applying smoothed parameters instead of raw parameters to a transport channel has an improved audio quality. This is particularly true when the smoothed parameters are upmix parameters, but for any other parameters such as envelope parameters or LPC parameters or noise parameters or scale factor parameters, the use or smoothed parameters obtained by the smoothing rule results in an improved subjective audio quality of the obtained processed audio scene.

[0026] An embodiment according to a third aspect of the present invention comprises an apparatus for processing an audio scene representing a sound field, the audio scene including a transport signal and information on a first parameter set, the apparatus further comprising a parameter processor for processing the first parameter set to obtain a second parameter set, the parameter processor being configured to calculate at least one raw parameter for each output time frame using at least one parameter of the first parameter set for an input time frame, to calculate smoothing information, such as a coefficient, for each raw parameter according to a smoothing rule, and to apply the corresponding smoothing information to the corresponding raw parameter to derive a parameter of the second parameter set for the output time frame, and an output interface for generating a processed audio scene using the second parameter set and the information on the transport signal.

[0027] By smoothing the raw parameters over time, strong variations of gain or parameters from one frame to the next are avoided. The smoothing coefficient determines the strength of the smoothing, which is adaptively calculated in a preferred embodiment by a parameter processor, which in an embodiment also has the function of a parameter converter for converting listener position related parameters into channel related parameters. The adaptive calculation makes it possible to obtain a quicker response whenever the audio scene changes suddenly. The adaptive smoothing coefficient is calculated for each band from the change in energy in the current band. The energy for each band is calculated in all subframes contained in the frame. Furthermore, the change in energy over time, characterized by two averages, the short-term average and the long-term average, does not affect the smoothing in extreme cases, while a not very large sudden increase in energy does not reduce the smoothing. Thus, a smoothing coefficient is calculated for each of the DTF-stereo subframes in the current frame from the quotient of the averages.

[0028] It should be mentioned herein that all the above and below mentioned alternatives or aspects can be used individually, i.e. without any aspect, however in other embodiments two or more aspects are combined with each other, in other embodiments all aspects are combined with each other to obtain an improved compromise between overall delay, achievable audio quality and required implementation effort.

[0029] Preferred embodiments of the present invention are described below with reference to the accompanying drawings. [Brief description of the drawings]

[0030] [Figure 1] 1 is a block diagram of an apparatus for processing an encoded audio scene using a parameter transformer according to an embodiment; [Figure 2a] 1 shows a schematic diagram of a first set of parameters and a second set of parameters according to an embodiment; [Figure 2b]1 is an embodiment of a parameter transformer or parameter processor for calculating raw parameters. [Figure 2c] 1 is an embodiment of a parameter transformer or parameter processor for combining raw parameters. [Diagram 3] 1 is an embodiment of a parameter transformer or parameter processor for performing a weighted combination of raw parameters. [Figure 4] 1 is an embodiment of a parameter transformer for generating side gain parameters and residual prediction parameters. [Figure 5a] 1 is an embodiment of a parameter transformer or processor for calculating smoothing coefficients for raw parameters. [Figure 5b] 1 is an embodiment of a parameter transformer or processor for calculating smoothing coefficients for frequency bands. [Figure 6] 1 shows a schematic diagram of averaging of a transport signal with a smoothing factor according to an embodiment; [Figure 7] 1 is an embodiment of a parameter transformer parameter processor for computing recursive smoothing; [Figure 8] 1 is an embodiment of an apparatus for decoding a transport signal. [Figure 9] 1 is an embodiment of an apparatus for processing an audio scene encoded using bandwidth extension. [Figure 10] 1 is an embodiment of an apparatus for obtaining a processed audio scene. [Figure 11] FIG. 2 is a block diagram of an embodiment of a multi-channel enhancer. [Figure 12] FIG. 1 is a block diagram of a conventional DirAC stereo upmix process. [Figure 13] 1 is an embodiment of an apparatus for obtaining a processed audio scene using parameter mapping; [Figure 14] 1 is an embodiment of an apparatus for obtaining an audio scene processed using bandwidth extension. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Fig. 1 shows an apparatus for processing an encoded audio scene 130, e.g. representing a sound field associated with a virtual listener position. The encoded audio scene 130 comprises information on a transport signal 122, e.g. a bitstream, and a first parameter set 112, e.g. a number of DirAC parameters also included in the bitstream, which are associated with the virtual listener position. The first parameter set 112 is input to a parameter converter 110 or parameter processor, which converts the first parameter set 112 into a second parameter set 114 associated with a channel representation including at least two or more channels. The apparatus can support different audio formats. The audio signals may be acoustic in nature, picked up by a microphone, or electrical in nature, which are assumed to be transmitted to a loudspeaker. Supported audio formats can be mono signals, low-band signals, high-band signals, multi-channel signals, first-order and higher-order Ambisonics components, and audio objects. The audio scene can also be described by combining different input formats.

[0032] The parameter converter 110 is configured to calculate the second parameter set 114 as parametric stereo or multi-channel parameters, e.g., two or more channels, which are input to the output interface 120. The output interface 120 is configured to generate the processed audio scene 124 by combining the transport signal 122 or information about the transport signal with the second parameter set 114 to obtain the transcoded audio scene as the processed audio scene 124. Another embodiment comprises upmixing the transport signal 122 using the second parameter set 114 to an upmix signal comprising two or more channels. In other words, the parameter converter 120 maps the first parameter set 112, e.g. used for DirAC rendering, to the second parameter set 114. The second parameter set may comprise side gain parameters used for panning and residual prediction parameters that, when applied in the upmix, result in an improved spatial image of the audio scene. For example, the parameters of the first parameter set 112 may include at least one of a direction of arrival parameter, a diffuseness parameter, a directional information parameter related to a sphere with the virtual listening position as the origin of the sphere, and a distance parameter. For example, the parameters of the second parameter set 114 may include at least one of a side gain parameter, a residual prediction gain parameter, an inter-channel level difference parameter, an inter-channel time difference parameter, an inter-channel phase difference parameter, and an inter-channel coherence parameter.

[0033] FIG. 2a shows a schematic diagram of the first parameter set 112 and the second parameter set 114 according to an embodiment. In particular, the parameter resolution of both parameters (first and second) is depicted. Each horizontal axis of FIG. 2a represents time, and each vertical axis of FIG. 2a represents frequency. As shown in FIG. 2a, the input time frame 210 to which the first parameter set 112 is related includes two or more input time subframes 212 and 213. Directly below, the output time frame 220 to which the second parameter set 114 is related is shown in a corresponding diagram related to the diagram above. This shows that the output time frame 220 is small compared to the input time frame 210, and the output time frame 220 is long compared to the input time subframes 212 or 213. It should be noted that the input time subframes 212 or 213 and the output time frame 220 can include multiple frequencies as frequency bands. The input frequency band 230 can include the same frequencies as the output frequency band 240. According to an embodiment, the frequency bands of the input frequency band 230 and the output frequency band 240 may not be connected or correlated with each other.

[0034] It should be noted that the side gains and residual gains described in Fig. 4 are typically calculated per frame, such that a single side gain and residual gain are calculated for each input frame 210. However, in other embodiments, not only a single side gain and a single residual gain are calculated for each frame, but a group of side gains and a group of residual gains are calculated for the input time frame 210, where each side gain and each residual gain is associated with, for example, a particular input time subframe 212 or 213 of a frequency band. Thus, in an embodiment, the parameter converter 110 calculates a group of side gains and a group of residual gains for each frame of the first parameter set 112 and the second parameter set 114, and the number of side and residual gains for the input time frame 210 is typically equal to the number of input frequency bands 230.

[0035] 2b shows an embodiment of the parameter converter 110 for calculating 250 raw parameters 252 of the second parameter set 114. The parameter converter 110 calculates the raw parameters 252 for each of two or more input time subframes 212 and 213 in a chronologically sequential manner. For example, the calculation 250 derives, for each input frequency band 230 and time point (input time subframes 212, 213), a primary direction of arrival (DOA) in azimuth angle θ and a primary direction of arrival in elevation angle φ and diffusivity parameter ψ.

[0036] For directional components such as X, Y, and Z, the first order spherical harmonic function at the center position is given by the omnidirectional components w(b,n) and the DirAC parameters using the following equation: TIFF0007600386000001.tif1366TIFF0007600386000002.tif13118TIFF0007600386000003.tif13118TIFF0007600386000004.tif1393The W channel represents the omnidirectional mono component of the signal, corresponding to the output of an omnidirectional microphone. The X, Y, and Z channels are the three-dimensional directional components. From these four FOA channels, a stereo signal (stereo version, stereo output) can be obtained by decoding, using the parameter converter 110, including the W and Y channels, which results in two cardioids pointing at azimuth angles of +90 degrees and -90 degrees. Therefore, the following equation shows the left-right relationship of the stereo signal, where the left channel L is represented by adding the Y channel to the W channel, and the right channel R is represented by subtracting the Y channel from the W channel. TIFF0007600386000005.tif1325 In other words, this decoding corresponds to a first-order beamforming pointing in two directions, which can be expressed using the following equation: TIFF0007600386000006.tif7109 As a result, there is a direct link between the stereo output (left and right channel) and the first parameter set 112, i.e. the DirAC parameters.

[0037] However, on the other hand, the second parameter set 114, i.e., the DFT parameters, depends on a model of the left L and right R channels based on the intermediate signal M and the side signal S, which can be expressed using the following equation: Here, M is transmitted as a mono signal (channel) corresponding to the omnidirectional channel W in the scene-based audio (SBA) mode. Furthermore, in the DFT, the stereo S is predicted from M using the side gain parameters described below.

[0038] 4 shows an embodiment of the parameter converter 110 for generating side gain parameters 455 and residual prediction parameters 456, for example using a calculation process 450. The parameter converter 110 preferably processes the calculations 250 and 450 for calculating the raw parameters 252, for example the side parameters 455 for the output frequency band 241, using the following equations: According to the formula, b is the output frequency band, sidegain is the side gain parameter 455, azimuth is the azimuth component of the direction of arrival parameter, and elevation is the elevation component of the direction of arrival parameter. As shown in FIG. 4, the first parameter set 112 includes the direction of arrival (DOA) parameters 456 for the input frequency bands 231 as previously described, and the second parameter set 114 includes the side gain parameters 455 for each input frequency band 230. However, if the first parameter set 112 further includes a spread parameter ψ 453 for the input frequency bands 231, then the parameter converter 110 is configured to calculate 250 the side gain parameters 455 for the output frequency bands 241 using the following formula: TIFF0007600386000009.tif13157 According to the formula, diff(b) is the diffusivity parameter ψ 453 for the input frequency band b 230. It should be noted that the directivity parameters 456 of the first parameter set 112 may include different value ranges, for example the azimuth parameter 451 is [0;360], the elevation parameter 452 is [0;180], and the resulting side gain parameter 455 is [-1;1]. As shown in Fig. 2c, the parameter converter 110 uses a combiner 260 to combine at least two raw parameters 252, resulting in the parameters of the second parameter set 114 associated with the output time frame 220.

[0039] According to an embodiment, the second parameter set 114 further includes residual prediction parameters 456 for the output frequency band 241 of the output frequency band 240 shown in Fig. 4. The parameter converter 110 can use the spread parameter ψ 453 from the input frequency band 231 as indicated by the residual selector 410 as the residual prediction parameter 456 for the output frequency band 241. If the input frequency band 231 and the output frequency band 241 are equal to each other, the parameter converter 110 uses the spread parameter ψ 453 from the input frequency band 231. From the spread parameter ψ 453 for the input frequency band 231, the spread parameter ψ 453 for the output frequency band 241 is derived, and the spread parameter ψ 453 is used for the output frequency band 241 as the residual prediction parameter 456 for the output frequency band 241. The parameter converter 110 can then use the spread parameter ψ 453 from the input frequency band 231.

[0040] In DFT stereo processing, the residual of the prediction using the residual selector 410 is assumed and expected to be incoherent, modeled by its energy, decorrelating the residual signal towards the left L and right R. The residual of the prediction of the side signal S with the intermediate signal M as a mono signal (channel) can be expressed as: TIFF0007600386000010.tif766 That energy is modeled in the DFT stereo processing using the residual prediction gain using the following formula: The residual gain represents the inter-channel incoherent components and spatial width of the stereo signal, and is therefore directly linked to the diffuse part modeled by DirAC. The residual energy can therefore be rewritten as a function of the DirAC diffuseness parameter: TIFF0007600386000012.tif748

[0041] 3 illustrates a parameter converter 110 for performing a weighted combination 310 of raw parameters 252 according to an embodiment. At least two raw parameters 252 are input to the weighted combination 310, and a weighting factor 324 of the weighted combination 310 is derived based on an amplitude-related measure 320 of the transport signal 122 in the corresponding input time subframe 212. Furthermore, the parameter converter 110 is configured to use an energy or power value of the transport signal 112 in the corresponding input time subframe 212 or 213 as the amplitude-related measure 320. The amplitude-related measure 320 measures, for example, the energy or power of the transport signal 122 in the corresponding input time subframe 212, so that the weighting factor 324 for that input subframe 212 is larger if the energy or power of the transport signal 122 in the corresponding input time subframe 212 is higher compared to the weighting factor 324 for an input subframe 212 in which the energy or power of the transport signal 122 in the corresponding input time subframe 212 is lower.

[0042] As mentioned before, the directivity, azimuth and elevation parameters have corresponding ranges of values. However, the directional parameters of the first parameter set 112 usually have a higher time resolution than the second parameter set 114, which means that more than one azimuth and elevation value must be used to calculate one side gain value. According to an embodiment, the calculation is based on energy-dependent weights that can be obtained as output of the amplitude-related measure 320. For example, all For the TIFF0007600386000013.tif74 input temporal subframes 212 and 213, the energy of the subframe nrg is calculated using the following formula: TIFF0007600386000014.tif2582, where TIFF0007600386000015.tif73 is the time domain input signal, TIFF0007600386000016.tif74 is the number of samples in each subframe, and TIFF0007600386000017.tif73 is the sample index. Furthermore, each output time frame For TIFF0007600386000018.tif72230, the weights 324 are then Each input time subframe in TIFF0007600386000019.tif72 The contribution of TIFF0007600386000020.tif73212,213 can be calculated as follows: TIFF0007600386000021.tif1352The side gain parameter 455 is then finally calculated using the following formula: Due to the similarity between the TIFF0007600386000022.tif13116 parameters, the diffuseness parameters 453 per band are directly mapped to the residual prediction parameters 456 of all subframes in the same band. The similarity can be expressed by the following equation: TIFF0007600386000023.tif786

[0043] 5a shows an embodiment of the parameter converter 110 or parameter processor for calculating a smoothing coefficient 512 for each raw parameter 252 according to a smoothing rule 514. Furthermore, the parameter converter 110 is configured to apply the smoothing coefficient 512 (corresponding smoothing coefficient for one raw parameter) to the raw parameters 252 (one raw parameter corresponding to the smoothing coefficient) to derive parameters of the second parameter set 114 for the output time frame 220, i.e. the parameters of the output time frame.

[0044] 5b shows an embodiment of the parameter converter 110 or parameter processor for calculating smoothing coefficients 522 for frequency bands using a compression function 540. The compression function 540 may be different for different frequency bands, so that the compression strength of the compression function 540 is stronger for lower frequency bands than for higher frequency bands. The parameter converter 110 is further configured to calculate the smoothing coefficients 512, 522 using a maximum boundary selection 550. In other words, the parameter converter 110 can obtain the smoothing coefficients 512, 522 by using different maximum boundaries for different frequency bands, such that the maximum boundary of the lower frequency band is higher than the maximum boundary of the higher frequency band.

[0045] Both the compression function 540 and the maximum boundary selection 550 are input to a calculation 520 that obtains a smoothing coefficient 522 for the frequency band 522. For example, the parameter converter 110 is not limited to using two calculations 510 and 520 to calculate the smoothing coefficients 512 and 522, so that the parameter converter 110 is configured to calculate the smoothing coefficients 512, 522 using only one calculation block that can output the smoothing coefficients 512 and 522. In other words, the smoothing coefficients are calculated for each band (for each raw parameter 252) from the changes in the energy in the current frequency band. For example, by using a parameter smoothing process, the side gain parameters 455 and the residual prediction parameters 456 are smoothed over time to avoid large fluctuations in gain. This requires a relatively strong smoothing in most cases, but a faster response whenever the audio scene 130 changes suddenly, so the smoothing coefficients 512, 522 that determine the strength of the smoothing are calculated adaptively.

[0046] Therefore, the energy per band nrg is calculated for all subframes using the following formula: Calculated for TIFF0007600386000024.tif73: TIFF0007600386000025.tif1981, where TIFF0007600386000026.tif73 is the frequency bins (real and imaginary) of the DFT transformed signal, TIFF0007600386000027.tif72 is the current frequency band This is the bin index across all bins in TIFF0007600386000028.tif73.

[0047] To capture the change in energy across the two averages, one short-term average 331 and one long-term average 332 are calculated using the amplitude-related measure 320 of the transport signal 122, as shown in FIG.

[0048] FIG. 6 shows a schematic diagram of an amplitude-related measure 320 averaging the transport signal 122 over the smoothing factor 512 according to an embodiment. The x-axis represents time and the y-axis represents energy (of the transport signal 122). The transport signal 122 shows a schematic portion of a sine function 122. As shown in FIG. 6, the second time portion 631 is shorter than the first time portion 632. The change in energy over the averages 331 and 332 is calculated for each band according to the following equation: Calculated for TIFF0007600386000029.tif73: TIFF0007600386000030.tif1361 and TIFF0007600386000031.tif1958, where TIFF0007600386000032.tif712 and TIFF0007600386000033.tif711 is the number of time subframes before which the individual averages are calculated For example, in this particular embodiment, TIFF0007600386000035.tif712 is set to a value of 3, TIFF0007600386000036.tif711 will be set to the value 10.

[0049] Furthermore, the parameter converter or parameter processor 110 is configured to use a calculation 510 to calculate a smoothing factor 512, 522 based on the ratio between the long-term average 332 and the short-term average 331. In other words, the quotient of the two averages 331 and 332 is calculated, so that a higher short-term average, indicating a recent increase in energy, leads to a decrease in smoothing. The following equation shows the correlation between the smoothing factor 512 and the two averages 331 and 312: Due to the fact that a higher long-term average 332, which indicates a decrease in energy, does not lead to a decrease in smoothing, the smoothing factor 512 is (for now) set to a maximum of 1. As a result, the above equation becomes: TIFF0007600386000038.tif725 minimum TIFF0007600386000039.tif1310 (0.3 in this embodiment). However, in extreme cases the coefficient needs to be close to 0, which can be achieved by using the following formula to find the value in the range Range from TIFF0007600386000040.tif1317 This is why the image is converted to [TIFF0007600386000041.tif79]. TIFF0007600386000042.tif19116

[0050] In an embodiment, the smoothing is excessively reduced compared to the smoothing shown previously, so that the coefficients are compressed by the root function towards the value 1. Since stability is particularly important in the lowest bands, the fourth order root is TIFF0007600386000043.tif711 and Used in TIFF0007600386000044.tif711. The formula for the minimum bandwidth is: TIFF0007600386000045.tif1359 All other bands The formula for TIFF0007600386000046.tif711 performs compression with a square root function using the following formula: TIFF0007600386000047.tif1359 All other bands By applying a square root function to TIFF0007600386000048.tif711, extreme cases where the energy may increase exponentially are made smaller, and less rapid increases in energy do not reduce the smoothing as much.

[0051] Furthermore, the maximum smoothing is set according to the frequency band for the following equation: Note that the factor 1 simply repeats the previous value without the contribution of the current gain. TIFF0007600386000049.tif788, where TIFF0007600386000050.tif721 represents a given implementation with five bands set according to the following table:

[0052] [Table 1] The smoothing coefficients are the sum of the DFT stereo subframes in the current frame. This is calculated for each of TIFF0007600386000052.tif73.

[0053] Figure 7 shows the side gain parameter TIFF0007600386000053.tif722455 and the residual prediction gain parameter TIFF0007600386000054.tif723456 show a parameter converter 110 according to an embodiment using recursive smoothing 710 in which both are recursively smoothed: TIFF0007600386000055.tif7161 and A recursive smoothing 710 over temporally subsequent output time frames of the current output time frame is calculated by combining the parameters of the preceding output time frame 532 weighted by the first weighting value and the raw parameters 252 for the current output time frame 220 weighted by the second weighting value. In other words, the smoothed parameters for the current output time frame are calculated such that the first weighting value and the second weighting value are derived from the smoothing coefficients for the current time frame.

[0054] These mapped and smoothed parameters (g side ,g pred ) is input to the output interface 120 for DFT stereo processing, i.e., a stereo signal ( TIFF0007600386000057.tif710 is the downmix TIFF0007600386000058.tif710, Residual prediction signal TIFF0007600386000059.tif712, and the mapped parameters TIFF0007600386000060.tif710 and Generated from TIFF0007600386000061.tif711. For example, downmix TIFF0007600386000062.tif710 is derived from the downmix by either enhanced stereo filling using an all-pass filter, or stereo filling using a delay.

[0055] The upmix is ​​described by the following equation: TIFF0007600386000063.tif7148 and TIFF0007600386000064.tif7156 Upmix is ​​done in the frequency bands as described in the table above. All bins in TIFF0007600386000065.tif73 Subframe in TIFF0007600386000066.tif72 Each side gain is processed. TIFF0007600386000068.tif710 is the downmix as above Energy and residual prediction gain parameters in TIFF0007600386000069.tif710 TIFF0007600386000070.tif712 or Energy normalization factor calculated from TIFF0007600386000071.tif722 Weighted by TIFF0007600386000072.tif712.

[0056] The mapped and smoothed side gains 755 and the mapped and smoothed residual gains 756 are input to the output interface 120 to obtain a smoothed audio scene. Based on the above description, processing the encoded audio scene using smoothed parameters thus results in an improved compromise between achievable audio quality and implementation effort.

[0057] Fig. 8 shows an apparatus for decoding a transport signal 122 according to an embodiment. The (encoded) audio signal 816 is input to a transport signal core decoder 810 for core decoding the (core encoded) audio signal 816 to obtain a (decoded raw) transport signal 812, which is input to the output interface 120. For example, the transport signal 122 can be the encoded transport signal 812 output from the transport signal core encoder 810. The transport signal (decoded) 812 is input to the output interface 120, which is configured to generate a raw representation 818 of two or more channels, for example a left channel and a right channel, using a parameter set 814 including the second parameter set 114. For example, the transport signal core decoder 810 for decoding the core encoded audio signal to obtain the transport signal 122 is an ACELP decoder. Furthermore, the core decoder 810 is configured to feed the decoded raw transport signal 812 to two parallel branches, a first of which comprises the output interface 120 and a second of which comprises the transport signal enhancer 820 or the multi-channel enhancer 990, or both. The signal combiner 940 is configured to receive a first input to be combined from the first branch and a second input to be combined from the second branch.

[0058] As shown in Fig. 9, the device for processing an encoded audio scene 130 can use a bandwidth extension processor 910. A low-band transport signal 901 is input to an output interface 120 to obtain a two-channel low-band representation of a transport signal 972. It should be noted that the output interface 120 processes the transport signal 901 in the frequency domain 955, for example during an upmix process 960, and transforms the two-channel transport signal 901 in the time domain 966. This is done by a transformer 970, which transforms the upmixed spectral representation 962 presenting the frequency domain 955 into the time domain to obtain a two-channel low-band representation of the transport signal 972.

[0059] As shown in Fig. 8, a single channel low-band transport signal 901 is input to a converter 950, which performs, for example, a conversion of a time portion of the transport signal 901 corresponding to the output time frame 220 into a spectral representation 952 of the transport signal 901, i.e. from the time domain 966 to the frequency domain 955. For example, as described in Fig. 2, the portion (of the output time frame) is shorter than the input time frame 210 in which the parameters 252 of the first parameter set 112 are organized.

[0060] The spectral representation 952 is input to an upmixer 960, for example to upmix the spectral representation 952 using the second parameter set 114 to obtain an upmixed spectral representation 962, which is (still) processed in the frequency domain 955. As mentioned above, the upmixed spectral representation 962 is input to a transformer 970 to transform the upmixed spectral representation 962, i.e. each channel of the two or more channels, from the frequency domain 955 to the time domain 966 (time representation) to obtain a low-band representation 972. Thus, the two or more channels in the upmixed spectral representation 962 are calculated. Preferably, the output interface 120 is configured to operate in the complex discrete Fourier transform domain, and the upmix operation is performed in the complex discrete Fourier transform domain. The transformation from the complex discrete Fourier transform domain to a real-valued time domain representation is performed using the transformer 970. In other words, the output interface 120 is configured to generate a raw representation of two or more channels using an upmixer 960 in a second domain, namely the frequency domain 955, while the first domain represents the time domain 966.

[0061] In an embodiment, the upmix operation of the upmixer 960 is based on the following equation: TIFF0007600386000073.tif77= TIFF0007600386000074.tif1345 and TIFF0007600386000075.tif1310= TIFF0007600386000076.tif1359, Where: TIFF0007600386000077.tif78 is a transport signal 901 for frame t and frequency bin k, TIFF0007600386000078.tif77 is the side gain parameter 455 for frame t and subband b, TIFF0007600386000079.tif76 is the residual prediction gain parameter 456 for frame t and subband b, g norm is an optional energy adjustment factor, TIFF0007600386000080.tif77 is the raw residual signal for frame t and frequency bin k.

[0062] The transport signals 902, 122 are processed in the time domain 966, in contrast to the low-band transport signal 901. The transport signal 902 is input to a bandwidth extension processor (BWE processor) 910 to generate a high-band signal 912 and to a multi-channel filter 930 to apply a multi-channel filling operation. The high-band signal 912 is input to an upmixer 920 to upmix the high-band signal 912 to an upmixed high-band signal 922 using the second parameter set 144, i.e., parameters of the output time frame 262, 532. For example, the upmixer 920 may apply a wideband panning process to the high-band signal 912 in the time domain 966, using at least one parameter from the second parameter set 114.

[0063] The low-band representation 972, the upmixed high-band signal 922, and the multi-channel filling transport signal 932 are input to a signal combiner 940, which combines in the time domain 966 the result of the wideband panning 922, the result of the stereo filling 932, and the low-band representation of the two or more channels 972. This combination results in a full-band multi-channel signal 942 in the time domain 966 as a channel representation. As outlined above, the converter 970 converts each channel of the two or more channels in the spectral representation 962 into a time representation to obtain a raw time representation of the two or more channels 972. Thus, the signal combiner 940 combines the raw time representation of the two or more channels with an extended time representation of the two or more channels.

[0064] In an embodiment, only the low-band (LB) transport signal 901 is input to the output interface 120 (DFT stereo) processing, and the high-band (HB) transport signal 912 is upmixed separately in the time domain (using an upmixer 920). Such a process is implemented for a panning operation using the BWE processor 910 and time-domain stereo filling using a multi-channel filler 930 for generating ambience contributions. The panning process includes wideband panning based on mapped side gains, e.g., the frame-wise mapped and smoothed side gains 755. Here, there is only one gain per frame covering the complete high-band frequency range, which simplifies the calculation of the left and right high-band channels from the downmix channel based on the following equation: Each subframe Sample in TIFF0007600386000081.tif73 For each TIFF0007600386000082.tif72, TIFF0007600386000083.tif7105 and TIFF0007600386000084.tif7121.

[0065] High-bandwidth stereo filling signal The TIFF0007600386000085.tif716, i.e., the multi-channel filling transport signal 932, is represented as follows: Delay TIFF0007600386000086.tif715, TIFF0007600386000087.tif714 Weight it by the energy normalization factor TIFF0007600386000088.tif712 is obtained by further using: All samples in the current time frame For TIFF0007600386000089.tif72 (done for the whole time frame 210, not the time subframes 213 and 213), TIFF0007600386000090.tif799 and TIFF0007600386000091.tif13101. TIFF0007600386000092.tif73 is the number of samples by which the highband downmix is ​​delayed to generate the filling signal 932 obtained by the multichannel filler 930. Other methods for generating the filling signal apart from the delay can be implemented, such as a more advanced decorrelation process or the use of a noise signal or any other signal derived from the transport signal in a different way compared to the delay.

[0066] Both the panned stereo signals 972 and 922 and the generated stereo filling signal 932 are combined (mixed back) into the core signal after DFT synthesis using a signal combiner 940 .

[0067] This described process of the ACELP highband is also in contrast to the high-delay DirAC processing, where the ACELP core and TCX frames are artificially delayed to be aligned with the ACELP highband, where the CLDFB (analysis) is performed on the complete signal, which means that the upmix of the ACELP highband is also done in the CLDFB domain (frequency domain).

[0068] Fig. 10 shows an embodiment of an apparatus for obtaining a processed audio scene 124. The transport signal 122 is input to the output interface 120 for generating a raw representation of the two or more channels 972 using the second parameter set 114 and a multi-channel enhancer 990 for generating an extended representation 992 of the two or more channels. For example, the multi-channel enhancer 990 is configured to perform at least one operation of a group of operations including a bandwidth expansion operation, a gap filling operation, a quality enhancement operation, or an interpolation operation. To obtain the processed audio scene 124, both the raw representation of the two or more channels 972 and the extended representation 992 of the two or more channels are input to a signal combiner 940.

[0069] 11 shows a block diagram of an embodiment of a multi-channel enhancer 990 for generating an enhanced representation 992 of two or more channels, including a transport signal enhancer 820, an upmixer 830, and a multi-channel filler 930. The transport signal 122 and / or the decoded raw transport signal 812 are input to the transport signal enhancer 820, which generates an enhanced transport signal 822, which is input to the upmixer 830 and the multi-channel filler 930. For example, the transport signal enhancer 820 is configured to perform at least one operation from a group of operations including a bandwidth expansion operation, a gap filling operation, a quality enhancement operation, or an interpolation operation.

[0070] 9, the multi-channel filler 930 generates a multi-channel filling transport signal 932 using the transport signal 902 and at least one parameter 532. In other words, the multi-channel enhancer 990 is configured to generate an extended representation of two or more channels 992 using the extended transport signal 822 and the second parameter set 114, or using the extended transport signal 822 and the upmixed extended transport signal 832. For example, the multi-channel enhancer 990 includes either the upmixer 830 or the multi-channel filler 930, or both the upmixer 830 and the multi-channel filler 930, to generate an extended representation 992 of two or more channels using the transport signal 122 or the extended transport signal 933 and at least one parameter of the second parameter set 532. In an embodiment, the transport signal enhancer 820 or the multi-channel enhancer 990 is configured to operate in parallel with the output interface 120 when generating the raw representation 972, or the parameter converter 110 is configured to operate in parallel with the transport signal enhancer 820.

[0071] In Fig. 13, the bitstream 1312 transmitted from the encoder to the decoder may be the same as the DirAC-based upmixing scheme shown in Fig. 12. The single transport channel 1312 derived from the DirAC-based spatial downmixing process is input to the core decoder 1310, decoded by the core decoder, e.g. an EVS or IVAS mono decoder, and transmitted together with the corresponding DirAC side parameters 1313.

[0072] In this DFT stereo approach to process audio scenes without extra delay, the initial decoding in the mono core decoder of the transport channel (IVAS mono decoder) also remains unchanged. Instead of passing through the CLDFB filter bank 1220 from Fig. 12, the decoded downmix signal 1314 is input to a DFT analysis 1320 to transform the decoded mono signal 1314 into the STFT domain (frequency domain), such as by using windows with very short overlap. Thus, the DFT analysis 1320 does not introduce any additional delay to the target system delay of 32 ms, using only the remaining headroom between the overall delay and the one already introduced by the MDCT analysis / synthesis of the core decoder.

[0073] The DirAC side parameters 1313 or the first parameter set 112 are input to a parameter mapping 1360, which may for example include a parameter transformer 110 or a parameter processor to obtain DFT stereo side parameters, i.e. the second parameter set 114. The frequency domain signal 1322 and the DFT side parameters 1362 are input to a DFT stereo decoder 1330, which generates a stereo upmix signal 1332, for example by using an upmixer 960 as described in Fig. 9. The two channels of the stereo upmix 1332 are input to a DFT synthesis, which transforms the stereo upmix 1332 from the frequency domain to the time domain, for example using a transformer 970 as described in Fig. 9, resulting in an output signal 1342 that may represent the processed audio scene 124.

[0074] FIG. 14 shows an embodiment for processing an audio scene encoded using bandwidth extension 1470. The bitstream 1412 is input to an ACELP core or low-band decoder 1410, instead of an IVAS mono decoder as described in FIG. 13, to generate a decoded low-band signal 1414. The decoded low-band signal 1414 is input to a DFT analysis 1420 to convert the signal 1414 to a frequency domain signal 1422, e.g., the spectral representation 952 of the transport signal 901 from FIG. 9. The DFT stereo decoder 1430 can represent an upmixer 960 that generates a LB stereo upmix 1432 using the decoded low-band signal 1442 in the frequency domain and the DFT stereo side parameters 1462 from the parameter mapping 1460. The generated LB stereo upmix 1432 is input to a DFT synthesis block 1440 that performs a transformation to the time domain, e.g., using the transformer 970 of FIG. 9. The low-band representation 972 of the transport signal 122, i.e. the output signal 1442 of the DFT synthesis stage 1440, is input to a signal combiner 940 which combines the upmixed high-band stereo signal 922 and the multi-channel filling high-band transport signal 932 with the low-band representation of the transport signal 972 resulting in a full-band multi-channel signal 942.

[0075] The decoded LB signal 1414 and parameters 1415 for the BWE 1470 are input to the ACELP BWE decoder 910 to generate the decoded highband signal 912. The mapped side gain 1462, e.g. the mapped and smoothed side gain 755 of the lowband spectral region, is input to the DFT stereo block 1430, and the mapped and smoothed single side gain of the entire highband is forwarded to the highband upmix block 920 and the stereo filling block 930. The HB upmix block 920 for upmixing the decoded HB signal 912 using the highband side gain 1472, e.g. the parameters 532 of the output time frame 262 from the second parameter set 114, generates the upmixed highband signal 922. The stereo filling block 930 for filling the decoded highband transport signal 912, 902 uses the parameters 532, 456 of the output time frame 262 from the second parameter set 114 and generates the highband filling transport signal 932.

[0076] In conclusion, embodiments according to the present invention create a concept for processing encoded audio scenes using parameter transformation and / or using bandwidth extension and / or using parameter smoothing, resulting in an improved compromise between overall delay, achievable audio quality and implementation effort.

[0077] Subsequently, further embodiments of aspects of the invention, in particular combinations of aspects of the invention, are presented. The proposed solution to achieve a low-delay upmix is ​​by using a parametric stereo approach, for example the approach described in [4] using a Short-Time Fourier Transform (STFT) filter bank rather than the DirAC renderer. In this "DFT-stereo" approach, an upmix of one downmix channel to a stereo output is described. The advantage of this method is that windows with a very short overlap are used for the DFT analysis in the decoder, which allows staying within the much lower overall delay required for communication codecs like EVS [3] or the upcoming IVAS codec (32 ms). Also, unlike DirAC CLDFB, the DFT stereo processing is not a post-processing step to the core coder, but is performed in parallel with part of the core processing, namely the Bandwidth Extension (BWE) of the Algebraic Code Exit Excitation Prediction (ACELP) audio coder, without exceeding this already given delay. Thus, with respect to the 32 ms delay of EVS, the DFT stereo processing can be called delay-free, since it operates with the same overall coder delay. DirAC, on the other hand, can be seen as a post-processor where CLDFB induces an additional 5 ms of delay to extend the overall delay to 37 ms.

[0078] In general, latency gains are achieved: the low latency comes from processing steps that are performed in parallel with the core processing, whereas in the exemplary CLDFB version, post-processing steps to perform the necessary rendering are performed after the core encoding.

[0079] Unlike DirAC, DFT-Stereo utilizes an artificial delay of 3.25 ms for all components except ACELP BWE, by only transforming those components into the DFT domain using windows with a very short overlap of 3.125 ms that fits the available headroom without inducing more delay. Thus, only TCX and ACELP without BWE are upmixed in the frequency domain, while ACELP BWE is upmixed in the time domain by a separate delay-free processing step called Inter-Channel Bandwidth Extension (ICBWE) [5]. For the special stereo output of a given embodiment, this time-domain BWE processing is slightly modified, which will be explained towards the end of the embodiment.

[0080] The transmitted DirAC parameters cannot be used directly for the DFT stereo upmix. It is therefore necessary to map the given DirAC parameters to the corresponding DFT stereo parameters. While DirAC uses azimuth and elevation angles for spatial positioning along with a diffuseness parameter, DFT stereo has a single side gain parameter used for panning and a residual prediction parameter that is closely related to the stereo width and thus the diffuseness parameter of DirAC. From the perspective of parameter resolution, each frame is divided into two subframes and several frequency bands per subframe. The side gains and residual gains used in DFT stereo are described in [6].

[0081] The DirAC parameters are derived from a band-by-band analysis of the audio scene, originally in B format or FOA. Then, for each band k and time instant n, the azimuth angle TIFF0007600386000093.tif714 and elevation angle TIFF0007600386000094.tif715 and diffusion coefficient Derive the main direction of arrival of TIFF0007600386000095.tif714. For the directional component, the first order spherical harmonic function at the center position is It can be derived using TIFF0007600386000096.tif714 and DirAC parameters. TIFF0007600386000097.tif1366TIFF0007600386000098.tif13118TIFF0007600386000099.tif13118TIFF0007600386000100.tif1393

[0082] Furthermore, from the FOA channel it is possible to obtain a stereo version by decoding with W and Y, which results in two cardioids pointing at azimuth angles of +90 degrees and -90 degrees. TIFF0007600386000101.tif1323This decoding corresponds to first-order beamforming pointing in two directions. As a result, there is a direct link between the stereo output and the DirAC parameters, while the DFT parameters depend on a model of the L and R channels that is based on the mid signal M and the side signal S. TIFF0007600386000103.tif1322M is transmitted as a mono channel and corresponds to the omnidirectional channel W in case of SBA mode. In the DFT, the stereo S is predicted from M using the side gain, which can be expressed using the DirAC parameters as follows: TIFF0007600386000104.tif13152

[0083] In DFT stereo, the prediction residual is assumed and expected to be incoherent, modeled by its energy, decorrelating the residual signal towards left and right. The residual of the prediction of S by M can be expressed as: TIFF0007600386000105.tif1362And the energy is modeled in DFT stereo using the prediction gain as follows: The residual gain represents the inter-channel incoherent components and spatial width of the stereo signal, and is therefore directly linked to the diffuse part modeled by DirAC. The residual energy can therefore be rewritten as a function of the DirAC diffuseness parameter: TIFF0007600386000107.tif748

[0084] The band configurations of commonly used DFT stereo are not the same as those of DirAC, so they need to be adapted to cover the same frequency range as the DirAC bands. For these bands, the directivity angle of DirAC is: can be mapped to the DFT stereo side gain parameters by TIFF0007600386000108.tif13149, where TIFF0007600386000109.tif73 is the current band, and the parameter range is TIFF0007600386000110.tif715, About elevation angle TIFF0007600386000111.tif715 and the resulting side gain values TIFF0007600386000112.tif714. However, the directivity parameters in DirAC usually have a higher time resolution than DFT stereo, which means that more than one azimuth and elevation value must be used for the calculation of one side gain value. One way would be to do averaging across subframes, but in this implementation the calculation is based on energy-dependent weights. All For TIFF0007600386000113.tif74DirAC subframes, the energy of the subframe is Calculated as TIFF0007600386000114.tif2582, where: TIFF0007600386000115.tif73 is the time domain input signal, TIFF0007600386000116.tif74 is the number of samples in each subframe, and TIFF0007600386000117.tif73 is the sample index for each DFT stereo subframe Regarding TIFF0007600386000118.tif72, Internal as TIFF0007600386000119.tif1352 Each DirAC subframe of TIFF0007600386000120.tif72 A weight can be calculated for the contribution of TIFF0007600386000121.tif73.

[0085] The side gain is then The final result is calculated as TIFF0007600386000122.tif19166.

[0086] Due to the similarity between the parameters, one spread value per band is directly mapped to the residual prediction parameters of all subframes in the same band. TIFF0007600386000123.tif763Furthermore, the parameters are smoothed over time to avoid strong fluctuations in the gain. This means that in most cases we need a relatively strong smoothing, but whenever the scene changes suddenly we need a faster response, so the smoothing coefficient that determines the strength of the smoothing is calculated adaptively. This adaptive smoothing coefficient is calculated for each band from the change in the energy in the current band. Thus, first all subframes The bandwidth energy needs to be calculated for TIFF0007600386000124.tif73: TIFF0007600386000125.tif1981 where, TIFF0007600386000126.tif73 is the frequency bins (real and imaginary) of the DFT transformed signal, TIFF0007600386000127.tif72 is the current bandwidth These are the bin indices of all bins in TIFF0007600386000128.tif73.

[0087] To capture the change in energy over time 2, one short-term and one long-term are then Regarding TIFF0007600386000129.tif73, TIFF0007600386000130.tif1361 and Calculated according to TIFF0007600386000131.tif1958.

[0088] Where: TIFF0007600386000132.tif712 and TIFF0007600386000133.tif711 is the number of subframes before which the individual averages are calculated TIFF0007600386000134.tif73. In this particular implementation, TIFF0007600386000135.tif712 is set to 3, TIFF0007600386000136.tif711 is set to 10. A smoothing factor is then calculated from the quotient of the averages, so that a higher short-term average, indicating a recent increase in energy, leads to less smoothing. TIFF0007600386000137.tif1371The smoothing factor is set here to a maximum of 1, since a higher long-term average, indicating a decrease in energy, does not lead to a decrease in smoothing.

[0089] The above formula is TIFF0007600386000138.tif725 minimum TIFF0007600386000139.tif1310 (0.3 in this implementation). However, in extreme cases the coefficients need to be close to 0, which means Values ​​range through TIFF0007600386000140.tif19116 Range from TIFF0007600386000141.tif1317 This is why the image is converted to [TIFF0007600386000142.tif79].

[0090] In the non-extreme cases, the smoothing is reduced too much, so the coefficients are compressed by the root function towards a value of 1. Since stability is especially important in the lowest bands, the fourth order roots are TIFF0007600386000143.tif711 and Used in TIFF0007600386000144.tif711: TIFF0007600386000145.tif1359 Meanwhile, all other bands TIFF0007600386000146.tif711 is the square root Compressed by TIFF0007600386000147.tif1359.

[0091] In this way, extreme cases remain close to 0, but sudden increases in energy do not reduce the smoothing as much.

[0092] Finally, the maximum smoothing is set according to the band (a factor of 1 simply repeats the previous value without any contribution of the current gain): TIFF0007600386000148.tif788 where, in the given implementation, it has 5 bands TIFF0007600386000149.tif721 is set according to the table below.

[0093] [Table 2] The smoothing coefficients are calculated for each DFT stereo subframe in the current frame. Calculated for TIFF0007600386000151.tif73.

[0094] In the final step, both the side gains and the residual prediction gains are recursively smoothed according to: TIFF0007600386000152.tif13153 and TIFF0007600386000153.tif13156 These mapped and smoothed parameters are now fed into the DFT stereo processing, where the stereo signal TIFF0007600386000154.tif78 is downmixed Generated from TIFF0007600386000155.tif710, residual prediction signal TIFF0007600386000156.tif712 (obtained from the downmix either by "enhanced stereo filling" using an all-pass filter [7] or by regular stereo filling using a delay) as well as the mapped parameters TIFF0007600386000157.tif710 and TIFF0007600386000158.tif711 is generated. The upmix is ​​generally described by the following equation [6]: TIFF0007600386000159.tif7148 and Bandwidth All bins in TIFF0007600386000160.tif73 Each subframe of TIFF0007600386000161.tif72 Regarding TIFF0007600386000162.tif73, TIFF0007600386000163.tif13149Furthermore, each side gain TIFF0007600386000164.tif710 is TIFF0007600386000165.tif710 and Energy normalization factor calculated from the energy of TIFF0007600386000166.tif712 Weighted by TIFF0007600386000167.tif712.

[0095] Finally, the upmix signal is transformed back to the time domain via IDFT and played back in a given stereo setting.

[0096] The "Time Domain Bandwidth Extension" (TBE) [8] used in ACELP generates its own delay (in the implementation, this embodiment is based on exactly 2.3125 ms) and therefore cannot be transformed into the DFT domain while the overall delay stays within 32 ms (3.25 ms remains for the stereo decoder where the STFT already uses 3.125 ms). Therefore, only the low band (LB) is put into the DFT stereo processing shown by 1450 in FIG. 14, while the high band (HB) must be upmixed separately in the time domain as shown in block 920 in FIG. 14. In normal DFT stereo, this is done via Inter-Channel Bandwidth Extension (ICBWE) [5] for panning for ambience and time domain stereo filling. In a given case, the stereo filling in block 930 is calculated in the same way as in normal DFT stereo. However, the ICBWE processing is skipped entirely due to missing parameters and replaced by a low resource requiring wideband panning in block 920 based on the mapped side gain 1472. In the given embodiment, there is only a single gain that covers the complete HB region, which simplifies the calculation of the left and right HB channels in block 920 from the downmix channel below. TIFF0007600386000168.tif7105 and Each subframe Sample in TIFF0007600386000169.tif73 About TIFF0007600386000170.tif72 TIFF0007600386000171.tif13115HB stereo filling signal TIFF0007600386000172.tif716 is processed in block 930 with a delay TIFF0007600386000173.tif714 and Weighting by TIFF0007600386000174.tif714, energy normalization factor as below Obtained by TIFF0007600386000175.tif712. TIFF0007600386000176.tif1393 and All samples in the current frame (not subframes, but full frames) About TIFF0007600386000177.tif72 TIFF0007600386000178.tif13101, where: TIFF0007600386000179.tif73 is the number of samples the HB downmix is ​​delayed relative to the filling signal.

[0097] Both the panned stereo signal and the generated stereo filling signal are finally mixed back to the core signal after DFT synthesis in combiner 940.

[0098] This special processing of ACELP HB is also in contrast to the high-delay DirAC processing, where the ACELP Core and TCX frames are artificially delayed to be aligned with the ACELP HB, and so the CLDFB is performed on the complete signal, i.e. the upmix of ACELP HB is also done in the CLDFB domain.

[0099] Advantages of the proposed method The lack of additional delay allows the IVAS codec to stay within the same overall delay as in EVS (32 ms) for this particular case of SBA input to stereo output.

[0100] Due to the overall simplicity and easier processing, the complexity of parametric stereo upmix via DFT is much lower than spatial DirAC rendering.

[0101] Further Preferred Embodiments 1. An apparatus, method or computer program for encoding or decoding as hereinbefore described.

[0102] 2. An apparatus or method for encoding or decoding, or a related computer program, comprising: a system in which the input is encoded by a model based on a spatial audio representation of an acoustic scene having a first set of parameters and decoded at the output using a stereo model for two output channels or a multi-channel model for more than two output channels having a second set of parameters; and / or Mapping spatial parameters to stereo parameters, and / or Transformation of one frequency domain based input representation / parameters into another frequency domain based output representation / parameters, and / or Transformation of parameters with higher time resolution into lower time resolution, and / or Lower output delay due to shorter window overlap of the second frequency translation, and / or Mapping DirAC parameters (directional angle, divergence) to DFT stereo parameters (side gain, residual prediction gain) to output SBA DirAC encoded content as stereo, and / or Transformation from CLDFB-based input representations / parameters to DFT-based output representations / parameters, and / or Conversion of 5ms resolution parameters to 10ms resolution parameters, and / or Advantage: An apparatus or method, or associated computer program, for encoding or decoding, including lower output delay due to shorter DFT window overlap compared to CLDFB.

[0103] It should be noted that all alternatives or aspects described above and all aspects defined by the independent claims in the following claims can be used individually, i.e., without any alternatives or purposes other than the contemplated alternatives, purposes or independent claims. However, in other embodiments, two or more alternatives or aspects or independent claims can be combined with each other, and in other embodiments, all aspects or alternatives and all independent claims can be combined with each other.

[0104] It should be outlined that different aspects of the present invention relate to a parameter transformation aspect, a smoothing aspect, and a bandwidth extension aspect, which can be implemented separately or independently of each other, or at least any two of the three aspects can be combined, or all three aspects can be combined in the above-mentioned embodiments.

[0105] The encoded signals of the present invention can be stored in a digital or non-transitory storage medium or can be transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0106] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step, and similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or function of a corresponding apparatus.

[0107] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software, for example using a digital storage medium such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored and which cooperate (or can cooperate) with a programmable computer system to perform the respective method.

[0108] Some embodiments of the present invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0109] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which operates to perform one of the methods when the computer program product is run on a computer. The program code may for example be stored on a machine readable carrier.

[0110] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

[0111] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0112] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium, or computer readable medium) having recorded thereon the computer program for performing one of the methods described herein.

[0113] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals may be adapted to be transferred via a data communication connection, such as for example the Internet.

[0114] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0115] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0116] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0117] The above-described embodiments merely exemplify the principles of the present invention. It is understood that modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the specific details presented as illustrative and explanatory of the embodiments herein.

[0118] References [1] V. Pulkki, M.-VVJ Laitinen, J. Ahonen, T. Lokki and T. Pihlajamaeki, "Directional audio coding-perception - based reproduction of spatial sound," in INTERNATIONAL WORKSHOP ON THE PRINCIPLES AND APPLICATION ON SPATIAL HEARING, 2009.

[0119] [2] G. Fuchs, O. Thiergart, S. Korse, S. Doehla, M. Multrus, F. Kuech, Boutheon, A. Eichenseer and S. Bayer, "Apparatus, method and computer program for encoding, decoding, scene processing and other procedures related to dirac based spatial audio coding using low-order, mid-order and high-order components generators". WO Patent 2020115311A1, 11 06 2020.

[0120] [3] 3GPP TS 26.445, Codec for Enhanced Voice Services (EVS); Detailed algorithmic description.

[0121] [4] S. Bayer, M. Dietz, S. Doehla, E. Fotopoulou, G. Fuchs, W. Jaegers, G. Markovic, M. Multrus, E. Ravelli and M. Schnell, " APPARATUS AND METHOD FOR ESTIMATING AN INTER-CHANNEL TIME DIFFERENCE". Patent WO17125563, 27 07 2017.

[0122] [5] V. S. C. S. Chebiyyam and V. Atti, "Inter-channel bandwidth extension". WO Patent 2018187082A1, 11 10 2018.

[0123] [6] J. Buethe, G. Fuchs, W. Jaegers, F. Reutelhuber, J. Herre, E. Fotopoulou, M. Multrus and S. Korse, "Apparatus and method for encoding or decoding a multichannel signal using a side gain and a residual gain". WO Patent WO2018086947A1, 17 05 2018.

[0124] [7] J. Buethe, F. Reutelhuber, S. Disch, G. Fuchs, M. Multrus and R. Geiger, "Apparatus for Encoding or Decoding an Encoded Multichannel Signal Using a Filling Signal Generated by a Broad Band Filter". WO Patent WO2019020757A2, 31 01 2019.

[0125] [8] V. A. e. al., "Super-wideband bandwidth extension for speech in the 3GPP EVS codec," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, 2015.

Claims

1. An apparatus for processing an audio scene (130) representing a sound field, said audio scene (130) comprising information about a transport signal (122) and a parameter set (112; 114), an output interface (120) for generating a processed audio scene (124) using said parameter set (112; 114) and information about said transport signal (122), said output interface (120) being configured to generate a raw representation of two or more channels using said parameter set (112; 114) and said transport signal (112; 144); a multi-channel enhancer (990) for generating an extended representation of the two or more channels using the transport signal (122), comprising: a transport signal enhancer (820) for generating an extended transport signal (822); and an upmixer (920) for upmixing the extended transport signal (822) to obtain the extended representation of the two or more channels, the transport signal enhancer (820) and the upmixer (920) being configured to operate in parallel with the output interface (120) when generating the raw representation of the two or more channels; a signal combiner (940) for combining the raw representation of the two or more channels and the augmented representation of the two or more channels to obtain the processed audio scene (124).

2. The transport signal (122) is a coded transport signal, and the apparatus a transport signal core decoder (810) for generating a decoded raw transport signal; the transport signal enhancer (820) is configured to generate the enhanced transport signal using the decoded raw transport signal; 2. The apparatus of claim 1, wherein the output interface (120) is configured to generate the raw representation of the two or more channels using the parameter set (112; 114) and the decoded raw transport signal.

3. 3. The apparatus of claim 1, wherein the multi-channel enhancer (990) comprises the upmixer (920) and a multi-channel filler (930) for generating the extended representation of the two or more channels using the extended transport signal (822) and at least one parameter of the parameter set (114).

4. the output interface (120) is configured to generate the raw representation of the two or more channels using an upmix in a second domain; the transport signal enhancer (820) is configured to generate the extended transport signal (822) in a first domain different from the second domain, and the multi-channel enhancer (990) is configured to generate the extended representation of the two or more channels using the extended transport signal (822) in the first domain; 4. The apparatus of claim 1, wherein the signal combiner (940) is configured to combine the raw representation of the two or more channels and the augmented representation of the two or more channels in the first region.

5. The apparatus of claim 4 , wherein the first domain is the time domain and the second domain is the spectral domain.

6. 6. The apparatus of claim 1, wherein the transport signal enhancer is configured to perform at least one operation from a group of operations including a bandwidth expansion operation, a gap filling operation, a quality enhancement operation, or an interpolation operation.

7. a parameter converter (110) for converting a received parameter set (112) into said parameter set (114), said parameter set (114) relating to a channel representation comprising said two or more channels for playback at a predetermined spatial position for said two or more channels, said channel representation corresponding to said processed audio scene (124), said parameter converter configured to operate in parallel with said transport signal enhancer (820), 7. Apparatus according to any one of claims 1 to 6.

8. 8. The apparatus of claim 2, wherein the transport signal core decoder (810) is configured to supply the decoded raw transport signal to two parallel branches, a first of the two parallel branches comprising the output interface (120), a second of the two parallel branches comprising the transport signal enhancer (820) and the upmixer (920), and the signal combiner (940) is configured to receive a first input to be combined from the first branch and a second input to be combined from the second branch.

9. The output interface (120) performing a conversion into a spectral representation of a time portion of said transport signal (122) corresponding to an output time frame (220); performing an upmix operation on the spectral representation using the parameter set (114) to obtain the two or more channels in the spectral representation; converting each of the two or more channels in the spectral representation into a time representation to obtain a raw time representation of the two or more channels; 9. The apparatus of claim 1, wherein the signal combiner (940) is configured to combine the raw time representation of the two or more channels and the expanded time representation of the two or more channels.

10. the apparatus being configured to receive a parameter set; the apparatus further comprising a parameter converter (110) for converting the received parameter set (112) into the parameter set (114), the parameter set (114) being associated with a channel representation comprising the two or more channels for playback at a predetermined spatial position for the two or more channels, the channel representation corresponding to the processed audio scene (124); 10. The apparatus of claim 1, wherein the output interface (120) is configured to generate the processed audio scene (124) using the parameter set (114) and the information regarding the transport signal (122).

11. the output interface (120) is configured to upmix the transport signal (122) using the parameter set (114) into an upmix signal comprising the two or more channels.

11. Apparatus according to any one of claims 1 to 10.

12. 12. The apparatus of claim 1, wherein the output interface (120) is configured to generate the processed audio scene (124) by combining the transport signal (122) or the information on the transport signal (122) with the parameter set (114) to obtain a transcoded audio scene as the processed audio scene (124).

13. the received parameter set (112) comprises at least one DirAC parameter for each input time frame (210) of a plurality of input time frames and for each input frequency band (231) of a plurality of input frequency bands (230); The apparatus of claim 10 , wherein the parameter transformer (110) is configured to compute the set of parameters (114) as parametric stereo or multi-channel parameters.

14. the at least one parameter includes at least one of a direction of arrival parameter, a diffuseness parameter, a directional information parameter related to a sphere with the virtual listening position as the sphere's origin, and a distance parameter; 14. The apparatus of claim 13, wherein the parametric stereo or multi-channel parameters include at least one of a side gain parameter (455), a residual prediction gain parameter (456), an inter-channel level difference parameter, an inter-channel time difference parameter, an inter-channel phase difference parameter, and an inter-channel coherence parameter.

15. an input time frame (210) to which the received parameter set (112) is associated includes two or more input time subframes, and an output time frame (220) to which the parameter set (114) is associated is smaller than the input time frame (210) and longer than a single input time subframe of the two or more input time subframes included in the input time frame (210); 15. The apparatus of claim 10, 13 or 14, wherein the parameter converter (110) is configured to calculate raw parameters (252) of the parameter set (114) for each of the two or more temporally succeeding input time sub-frames comprised in the input time frame (210) and to combine at least two raw parameters to derive parameters of the parameter set (114) associated with the output time frame (220).

16. 16. The apparatus of claim 15, wherein the parameter converter (110) is configured in the combining of the at least two raw parameters to perform a weighted combination of the at least two raw parameters, weight coefficients of the weighted combination being derived based on an amplitude-related measure (320) of the transport signal (122) in the corresponding input time subframe.

17. 17. The apparatus of claim 16, wherein the parameter converter (110) is configured to use energy or power as the amplitude-related measure (320), and a weighting factor for an input subframe is larger if the energy or power of the transport signal (122) in the corresponding input time subframe is higher compared to a weighting factor for an input subframe in which the energy or power of the transport signal (122) in the corresponding input time subframe is lower.

18. the parameter converter (110) is configured to calculate at least one raw parameter (252) for each output time frame (220) using at least one parameter of the received parameter set (112) for the input time frame (210); said parameter converter (120) is adapted to calculate a smoothing coefficient (512; 522) for each raw parameter (252) according to a smoothing rule; 18. The apparatus of claim 13, wherein the parameter converter (110) is configured to apply corresponding smoothing coefficients (512; 522) to the corresponding raw parameters (252) to derive the parameters of the parameter set (114) for the output time frame (220).

19. The parameter converter (110) calculating a long-term average (332) over the amplitude-related measure (320) of the first time portion of the transport signal (122); calculating a short-term average (331) over the amplitude-related measure (320) of a second time portion of the transport signal (120), the second time portion being shorter than the first time portion; 19. The device according to claim 18, configured to calculate the smoothing factor (512; 522) based on a ratio between the long-term average (332) and the short-term average (331).

20. 20. The apparatus according to claim 18 or 19, wherein the parameter converter (110) is configured to calculate the smoothing coefficients (512; 522) for bands using a compression function (540), the compression function being different for different frequency bands, the compression strength of the compression function being stronger for lower frequency bands than for higher frequency bands.

21. 21. The apparatus according to any one of claims 18 to 20, wherein the parameter converter (110) is configured to calculate the smoothing coefficients (512; 522) using different maximum bounds for different bands, the maximum bound of a low band being higher than the maximum bound of a high band.

22. 22. The apparatus of claim 18, wherein the parameter converter (110) is configured to apply as the smoothing rule a recursive smoothing rule (710) over temporally subsequent output time frames, such that smoothed parameters for a current output time frame (220) are calculated by combining the parameters for a previous output time frame (220) weighted by a first weighting value and the raw parameters (252) for the current output time frame (220) weighted by a second weighting value, the first and second weighting values ​​being derived from the smoothing coefficients (512; 522) for the current output time frame (220).

23. The output interface (120) performing a conversion into a spectral representation of a time portion of said transport signal (122) corresponding to an output time frame (220), said portion being shorter than the input time frame (210) in which said parameters of said received parameter set (112) are organized; performing an upmix operation on the spectral representation using the parameter set (114) to obtain the two or more channels in the spectral representation; 23. An apparatus according to any preceding claim, configured to convert each of the two or more channels in the spectral representation to a temporal representation.

24. The output interface (120) Transform into the complex discrete Fourier transform domain, performing the upmix operation in the complex discrete Fourier transform domain; 24. The apparatus of claim 23, configured to perform the transformation from the complex discrete Fourier transform domain to a real-valued time domain representation.

25. The output interface (120) is configured to perform the upmix operation based on the following equation: ＝ and ＝ 、 Where: is the transport signal (122) for frame t and frequency bin k, is a first channel of the two or more channels in the spectral representation for frame t and frequency bin k; is a second channel of the two or more channels in the spectral representation for frame t and frequency bin k; is the side gain parameter (455) for frame t and subband b, is the residual prediction gain parameter (456) for frame t and subband b, and g ｎｏｒｍ is an optional energy adjustment factor, 25. The apparatus of claim 23 or 24, wherein: {overscore (x)} is the raw residual signal for frame t and frequency bin k.

26. the received parameter set (112) being a direction of arrival parameter for an input frequency band (231), the parameter set (114) including a side gain parameter (455) for each input frequency band (231); The parameter converter (110) is configured to calculate the side gain parameters (455) for an output frequency band (241) using the following formula: where b is the output frequency band (241), sidegain is the side gain parameter (455), azimuth is the azimuth component of the direction of arrival parameter, and elevation is the elevation component of the direction of arrival parameter.

23. Apparatus according to any one of claims 10, 13 to 22.

27. the received parameter set (112) further comprises a spreading parameter for the input frequency band (231), and the parameter converter (110) is configured to calculate the side gain parameter (455) for the output frequency band (241) using the following formula: where diff(b) is the spreading parameter for the input frequency band (231) b; 27. The apparatus of claim 26.

28. the received parameter set (112) comprises a spreading parameter for each input frequency band (231); the parameter set (114) comprises a residual prediction gain parameter (456) for an output frequency band (241); the parameter converter (110) uses the spread parameter from the input frequency band (231) as the residual prediction gain parameter (456) for the output frequency band (241) when the input frequency band and the output frequency band are equal to each other, or derives a spread parameter for the output frequency band (241) from the spread parameter for the input frequency band (231) and then uses the spread parameter for the output frequency band (241) as the residual prediction gain parameter (456) for the output frequency band (241); 23. Apparatus according to any one of claims 10, 13 to 22.

29. The information about the transport signal (122) includes a core encoded audio signal, the apparatus comprising: a transport signal core decoder (810) for core decoding the core encoded audio signal to obtain the transport signal (122), 29. Apparatus according to any one of claims 1 to 28.

30. said transport signal core decoder (810) being within an ACELP decoder; or the output interface (120) is configured to convert (1420) the transport signal (122), which is a low-band signal, into a spectral representation, upmix (1430) the spectral representation, and transform (1440) the upmixed spectral representation in the time domain to obtain a low-band representation (1442) of the two or more channels; the transport signal enhancer (820) comprises a bandwidth extension processor (910) for generating a highband signal (912) from the transport signal (122) in the time domain; and a multi-channel filler (930) configured to apply a stereo filling operation from the transport signal (122) to the highband signal (912) in the time domain; the upmixer (920) is configured to apply wideband panning in the time domain from the transport signal (122) to the highband signal (912) using at least one parameter (1472) from the parameter set (114), the result of the wideband panning (922) and the result of the stereo filling operation (932) representing the extended representation of the two or more channels; 30. The apparatus of claim 2 or 29, wherein the signal combiner (940) is configured to combine, in the time domain, the result of the wideband panning (922), the result of the stereo filling operation (932), and the lowband representation (1442) of the two or more channels as the raw representation of the two or more channels to obtain a full-band multi-channel signal (942) in the time domain as the processed audio scene (124).

31. A method for processing an audio scene (130) representing a sound field associated with a virtual listener position, said audio scene (130) comprising information about a transport signal and a parameter set, generating a processed audio scene (124) using the parameter set and the information regarding the transport signal, said generating comprising generating a raw representation of two or more channels using the parameter set and the transport signal; multi-channel generating an extended representation of the two or more channels using the transport signal, the multi-channel generating comprising: generating an extended transport signal (822) and upmixing the extended transport signal (822) to obtain the extended representation of the two or more channels, the generating the extended transport signal (822) and upmixing the extended transport signal (822) being performed in parallel with generating the raw representation of the two or more channels; combining the raw representation of the two or more channels and the augmented representation of the two or more channels to obtain the processed audio scene (124).

32. 32. A computer program for performing the method of claim 31 when the computer program is run on a computer or processor.

Images