Frequency domain audio encoding for supporting transform length switching

Abstract

To provide a frequency domain audio codec extended in a downward compatible manner.SOLUTION: A frequency domain audio decoder 10 includes: a frequency domain (FD) coefficient extractor 12; a scaling coefficient extractor 14; an inverter 16; and a coupler 18. In input, the FD coefficient extractor and the scaling coefficient extractor access an incoming data stream 20. Outputs of the FD coefficient extractor and the scaling coefficient extractor are connected to inputs of the inverter, and output of the inverter is connected to input of the coupler. The coupler outputs a reconstructed audio signal 22. The FD coefficient extractor extracts FD coefficients 24 of a frame 26 of the audio signal from the data stream. The FD coefficient describes a spectrum of the audio signal in respective frames at various spectral-temporal resolutions and collectively indicates a spectrogram 28 of the audio signal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This application relates to frequency domain audio coding supporting transform length switching. [Background technology]

[0002] Modern frequency-domain speech / audio coding systems, such as the Opus / Celt codecs of the IETF [1], MPEG-4(HE-)AAC [2], or especially MPEG-D xHE-AAC(USAC) [3], provide the means to code an audio frame using either one long transform, i.e., a long block, or eight successive short transforms, i.e., short blocks, depending on the temporal stability of the signal.

[0003] For certain audio signals, such as rain or crowd applause, neither long nor short block coding leads to satisfactory quality at low bit rates. This can be explained by the significant density of transients in such recordings: coding only with long blocks can cause a temporal blur of frequent, audible coding errors, also known as pre-echoes, while coding only with short blocks is generally inefficient due to the increased data overhead that results in spectral holes.

[0004] It would therefore be preferable to have at hand a frequency domain audio coding concept that is also suitable for the kind of audio signals just outlined. Of course, it is feasible to build new frequency domain audio codecs that support switching between a set of transform lengths that encompass, among other things, specific desired transform lengths that are suitable for a particular kind of audio signal. However, it is not an easy task to introduce a new frequency domain audio codec that is adopted by the market. Well-known codecs are already available and are frequently used. It would therefore be preferable to be able to obtain a concept that allows existing frequency domain audio codecs to be extended to additionally support new desired transform lengths, but nevertheless maintain backward compatibility with existing coders and decoders. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] [1] Internet Engineering Task Force (IETF), RFC 6716, “Definition of the Opus Audio Codec,” Proposed Standard, Sep. 2012. Available online at http: / / tools.ietf.org / html / rfc6716. [Non-Patent Document 2] [2] International Organization for Standardization, ISO / IEC 14496-3:2009, “Information Technology - Coding of audio-visual objects - Part 3: Audio,” Geneva, Switzerland, Aug. 2009. [Non-Patent Document 3] [3] M. Neuendorf et al., “MPEG Unified Speech and Audio Coding - The ISO / MPEG Standard for High-Efficiency Audio Coding of All Content Types,” in Proc. 132nd Convention of the AES, Budapest, Hungary, Apr. 2012. Also to appear in the Journal of the AES, 2013. [Non-Patent Document 4] [4] International Organization for Standardization, ISO / IEC 23003-3:2012, “Information Technology - MPEG audio - Part 3: Unified speech and audio coding,” Geneva, Jan. 2012. [Non-Patent Document 5] [5] JDJohnston and AJFerreira, “Sum-Difference Stereo Transform Coding”, in Proc. IEEE ICASSP-92, Vol. 2, March 1992. [Non-Patent Document 6] [6] N.Rettelbach, et al., European Patent EP2304719A1, “Audio Encoder, Audio Decoder, Methods for Encoding and Decoding an Audio Signal, Audio Stream and Computer Program”, April 2011. Summary of the Invention [Problem to be solved by the invention]

[0006] It is therefore an object of the present invention to provide a concept that allows existing frequency domain audio codecs to be extended in a backward compatible manner towards supporting additional transform lengths, such that they switch between transform lengths including this new transform length. [Means for solving the problem]

[0007] This object is achieved by the subject matter of the independent claims attached hereto.

[0008] The invention is based on the observation that it is possible to provide a frequency domain audio codec capable of additionally supporting specific transform lengths in a backward compatible manner when the frequency domain coefficients of each frame are transmitted interleaved, irrespective of the signaling signaling for each frame which transform length is actually applied, and further when the frequency domain coefficient extraction and the scale factor extraction operate independently of that signaling. This approach allows older frequency domain audio coders / decoders that do not support said signaling to nevertheless operate without errors and with a reasonable reproduction quality. At the same time, frequency domain audio coders / decoders that support switching to / from the additionally supported transform lengths achieve even better quality despite being backward compatible. As far as the coding efficiency penalty due to the frequency domain coefficients being coded transparently to older decoders is concerned, this is of a relatively insignificant nature due to the interleaving.

[0009] Advantageous embodiments of the present application are the subject matter of the dependent claims. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic block diagram of a frequency domain audio decoder according to one embodiment; [Diagram 2] FIG. 2 is a schematic diagram illustrating the function of the inverter of FIG. 1; [Diagram 3]FIG. 3 is a schematic diagram illustrating a possible upstream displacement of the inverse TNS filtering process of FIG. 2 according to one embodiment. [Figure 4] FIG. 13 illustrates window selection possibilities when using transform splitting of long stop-start windows in USAC, according to one embodiment. [Diagram 5] FIG. 2 is a block diagram of a frequency domain audio encoder according to one embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In particular, preferred embodiments of the present application are described below with reference to the drawings.

[0012] Figure 1 shows a frequency domain audio decoder supporting transform length switching according to an embodiment of the present application. The frequency domain audio decoder of Figure 1 is generally indicated using the reference number 10 and comprises a frequency domain coefficient extractor 12, a scaling coefficient extractor 14, an inverse transformer 16 and a combiner 18. At their inputs, the frequency domain coefficient extractor 12 and the scaling coefficient extractor 14 have access to an incoming data stream 20. The outputs of the frequency domain coefficient extractor 12 and the scaling coefficient extractor 14 are connected to respective inputs of the inverse transformer 16. The output of the inverse transformer 16 is connected to an input of the combiner 18. The combiner 18 outputs a reconstructed audio signal at an output 22 of the encoder 10.

[0013] The frequency domain coefficient extractor 12 is configured to extract frequency domain coefficients 24 of frames 26 of the audio signal from the data stream 20. The frequency domain coefficients 24 may be MDCT coefficients or may belong to some other transform, such as another lapped transform. As will be further explained below, the frequency domain coefficients 24 belonging to a certain frame 26 describe the spectrum of the audio signal in the respective frame 26 with different spectro-temporal resolutions. The frames 26 represent time portions in which the audio signal is successively partitioned in time. Taken together, all the frequency domain coefficients 24 of all frames represent a spectrogram 28 of the audio signal. The frames 26 may, for example, be of equal length. Due to the type of audio content of the audio signal changing over time, it may be disadvantageous to describe the spectrum of each frame 26 with a continuous spectro-temporal resolution, for example by using a transform with a constant transform length. The transform length, for example, spans the time length of each frame 26, i.e. includes the sample values ​​in this frame 26 of the audio signal as well as the time domain samples preceding and following the respective frame. For example, a lossy transmission of the spectrum of each frame in the form of frequency domain coefficients 24 may result in pre-echo artifacts. Thus, in a manner outlined further below, the frequency domain coefficients 24 of each frame 26 describe the spectrum of the audio signal in this frame 26 with a spectro-temporal resolution that is switchable by switching between different transform lengths. However, as far as the frequency domain coefficient extractor 12 is concerned, the latter situation is transparent to this. The frequency domain coefficient extractor 12 operates independently of any signaling that signals the just mentioned switching between different spectro-temporal resolutions of the frame 26.

[0014] The frequency domain coefficient extractor 12 may use entropy coding to extract the frequency domain coefficients 24 from the data stream 20. For example, the frequency domain coefficient extractor may use context-based entropy decoding, such as variable context arithmetic decoding, to extract the frequency domain coefficients 24 from the data stream 20 by assigning the same context to each of the frequency domain coefficients 24, regardless of the signaling described above that signals the spectro-temporal resolution of the frame 26 to which the respective frequency domain coefficient belongs. Alternatively, as a second example, the extractor 12 may use Huffman decoding to define a set of Huffman code words, regardless of the signaling described above that specifies the resolution of the frame 26.

[0015] There are several different possibilities how the frequency domain coefficients 24 describe the spectrogram 28. For example, the frequency domain coefficients 24 may simply represent some prediction residual. For example, the frequency domain coefficients may represent a prediction residual that has been obtained, at least in part, by stereo prediction from another audio signal representing a corresponding audio channel or a downmix from the multi-channel audio signal to which the signal spectrogram 28 belongs. Alternatively, or in addition to the prediction residual, the frequency domain coefficients 24 may represent a sum (middle) or difference (outer) signal according to the M / S stereo paradigm [5]. Furthermore, the frequency domain coefficients 24 may have undergone temporal noise shaping.

[0016] Moreover, the frequency domain coefficients 12 are quantized and spectrally modified in order to keep the quantization error below a psychoacoustic detection (or masking) threshold, e.g., the quantization step size is controlled via the respective scaling factors associated with the frequency domain coefficients 24. The scale factor extractor 14 is responsible for extracting the scaling factors from the data stream 20.

[0017] A little more detailed information on switching between different spectral-temporal resolutions from frame to frame will be briefly described below. As will be explained in more detail below, switching between different spectral-temporal resolutions indicates either that all frequency-domain coefficients 24 in a particular frame 26 belong to one transform, or that the frequency-domain coefficients 24 of each frame 26 actually belong to different transforms, for example two transforms whose transform length is half that of the one just mentioned. The embodiment described below in conjunction with the figures assumes switching between one transform on the one hand and two transforms on the other hand, but in practice switching between one transform and more than two transforms is in principle feasible, and the embodiment given below can be easily converted to such alternative embodiments.

[0018] Fig. 1 shows, using hatching, an exemplary case where the current frame is of a type represented by two short transforms, one of which is derived using the second half of the current frame 26 of the audio signal, and the other one is obtained by transforming the first half of the current frame 26 of the audio signal. Due to the shortened transform length, the spectral resolution with which the frequency domain coefficients 24 describe the spectrum of the frame 26 is reduced, i.e. halved when using two short transforms, while the time resolution is increased, i.e. doubled in this case. In Fig. 1, for example, the frequency domain coefficients 24 shown hatched belong to the preceding transform, and the frequency domain coefficients 24 not hatched belong to the following transform. Spectrally co-located frequency domain coefficients 24 thus describe the same spectral components of the audio signal in the frame 26, but at slightly different times, i.e. in two successive transform windows of the transform split frame.

[0019] In the data stream 20, the frequency domain coefficients 24 are transmitted in an interleaved manner, such that the spectrally corresponding frequency domain coefficients of two different transforms immediately follow each other. In other words, if the frequency domain coefficients 24 as received from the frequency domain coefficient extractor 12 are ordered consecutively as if they were the frequency domain coefficients of a long transform, then the frequency domain coefficients 24 of the split transform frame, i.e., the frame 26 for which the transform split is signaled in the data stream 20, are transmitted so that they are arranged in an interleaved manner in this sequence, such that the spectrally co-located frequency domain coefficients 24 are immediately adjacent to each other and such pairs of spectrally co-located frequency domain coefficients 24 are ordered according to the spectral / frequency order. Interestingly, when so ordered, the sequence of interleaved frequency domain coefficients 24 looks similar to the sequence of frequency domain coefficients 24 obtained by one long transform. Again, as far as the frequency domain coefficient extractor 12 is concerned, the switching between different transform lengths or spectro-temporal resolutions in units of frames 26 is transparent to it, so that the context selection for context-adaptive entropy coding of the frequency domain coefficients 24 results in the same context being selected, regardless of whether the current frame is actually a long transform without the extractor 12 knowing it, or whether the current frame is of split transform type. For example, the frequency domain coefficient extractor 12 can select the context to be utilized for a particular frequency domain coefficient based on already coded / decoded frequency domain coefficients that are spectro-temporally adjacent, this spectro-temporal adjacent being defined in the interleaved state shown in FIG. 1. This has the following consequences: Assume that the currently coded / decoded frequency domain coefficient 24 was part of a previous transform, which is shown using hatching in FIG. 1. The spectrally immediately adjacent frequency domain coefficients are then in fact frequency domain coefficients 24 of the same previous transform (i.e. the one with hatching in FIG. 1).However, the frequency domain coefficient extractor 12 nevertheless uses for context selection the frequency domain coefficients 24 belonging to the subsequent transform, i.e., those that are spectrally adjacent (according to the reduced spectral resolution of the shortened transform), assuming that they are spectrally immediate neighbors of one longer transform of the current frequency domain coefficient 24. Similarly, in the context selection for the frequency domain coefficients 24 of the subsequent transform, the frequency domain coefficient extractor 12 uses the frequency domain coefficients 24 belonging to the preceding transform and that are actually spectrally co-located with it as spectrally immediate neighbors. In particular, the decoding order defined among the coefficients 24 of the current frame 26 can, for example, proceed from the lowest frequency to the highest frequency. A similar observation is valid when the frequency domain coefficient extractor 12 is configured to entropy decode the frequency domain coefficients 24 of the current frame 26 in groups / tuples of immediately consecutive frequency domain coefficients 24 when ordered but not deinterleaved. Instead of using tuples of spectrally adjacent frequency domain coefficients 24 that belong only to the same short transform, the frequency domain coefficient extractor 12 may select a context for a particular spectrally adjacent tuple of mixed frequency domain coefficients 24 that belong to different short transforms based on such a spectrally adjacent tuple of mixed frequency domain coefficients 24 that belong to different short transforms.

[0020] As shown above, due to the fact that in the interleaved state, the resulting spectrum as obtained by two short transforms looks very similar to the spectrum obtained by one long transform, the entropy coding penalty resulting from the operation of the frequency domain coefficient extractor 12 independent of transform length switching is low.

[0021] As mentioned above, we resume the description of the decoder 10 with the scaling coefficient extractor 14 responsible for extracting the scaling coefficients for the frequency domain coefficients 24 from the data stream 20. The spectral resolution at which the scale coefficients are assigned to the frequency domain coefficients 24 is coarser than the relatively fine spectral resolution supported by the long transform. As indicated by the curly brackets 30, the frequency domain coefficients 24 can be grouped into a number of scale coefficient bands. The partitioning in the scale coefficient bands may be selected based on psychoacoustic considerations, for example coinciding with so-called Bark (or critical) bands. Since the scaling coefficient extractor 14 does not depend on transform length switching just as the frequency domain coefficient extractor 12 does, the scaling coefficient extractor 14 assumes that each frame 26 is partitioned into a number of equal scale coefficient bands 30 and extracts a scale coefficient 32 for each such scale coefficient band 30, regardless of the transform length switching signaling. At the encoder side, the assignment of the frequency domain coefficients 24 to these scale coefficient bands 30 is done in the non-deinterleaved state shown in FIG. 1. As a result, for a frame 26 corresponding to a split transform, each scale coefficient 32 belongs to a group that includes both frequency-domain coefficients 24 of the preceding transform and frequency-domain coefficients 24 of the succeeding transform.

[0022] The inverse transformer 16 is configured to receive, for each frame 26, the corresponding frequency domain coefficients 24 and the corresponding scale factors 32, and to inverse transform the frequency domain coefficients 24 of the frame 26, scaled according to the scale factors 32, to obtain a time domain portion of the audio signal. The inverse transformer 16 may use, for example, a lapped transform, such as the modified discrete cosine transform (MDCT). The combiner 18 combines the time domain portions, for example by using a suitable overlap-add technique, to obtain the audio signal. The overlap-add technique, for example, provides time domain aliasing cancellation among overlapping portions of the time domain portions output by the inverse transformer 16.

[0023] Of course, the inverse transformer 16 is responsive to the previously described transform length switches signaled in the data stream 20 for the frames 26. The operation of the inverse transformer 16 will now be described in more detail with reference to FIG.

[0024] Figure 2 shows a possible internal structure of the inverse transformer 16 in more detail. As shown in Figure 2, the inverse transformer 16 receives, for a current frame, the frequency domain coefficients 24 associated with that frame and corresponding scale factors 32 for dequantizing the frequency domain coefficients 24. Additionally, the inverse transformer 16 is controlled by signaling 34 present in the data stream 20 for each frame. The inverse transformer 16 can be further controlled via other components of the data stream 20 that are optionally included within the data stream 20. Details regarding these additional parameters are provided in the following description.

[0025] As shown in Fig. 2, the inverse transformer 16 of Fig. 2 comprises an inverse quantizer 36, an activatable deinterleaver 38 and an inverse transform stage 40. To facilitate understanding of the following description, the incoming frequency domain coefficients 24 as derived for the current frame from the frequency domain coefficient extractor 12 are shown labeled 0 to N-1. Again, since the frequency domain coefficient extractor 12 is agnostic to the signaling 34, i.e. operates independently of the signaling 34, the frequency domain coefficient extractor 12 provides the frequency domain coefficients 24 to the inverse transformer 16 in the same way regardless of whether the current frame is of split transform type or one transform type, i.e. the number of frequency domain coefficients 24 is N in this example, and the association of the indexes 0 to N-1 to the N frequency domain coefficients 24 also remains the same regardless of the signaling 34. If the current frame is of 1 or long transform type, the indices 0 to N-1 correspond to an ordering of the frequency domain coefficients 24 from lowest to highest frequency, and if the current frame is of split transform type, the indices correspond to an ordering for the frequency domain coefficients, where the frequency domain coefficients are then spectrally arranged according to spectral order, but interleaved such that every second and every other frequency domain coefficient 24 belongs to a subsequent transform, while the other frequency domain coefficients 24 belong to a preceding transform.

[0026] The same applies to the scale coefficients 32. Because the scale coefficient extractor 14 operates independent of the signaling 34, the number and order as well as the values ​​of the scale coefficients 32 coming from the scale coefficient extractor 14 are independent of the signaling 34, and the scale coefficients 32 in FIG. 2 are exemplarily numbered S0 to S1 with indices corresponding to the consecutive order among the scale coefficient bands to which they are associated. M is shown as:

[0027] Similar to the frequency domain coefficient extractor 12 and the scale factor extractor 14, the inverse quantizer 36 can operate independent of or on the signaling 34. The inverse quantizer 36 inverse quantizes or scales the incoming frequency domain coefficients 24 using scale factors associated with the scale factor band to which the respective frequency domain coefficient belongs. Again, the membership of the incoming frequency domain coefficients 24 to the individual scale factor bands, and thus the association of the incoming frequency domain coefficients 24 to the scale factors 32, is independent of the signaling 34, and thus the inverse transformer 16 scales the frequency domain coefficients 24 with the scale factors 32 at a spectral resolution independent of the signaling 34. For example, the inverse quantizer 36 may assign the frequency domain coefficients indices 0-3 for the first scale factor band, thus the first scale factor being S0, indices 4-9 for the second scale factor band, thus the scale factor being S1, etc., independent of the signaling 34. The scale factor boundaries are intended to be exemplary only. The inverse quantizer 36 may, for example, perform multiplications with the associated scale factors to inverse quantize the frequency domain coefficients 24, i.e., x0 to x0·s0, x1 to x1·s0, ... x3 to x3·s0, x4 to x4·s1, ... x9 to x9·s1, and so on. Alternatively, the inverse quantizer 36 may perform an interpolation of the scale factors actually used to inverse quantize the frequency domain coefficients 24 from a coarse spectral resolution defined by the scale factor bands. The interpolation may be independent of the signaling 34. However, the latter interpolation may alternatively be signaling dependent to take into account different spectro-temporal sampling positions of the frequency domain coefficients 24 depending on whether the current frame is of the split transform type or the 1 / long transform type.

[0028] FIG. 2 shows that up to the input of the activatable deinterleaver 38, the order among the frequency domain coefficients 24 remains the same, and the same is at least partially true with respect to the overall operation up to that point. FIG. 2 shows that further operations can be performed by the inverse transformer 16 upstream of the activatable deinterleaver 38. For example, the inverse transformer 16 can be configured to perform noise filling on the frequency domain coefficients 24. For example, in the sequence of frequency domain coefficients 24, scale factor bands, i.e. groups of frequency domain coefficients that have come in order according to indexes 0 to N-1, can be identified, where all frequency domain coefficients 24 of each scale factor band are quantized to zero. Such frequency domain coefficients can be filled using artificial noise generation, for example using a pseudorandom number generator. The intensity / level of the noise filled in the zero quantized scale factor bands can be adjusted using the scale factor of the respective scale factor band, since it is not needed for scaling since the spectral coefficients inside are all zero. Such noise filling is shown at 40 in FIG. 2 and is described in more detail in one embodiment in European Patent Application Publication No. EP2304719A1 [6].

[0029] Fig. 2 further shows that the inverse transformer 16 can be configured to support joint stereo coding and / or inter-channel stereo prediction. In the framework of inter-channel stereo prediction, the inverse transformer 16 can predict 42 the spectrum of the deinterleaved sequence represented by the order of indices 0 to N-1, for example, from another channel of the audio signal. This can mean that the frequency domain coefficients 24 describe a spectrogram of a channel of the stereo audio signal, and that the inverse transformer 16 is configured to process the frequency domain coefficients 24 as a prediction residual of a prediction signal derived from the other channel of this stereo audio signal. This inter-channel stereo prediction can be performed, for example, at a certain spectral granularity independent of the signaling 34. Complex prediction parameters 44 controlling the complex stereo prediction 42 can trigger the complex stereo prediction 42, for example, for a particular one of the scale factor bands mentioned above. For each scale factor band for which complex prediction is initiated by the complex prediction parameters 44, the scaled frequency domain coefficients 24 ordered from 0 to N-1 present in the respective scale factor band are summed with an inter-channel predicted signal obtained from the other channel of the stereo audio signal. The complex coefficients contained in the complex prediction parameters 44 for the respective scale factor band can control the predicted signal.

[0030] Furthermore, within the framework of joint stereo coding, the inverse transformer 16 can be configured to perform MS decoding 46. That is, the decoder 10 of FIG. 1 can perform the operations described so far twice, once for the first channel of the stereo audio signal and once for the second channel, and controlled via the MS parameters in the data stream 20, the inverse transformer 16 can MS decode these two channels or leave them as they are, i.e. as the left and right channels of the stereo audio signal. The MS parameters 48 can switch between MS coding at the frame level, or even at some finer level, such as per scale factor band or group thereof. For example, in case of MS decoding being triggered, the inverse transformer 16 can form the sum or difference of the corresponding frequency domain coefficients 24 in the order 0 to N-1 of the coefficients with the corresponding frequency domain coefficients of the other channel of the stereo audio signal.

[0031] Thus, Fig. 2 shows that the activatable deinterleaver 38 responds to the signaling 34 for the current frame as follows: deinterleaving the incoming frequency domain coefficients to obtain two transforms, i.e., a leading transform 50 and a trailing transform 52, when the current frame is signaled to be a divided transform frame by the signaling 34, and leaving the frequency domain coefficients interleaved to yield one transform 54 when the signaling 34 indicates that the current frame is a long transform frame. When deinterleaving, the deinterleaver 38 forms one of the transforms 50 and 52, i.e., one short transform from the frequency domain coefficients with even indices and the other short transform from the frequency domain coefficients at odd index positions. For example, the even-indexed frequency domain coefficients form the leading transform (starting at index 0), while the other frequency domain coefficients form the trailing transform. Those transforms 50 and 52 undergo an inverse transform of the shorter transform length resulting in time domain portions 56 and 58, respectively. The combiner 18 of Fig. 1 positions the time domain portions 56 and 58 correctly in time, i.e. positions the time domain portion 56 resulting from the preceding transform 50 before the time domain portion 58 resulting from the following transform 52, and performs an overlap-add process between them with the time domain portions resulting from the preceding and following frames of the audio signal. If not deinterleaved, the frequency domain coefficients arriving at the interleaver 38 would directly form the long transform 54, and the inverse transform stage 40 performs an inverse transform on said frequency domain coefficients, such that the inverse transform stage 40 results in a time domain portion 60 that spans the entire time interval of the current frame 26 and beyond. The combiner 18 combines the time domain portion 60 with the respective time domain portions resulting from the preceding and following frames of the audio signal.

[0032] The frequency domain audio decoders described so far allow transform length switching to allow compatibility with frequency domain audio decoders that do not support signaling 34. In particular, such "legacy" decoders may erroneously assume that the frames signaled by signaling 34 are of long transform type when in fact they are of split transform type. That is, they may erroneously leave the split type frequency domain coefficients interleaved and perform a long transform length inverse transform. However, the resulting quality of the affected frames of the reconstructed audio signal is still quite reasonable.

[0033] In turn, the coding efficiency penalty is still quite reasonable. The coding efficiency penalty comes from ignoring the signaling 34 since the frequency domain coefficients and scale factors are coded without taking into account the meaning of the various coefficients and without exploiting this variation to increase the coding efficiency. However, the latter penalty is relatively small compared to the advantage of allowing backward compatibility. The latter statement also applies with respect to the restriction to start and stop the noise filler 40, the complex stereo prediction 42 and the MS decoding 46 only within a continuous spectral part (scale factor band) in the deinterleaved state defined by the indexes 0 to N-1 in Fig. 2. The opportunity to allow control of these coding tools specifically for a type of frame (e.g. with two noise levels) may bring advantages in some cases, but these advantages are overcompensated by the advantage of having backward compatibility.

[0034] FIG. 2 shows that the decoder of FIG. 1 can be further configured to support TNS (Temporal Noise Shaping) coding while still maintaining backward compatibility with decoders that do not support the signaling 34. In particular, FIG. 2 shows that inverse TNS filtering, if any, may occur after any complex stereo prediction 42 and MS decoding 46. To maintain backward compatibility, the inverse transformer 16 is configured to perform inverse TNS filtering 62 on the sequence of N coefficients, regardless of the signaling 34, using the respective TNS coefficients 64. With this approach, the data stream 20 codes the TNS coefficients 64 equally, regardless of the signaling 34; i.e., the number of TNS coefficients and the manner in which they are coded are the same. However, the inverse transformer 16 is configured to apply the TNS coefficients 64 differently. If the current frame is a long transform frame, the inverse TNS filtering is performed on the long transform 54, i.e. the interleaved sequence of frequency domain coefficients, and if the current frame is signaled by the signaling 34 as a divided transform frame, the inverse transformer 16 inverse TNS filters 62 the concatenation of the preceding transform 50 and the subsequent transform 52, i.e. the sequence of frequency domain coefficients with indices 0, 2, ..., N-2, 1, 3, 5, ..., N-1. The inverse TNS filtering 62 can, for example, comprise the inverse transformer 16 applying a filter whose transfer function is set according to the TNS coefficients 64 on the deinterleaved or interleaved sequence of coefficients passed through the processing sequence upstream of the deinterleaver 38.

[0035] Thus, an "old" decoder that erroneously processes a split frame type frame as a long transform frame will apply the TNS coefficients 64 that have been generated by the encoder by analyzing the concatenation of two real-time transforms, 50 and 52, to transform 54, thus generating an inaccurate time domain portion 60 due to the inverse transform applied to transform 54. However, if one restricts the use of such split transform frames to cases where the signal represents rain or applause, etc., then even if this degradation in quality does occur in such a decoder, it may be tolerable to a listener.

[0036] For completeness, Fig. 3 shows that the inverse TNS filtering 62 of the inverse transformer 16 can be inserted elsewhere in the processing sequence shown in Fig. 2. For example, the inverse TNS filtering 62 can be placed upstream of the complex stereo prediction 42. In order to preserve the deinterleaved domain downstream and upstream of the inverse TNS filtering 62, Fig. 3 shows that in the case where the frequency domain coefficients 24 have only been deinterleaved 66 beforehand, the deinterleaving is reversed 70 to obtain the frequency domain coefficients again in their inverse TNS filtered version in their interleaved order 0, 1, 2, ..., N-1, in order to perform the inverse TNS filtering 68 in the deinterleaved concatenated state in which the frequency domain coefficients 24 as processed so far are in the order of indexes 0, 2, 4, ..., N-2, 1, 3, ..., N-3, N-1. The position of the inverse TNS filtering 62 in the processing step sequence shown in Fig. 2 may be fixed or may be signaled via the data stream 20, such as, for example, on a frame-by-frame basis or at some other granularity.

[0037] It should be noted that, for ease of explanation, the above embodiment focuses only on the juxtaposition of long transform frames and split transform frames. However, the embodiment of the present application can be extended by introducing frames of other transform types as well, such as frames consisting of eight short transforms. In this connection, it should be noted that the aforementioned independence only concerns frames that are distinguished from such other frames of any third transform type by further signaling, whereby a "legacy" decoder would erroneously process a split transform frame as a long transform frame by inspecting the further signaling contained in all frames, and only frames that are distinguished from other frames (all except split transform and long transform frames) include the signaling 34. As far as such other frames (all except split transform and long transform frames) are concerned, it should be noted that the operation modes of the extractors 12 and 14, such as context selection, etc., may depend on the further signaling, i.e., such operation modes may be different from the operation modes applied to split transform and long transform frames.

[0038] Before describing a suitable encoder compatible with the above-mentioned decoder embodiments, we will describe an implementation of the above-mentioned embodiments that is suitable for adaptively updating an xHE-AAC-based audio encoder / decoder to enable it to support backward-compatible transform splitting.

[0039] That is, in the following, a possibility of implementing transform length partitioning in an audio codec based on MPEG-D xHE-AAC (USAC) is described in order to improve the coding quality of a particular audio signal at low bit rates. The transform partitioning tool is signaled in a semi-backward compatible manner so that a legacy xHE-AAC decoder can parse and decode the bitstream according to the above embodiment without obvious audio errors or loss. As shown below, this semi-backward compatible signaling makes use of the unused possible values ​​of frame syntax elements that control the usage of noise filling in a conditional coding manner. Legacy xHE-AAC decoders do not support these possible values ​​of the respective noise filling syntax elements, whereas the improved audio decoder does.

[0040] In particular, the embodiment described below, in line with the above-mentioned embodiment, allows to provide intermediate transform lengths for coded signals similar to rain or applause, preferably split long blocks, i.e. two successive transforms, each half or quarter of the spectral length of the long block, with the maximum temporal overlap between these transforms being smaller than the maximum temporal overlap between successive long blocks. In order to allow the coded bitstream with transform splitting, i.e. signaling 34, to be read and parsed by legacy xHE-AAC decoders, the splitting should be used semi-backward compatible, and the presence of such a transform splitting tool should not cause legacy decoders to stop decoding or even to not start decoding. The readability of such bitstreams by the xHE-AAC infrastructure can also facilitate market adoption. To achieve the just mentioned semi-backward compatibility objective for using transform splitting for xHE-AAC or its possible derivatives, the transform splitting is signaled via the noise-filled signaling of xHE-AAC. According to the above-mentioned embodiment, to construct the transform splitting for the xHE-AAC coder / decoder, instead of the frequency domain (FD) stop-start window sequence, a split transform consisting of two separate half-length transforms can be used. The temporally consecutive half-length transforms are interleaved into a single stop-start-like block per coefficient for decoders that do not support transform splitting, i.e., legacy xHE-AAC decoders. The signaling via noise filling signaling is performed as described below. In particular, 8-bit noise filling side information can be used to signal the transform splitting. This is feasible because the MPEG-D standard [4] states that all 8 bits are transmitted even if the noise level to be applied is zero. In that situation, some of the noise filling bits can be reused for the transform splitting, i.e., signaling 34.

[0041] Semi-backward compatibility for bitstream parsing and playback by legacy xHE-AAC decoders can be ensured as follows: the transform split is signaled via a noise level of zero, i.e. the first three noise-filling bits have a value of all zero, followed by five non-zero bits (conventionally representing a noise offset) that contain side information about the transform split and the noise level that is lost. Since legacy xHE-AAC decoders ignore the value of the 5-bit offset when the 3-bit noise level is zero, the presence of the transform split signaling 34 only affects the noise filling in legacy decoders. That is, since the first three bits are zero, the noise filling is turned off and the remaining decoding operations work as intended. In particular, the split transform is processed like a conventional stop-start block with a full-length inverse transform (due to the coefficient interleaving described above) and no deinterleaving is performed. Thus, a conventional decoder still allows a "graceful" decoding of the improved data stream / bitstream 20, since it does not have to attenuate the output signal 22 or even interrupt the decoding when a transform split type frame arrives. Of course, such a conventional decoder is not able to provide an exact reconstruction of the split transform frames, resulting in a degraded quality in the affected frames compared to the decoding by a suitable decoder, for example, according to Fig. 1. Nevertheless, assuming that transform splitting is used as intended, i.e. only for transient or noisy inputs at low bit rates, the quality by the xHE-AAC decoder should be better than if the affected frames were dropped due to attenuation or would otherwise result in obvious playback errors.

[0042] Specifically, the extension of the xHE-AAC encoder / decoder towards transform splitting can be as follows:

[0043] According to the above description, the new tool to be used in xHE-AAC can be called transform splitting (TS). Transform splitting is a new tool in the frequency domain (FD) coder of xHE-AAC or, for example, in MPEG-H 3D-Audio, which is based on USAC [4]. Transform splitting can then be used for certain transient signal passages as an alternative to the usual long transform (which leads to temporal blurring, especially pre-echoes, at low bit rates) or to eight short transforms (which lead to spectral holes and bubble artifacts at low bit rates). Transform splitting can then be signaled semi-backward compatible by interleaving the FD coefficients into a long transform that can be parsed correctly by a traditional MPEG-D USAC decoder.

[0044] The description of this tool is similar to that above. When transform splitting is active in a long transform, two half-length MDCTs are utilized instead of one full-length MDCT, and the coefficients of the two MDCTs, i.e. 50 and 52, are transmitted line-by-line interleaved. Interleaved transmission is already used, for example, in the case of frequency domain (stop) start transforms, where the coefficients of the first MDCT in time are placed at even indices and the coefficients of the second MDCT in time are placed at odd indices (when indexing starts at zero), but a decoder that is not capable of processing stop-start transforms is not able to correctly parse the data stream. That is, any decoder that is not capable of supporting stop-start windows had to ignore the respective stop-start windowed frames, since the different contexts used to entropy code the frequency domain coefficients are valid for such stop-start transforms, i.e. a changed syntax streamlined to half transforms.

[0045] Briefly referring back to the embodiment described above, this may enable the decoder of Fig. 1 to support partitioning of a particular frame 26 beyond the description presented thus far or into more than two transforms using signaling that extends further transform lengths, i.e., signaling 34. However, with respect to the concurrence of transform partitioning of a frame 26 other than the partition transform initiated using signaling 34, the FD coefficient extractor 12 and the scaling coefficient extractor 14 are responsive to this signaling in that their operation modes change in response to further signaling in addition to signaling 34. Furthermore, the streamlined transmission of TNS coefficients, MS parameters and complex prediction parameters adjusted for signaled transform types other than the partition transform types according to 56 and 59 requires that each decoder must be able to respond to, i.e., understand, the signaling selection between frames containing these "known transform types" or long transform types according to 60 and other transform types such as, for example, one partitioned frame into eight short transforms as in the case of AAC. This "known signaling" then identifies frames in which signaling 34 signals a split transform type as long transform type frames, so that a decoder not capable of understanding signaling 34 will process these frames as long transform frames rather than other types of frames, such as the eight short transform type frames.

[0046] Returning again to the description of possible extensions of xHE-AAC, certain operational restrictions may arise in order to incorporate transform splitting tools into this coding framework. For example, transform splitting may be allowed to be used only in long start or stop-start windows in the frequency domain, i.e., the underlying syntax element window_sequence may be required to be equal to 1. In addition, due to semi-backward compatible signaling, there may be a requirement that transform splitting can only be applied when the syntax element noiseFilling is 1 in the syntax container UsacCoreConfig(). When transform splitting is signaled to be active, all frequency domain tools except TNS and inverse MDCT operate on interleaved (long) TS coefficient sets. This allows reusing the scale factor band offset and long transform arithmetic coder tables as well as the window shapes and overlap lengths.

[0047] In the following, we present the terms and definitions that will be used below to explain how the USAC standard described in [4] can be extended to provide a backward-compatible transformation splitting facility. For the interested reader, we may refer to sections within that standard.

[0048] The new data elements may be: split_transform: A binary flag indicating whether a split transform is used for the current frame and channel.

[0049] The new auxiliary elements may be: window_sequence: Frequency domain window sequence type for the current frame and channel (Section 6.2.9) noise_offset: A noise filling offset to modify the scale factor of the zero quantization band (section 7.2). noise_level: The noise filling level that describes the amount of spectral noise added (Section 7.2). half_transform_length: half of coreCoderFrameLength (ccfl, transform length, section 6.1.1) half_lowpass_line: Half the number of MDCT lines transmitted for the current channel

[0050] Decoding of a frequency domain (stop-)start transform using transform splitting (TS) in the USAC framework can be performed in purely sequential steps as follows.

[0051] First split_transform and half_lowpass_line decoding can be performed.

[0052] split_transform does not actually represent a separate bitstream element, but is derived from the noise filling factors, noise_offset and noise_level, and, in the case of UsacChannelPairElement(), the common_window flag in StereoCoreToolInfo(). If noiseFilling == 0, then split_transform is 0. Otherwise:

number

[0053] In other words, if noise_level == 0, noise_offset contains the split_transform flag, followed by 4 bits of noise filling data, which are then reordered. This operation must be performed before the noise filling process of section 7.2, since it changes the values ​​of noise_level and noise_offset. Furthermore, if common_window == 1 in UsacChannelPairElement(), split_transform is determined only in the left (first) channel, the right channel's split_transform is set equal to (and replicated from) the left channel's split_transform, and the above pseudocode is not performed on the right channel.

[0054] half_lowpass_line is determined from the "long" scale factor band offset table swb_offset_long_window and the max_sfb of the current channel, or max_sfb_ste if stereo and common_window == 1.

[0055] In elements with StereoCoreToolInfo() and common_window == 1, max_sfb_ste, else lowpass_sfb =max_sfb. Based on the igFilling flags, half_lowpass_line is derived as follows:

number

[0056] Then, as a second step, half-length spectral deinterleaving for temporal noise shaping is performed.

[0057] After application of spectral dequantization, noise filling, and scale factors, but before application of Temporal Noise Shaping (TNS), the TS coefficients in spec[] are deinterleaved using a helper buffer[].

number

[0058] In-place deinterleaving effectively places the two half-length TS spectra on top of each other, and the TNS tools operate normally on the resulting full-length pseudospectrum.

[0059] See above, such a procedure is described in relation to FIG.

[0060] Then, as a third step, temporal re-interleaving is used with two successive inverse MDCTs.

[0061] If common_window == 1 in the current frame or stereo decoding is performed after TNS decoding (tns_on_lr == 0 in Section 7.8), spec[] must be time-reinterleaved into a full-length spectrum.

number

[0062] The resulting pseudospectrum is used for stereo decoding (Section 7.7) and dmx_re_prev[] is updated (Sections 7.7.2 and A.1.4). If tns_on_lr == 0, the stereo decoded full-length spectrum is deinterleaved again by repeating the process of Section A.1.3.2. Finally, two inverse MDCTs are computed using the ccfl and the window_shape of that channel of the current and last frames. See Section 7.9 and Figure 1.

[0063] Some modifications can be made to the complex predictive stereo decoding of xHE-AAC.

[0064] To incorporate TS within xHE-AAC, an implicit semi-backward compatible signaling method can be used as an alternative.

[0065] Above, we described a technique of using one bit in the bitstream to signal the use of the inventive transform splitting in split_transform to the inventive decoder. In particular, such signaling (called explicit semi-backward compatible signaling) allows the following legacy bitstream data (herein, noise-filling side information) to be used independently of the inventive signal. That is, in an embodiment of the present invention, the noise-filling data does not depend on the transform splitting data, and vice versa. For example, noise-filling data consisting of all zeros (noise_level = noise_offset = 0) can be sent, while split_transform can hold any possible value (being a binary flag, either 0 or 1).

[0066] As such, if strict independence between the legacy bitstream data and the inventive bitstream data is not required and the inventive signaling is a binary decision, it is possible to avoid explicitly sending a signaling bit, which can be signaled by the presence or absence of what can be called implicit semi-backward compatible signaling. Taking the above embodiment as an example again, the usage of transform splitting can be signaled simply by using the inventive signaling. That is, if noise_level is zero and at the same time noise_offset is non-zero, split_transform is set equal to 1. If both noise_level and noise_offset are non-zero, split_transform is set equal to 0. When both noise_level and noise_offset are zero, a dependency of the inventive implicit signaling on the legacy noise-filling signal occurs. In this case, it is unclear whether the legacy implicit signaling or the inventive implicit signaling is used. To avoid such ambiguity, the value of split_transform must be specified in advance. In this example, where the noise filling data consists of all zeros, it is appropriate to specify split_transform = 0, since this is what a legacy encoder without transform splitting should signal when noise filling should not be used in a frame.

[0067] The problem that remains to be solved in the case of implicit semi-backwards compatible signaling is how to simultaneously signal split_transform == 1 and no noise filling. As mentioned before, the noise filling data must not be all zero, and if a zero noise magnitude is desired, noise_level ((noise_offset & 14) / 2 as above) must be equal to 0. This allows for noise_offset greater than 0 ((noise_offset & 1 as above) *16) remains as the only solution. Advantageously, if noise filling is not performed in a decoder based on USAC[4], the value of noise_offset is ignored, so it turns out that this approach is feasible in an embodiment of the present invention. Therefore, the signaling of split_transform in the pseudocode above can be modified as follows, using the TS signaling bits reserved for transmitting noise_offset to transmit two bits (four values) instead of one bit for noise_offset:

number

[0068] Therefore, applying this alternative, the USAC description can be expanded using the following explanation.

[0069] The tool descriptions are broadly the same: When transform splitting (TS) is active in the long transform, two half-length MDCTs are utilized instead of one full-length MDCT. The coefficients of the two MDCTs are transmitted line-by-line interleaved as in a conventional frequency domain (FD) transform, with the coefficients of the first MDCT in time being placed at the even indices and the coefficients of the second MDCT in time being placed at the odd indices.

[0070] Operational restrictions may require that TS can only be used in FD long-start or stop-start windows (window_sequence == 1), and that TS can only be applied when noiseFilling is 1 in UsacCoreConfig(). When TS is signaled, all FD tools except TNS and inverse MDCT operate on interleaved (long) TS coefficient sets. This allows reusing scale factor band offsets and long transform arithmetic coder tables as well as window shapes and overlap lengths.

[0071] The terms and definitions used below include the following subsections: common_window: Indicates if channel 0 and channel 1 of the CPE use the same window parameters (see ISO / IEC 23003-3:2012 Section 6.2.5.1.1). window_sequence: FD window sequence type for the current frame and channel (see ISO / IEC 23003-3:2012 section 6.2.9). tns_on_lr: Indicates the operation mode of TNS filtering (see ISO / IEC 23003-3:2012 section 7.8.2). noiseFilling: This flag signals the usage of noise filling of spectral holes in the FD core encoder (see ISO / IEC 23003-3:2012 Section 6.1.1.1). noise_offset: Noise filling offset (see ISO / IEC 23003-3:2012 section 7.2) to modify the scale factor of the zero quantization band. noise_level: The noise filling level (see ISO / IEC 23003-3:2012 section 7.2) which describes the amount of spectral noise added. split_transform: A binary flag indicating whether TS is used in the current frame and channel. half_transform_length: half of coreCoderFrameLength (ccfl, transform length, see ISO / IEC 23003-3:2012 section 6.1.1). half_lowpass_line: Half the number of MDCT lines transmitted for the current channel.

[0072] The decoding process including TS can be described as follows: In particular, the decoding of the FD(Stop-)Start conversion with TS is performed in three successive steps as follows:

[0073] First, decoding of split_transform and half_lowpass_line is performed. The auxiliary element split_transform does not represent an independent bitstream element, but is derived from the noise filling elements noise_offset and noise_level and, in the case of UsacChannelPairElement(), the common_window flag in StereoCoreToolInfo(). If noiseFilling == 0, then split_transform is 0. Otherwise:

number

[0074] In other words, if noise_level == 0, noise_offset contains the split_transform flag, followed by 4 bits of noise filling data, which are then reordered. This operation changes the values ​​of noise_level and noise_offset and therefore must be performed before the noise filling process of ISO / IEC 23003-3:2012 section 7.2.

[0075] Furthermore, if common_window == 1 in UsacChannelPairElement(), then split_transform is determined only on the left (first) channel, the right channel's split_transform is set equal to (and replicated from) the left channel's split_transform, and the above pseudocode is not executed on the right channel.

[0076] The auxiliary element half_lowpass_line is determined from the "long" scale factor band offset table, swb_offset_long_window and max_sfb of the current channel, or max_sfb_ste if stereo and common_window == 1.

number

[0077] Based on the igFilling flags the half_lowpass_line is derived as follows:

number

[0078] Afterwards, half-length spectral de-interleaving for temporal noise shaping is performed.

[0079] After application of spectral dequantization, noise filling, and scale factors, but before the application of temporal noise shaping (TNS), the TS coefficients in spec[ ] are deinterleaved using the helper buffer[ ].

number

[0080] In-place deinterleaving effectively places the two half-length TS spectra on top of each other, and the TNS tool then operates normally on the resulting full-length pseudospectrum.

[0081] Finally, time re-interleaving and two successive inverse MDCTs can be used.

[0082] If common_window == 1 in the current frame or stereo decoding is performed after TNS decoding (tns_on_lr == 0 in Section 7.8), spec[] must be time-reinterleaved into a full-length spectrum.

number

[0083] The resulting pseudospectrum is used for stereo decoding (ISO / IEC 23003-3:2012 section 7.7), dmx_re_prev[] is updated (ISO / IEC 23003-3:2012 section 7.7.2), and if tns_on_lr == 0, the stereo decoded full-length spectrum is deinterleaved again by repeating the process in that section. Finally, two inverse MDCTs are computed using the ccfl and the window_shape of that channel for the current and last frames.

[0084] The processing for the TS shall follow the description given in ISO / IEC 23003-3:2012 section "7.9 Filter banks and block switching". The following additions shall be taken into consideration:

[0085] The TS coefficients in spec[] are deinterleaved using a helper buffer[] with a window length N based on the window_sequence value.

number

[0086] In this case, the IMDCT for the half-length TS is defined as follows:

number

[0087] The subsequent windowing and block switching steps are defined in the next subsections.

[0088] A transformation split by STOP_START_SEQUENCE looks like the following description:

[0089] The STOP_START_SEQUENCE combined with transform splitting is shown in Figure 2. It includes two overlap-and-add half-length windows 56, 58 with length N_l / 2, which is 1024 (960, 768), with N_s set to 256 (240, 192), respectively.

[0090] The window (0,1) for the two half-length IMDCTs is given by:

number

number

number

[0091] The overlap and addition between two half-long windows resulting in the windowed time-domain values ​​zi,n is described as follows: where N_l is set to 2048 (1920, 1536) and N_s is set to 256 (240, 192), respectively.

number

[0092] A transformation split by LONG_START_SEQUENCE looks like the following description:

[0093] The LONG_START_SEQUENCE combined with transform splitting is shown in Figure 4. It contains three windows defined as follows, where N_l is set to 1024 (960, 768) and N_s is set to 256 (240, 192), respectively:

number

number

[0094] The left / right window halves are given by:

number

number

[0095] The third window is equal to the left half of LONG_START_WINDOW.

number

number

[0096] Intermediate windowed time domain values

number

number

[0097] By applying W2, the final windowed time domain value Z i,n is obtained.

number

[0098] Regardless of whether the semi-backward compatible signaling used is explicit or implicit (both described above), some modifications may be necessary to the complex predictive stereo decoding of xHE-AAC in order to achieve meaningful operation on the interleaved spectrum.

[0099] The modification to the complex predictive stereo decoding can be implemented as follows.

[0100] When TS is active in a channel pair, no changes are needed to the underlying M / S or complex prediction process, since the FD stereo tool operates on an interleaved pseudospectrum. However, the derivation of the previous frame's downmix dmx_re_prev[] and the calculation of the downmix MDST dmx_im[] in ISO / IEC 23003-3:2012 section 7.7.2 need to be adapted if TS is used in either channel of the last or current frame.

[0101] If the TS has changed actively in any channel from the last to the current frame, use_prev_frame must be 0. In other words, dmx_re_prev[] must not be used in that case due to the transformation length.

[0102] If a TS was or is active, then dmx_re_prev[] and dmx_re[] specify interleaved pseudospectra which must be deinterleaved into their corresponding two half-length TS spectra for accurate MDST computation.

[0103] When TS is active, two half-length MDST downmixes are computed using the adapted filter coefficients (Tables 1 and 2) and interleaved into the full-length spectrum dmx_im[] (just like dmx_re[]).

[0104] · window_sequence: Downmix MDST estimates are computed for each group window pair. use_prev_frame is evaluated only for the first half-window pair out of two half-window pairs. For the remaining window pairs, the previous window pair is always used for MDST estimation, which implies use_prev_frame = 1.

[0105] Window shape: The MDST estimation parameters for the current window are the filter coefficients as described below and depend on the shapes of the left and right window halves. For the first window, this means that the filter parameters are a function of the window_shape flags of the current and previous frames. The remaining windows are only affected by the current window_shape.

[0106] [Table 1]

[0107] [Table 2]

[0108] Finally, for the sake of completeness, Fig. 5 shows a possible frequency domain audio encoder supporting transform length switching, conforming to the embodiments outlined above: the encoder of Fig. 5, generally indicated using reference number 100, is capable of encoding an audio signal 102 into a data stream 20, such that the decoder of Fig. 1 described above and corresponding variants can utilize a transform splitting mould for some frames, while "legacy" decoders can still process the TS frames without parsing errors etc.

[0109] The encoder 100 of Fig. 5 comprises a transformer 104, an inverse scaler 106, a frequency domain coefficient inserter 108 and a scale factor inserter 110. The transformer 104 is arranged to receive an audio signal 102 to be coded and to transform a time domain portion of the audio signal to obtain frequency domain coefficients of frames of the audio signal. In particular, as becomes clear from the above description, the transformer 104 decides for each frame which partitioning of these frames 26 into transforms, or transform windows, will be used. As explained above, the frames 26 may be of equal length and the transform may be a lapped transform using overlapping transforms of different lengths. Fig. 5 illustrates, for example, a case where frame 26a undergoes one long transform, frame 26b undergoes transform splitting, i.e. two transforms of half length, and a further frame 26c undergoes two transforms of long transform length. -n Length of 3 or more, i.e. 2 n >2 even shorter transforms. As mentioned above, this strategy enables the encoder 100 to adapt the spectro-temporal resolution of the spectrogram represented by the lapped transforms implemented by the transformer 104 to the time-varying audio content or type of audio content of the audio signal 102.

[0110] That is, frequency domain coefficients representing the spectrogram of the audio signal 102 are provided at the output of the transformer 104. The inverse scaler 106 is connected to the output of the transformer 104 and is configured to inversely scale and simultaneously quantize the frequency domain coefficients according to a scale factor. In particular, the inverse scaler operates on the frequency coefficients as they are obtained by the transformer 104. That is, the inverse scaler 106 necessarily needs to know the transform length assignment or the transform mode assignment for the frame 26. It is also noted that the inverse scaler 106 needs to determine the scale factor. For this purpose, the inverse scaler 106 is for example part of a feedback loop that evaluates the psychoacoustic masking threshold determined for the audio signal 102 and keeps the quantization noise introduced by the quantization and progressively set according to the scale factor as much below the psychoacoustic detection threshold, with or without any bitrate limitation.

[0111] At the output of the inverse scaler 106 the scale factor and the inverse scaled and quantized frequency domain coefficients are provided, the scale factor inserter 110 is configured to insert the scale factor into the data stream 20, and the frequency domain coefficient inserter 108 is configured to insert the frequency domain coefficients of the frame of the audio signal, inverse scaled and quantized according to the scale factor, into the data stream 20. To be compatible with the decoder, both inserters 108 and 110 operate independently of the transform mode associated with the frame 26 as far as the juxtaposition of the frame 26a in the long transform mode and the frame 26b in the transform split mode is concerned.

[0112] In other words, the inserters 110 and 108 operate independently of the above-mentioned signaling 34 that the converter 104 is configured to transmit in or insert into the data stream 20 for frames 26a and 26b, respectively.

[0113] In other words, in the above embodiment, it is the transformer 104 that arranges the transform coefficients of the long transform and split transform frames appropriately, i.e. by planar consecutive arrangement or interleaving, and the inserter actually operates independently of 109. However, in a more general sense, it is sufficient that the independence of the frequency domain coefficient inserter from the signaling is limited to inserting into the data stream a sequence of frequency domain coefficients of each long transform and split transform frame of the audio signal that has been inversely scaled according to a scale factor, in that, depending on the signaling, if the frame is a long transform frame, the sequence of frequency domain coefficients is formed by consecutively arranging the frequency domain coefficients of one transform in a non-interleaved manner, and if each frame is a split transform frame, the sequence of frequency domain coefficients is formed by interleaving the frequency domain coefficients of two or more transforms of each frame.

[0114] As far as the frequency domain coefficient inserter 108 is concerned, the fact that it operates independently of the signaling 34 distinguishing between frames 26a on the one hand and frames 26b on the other hand means that the inserter 108 inserts into the data stream 20 the frequency domain coefficients of the frames of the audio signal that have been inversely scaled according to a scale factor, either consecutively if one transform is performed for each frame in a non-interleaved manner, or using interleaving to insert the frequency domain coefficients of each frame if two or more transforms, i.e. two transforms in the example of Fig. 5, are performed for each frame. However, as already indicated above, the transform splitting mode can also be implemented differently, such as splitting one transform into three or more transforms.

[0115] Finally, it should be noted that the encoder of FIG. 5 can also be adapted to implement all other additional encoding means outlined above in relation to FIG. 2, such as MS encoding, complex stereo prediction 42 and TNS, and its respective parameters 44, 48 and 64 are determined for this purpose.

[0116] Although some aspects are described in terms of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, with blocks or devices corresponding to method steps or features of method steps. Similarly, aspects described in terms of a method step also represent a description of a corresponding block, object or feature of a corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, any one or more of the most important methods can be performed by such an apparatus.

[0117] Depending on the specific implementation requirements, the embodiments of the present invention can be implemented in hardware or software. The implementation can be realized using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or Flash memory, on which electronically readable signals are stored that cooperate (or can cooperate) with a programmable computer system to realize the respective method. The digital storage medium can therefore be computer readable.

[0118] Some embodiments according to the invention include a data carrier having stored thereon an electronically readable signal capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0119] Generally, embodiments of the present invention can be realized as a computer program product having program code operable to perform one of the above methods when the computer program product runs on a computer. The program code can, for example, be stored on a machine-readable carrier.

[0120] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0121] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0122] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein, said data carrier, digital storage medium or computer readable medium being generally tangible and / or non-transitory.

[0123] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, which data stream or sequence of signals can for example be arranged to be transmitted via a data communication connection, for example the Internet.

[0124] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0125] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0126] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0127] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods of the present invention are preferably performed by any hardware apparatus.

[0128] The above-described embodiments are merely illustrative of the principles of the present invention. Of course, modifications and variations of the configurations and details described herein will be apparent to others skilled in the art. It is therefore the intention that the present invention be limited only by the scope of the appended claims and not by the specific details presented by the description and illustration of the embodiments herein.

[0129] [Claim 1] 1. A frequency domain audio decoder supporting transform length switching, comprising: a frequency domain coefficient extractor (12) configured to extract frequency domain coefficients (24) of frames of an audio signal from the data stream; a scale factor extractor (14) configured to extract scale factors from the data stream; an inverse transformer (16) configured to inverse transform the frequency domain coefficients of the frames, scaled according to the scale factor, to obtain a time domain portion of the audio signal; a combiner (18) configured to combine the time domain portions to obtain the audio signal, The inverse transformer is responsive to a signalling in the frame of the audio signal, whereby in response to the signalling forming a transform by sequentially arranging the frequency domain coefficients of each frame scaled according to the scale factor in a non-deinterleaved manner, and performing an inverse transform of a first transform length on the transform; or forming two or more transforms by deinterleaving the frequency domain coefficients of the respective frames scaled according to the scale factor, and performing an inverse transform of each of the two or more transforms with a second transform length that is shorter than the first transform length; A frequency domain audio decoder, wherein the frequency domain coefficient extractor and the scale factor extractor operate independently of the signaling. [Claim 2] 2. The frequency domain audio decoder of claim 1, wherein the scale factor extractor (14) is configured to extract the scale factors from the data stream with a spectro-temporal resolution that is independent of the signaling. [Claim 3] 3. The frequency domain audio decoder of claim 1 or 2, wherein the frequency domain coefficient extractor (12) uses context-based or codebook-based entropy decoding to extract the frequency domain coefficients from the data stream by, for each frequency domain coefficient, assigning the same context or codebook to the respective frequency domain coefficient regardless of the signaling. [Claim 4] 4. The frequency domain audio decoder of claim 1, wherein the inverse transformer is configured to scale the frequency domain coefficients by the scale factor at a spectral resolution independent of the signaling. [Claim 5] 5. The frequency domain audio decoder of claim 1, wherein the inverse transformer is configured to perform noise filling on the frequency domain coefficients, the frequency domain coefficients being arranged consecutively such that they are not deinterleaved and with a spectral resolution independent of the signaling. [Claim 6] The inverter comprises: applying inverse temporal noise shaping filtering to the frequency domain coefficients in said forming one transform, wherein the frequency domain coefficients are sequentially arranged such that they are not deinterleaved; 6. The frequency domain audio decoder of claim 1, further comprising: a decoder configured to apply inverse temporal noise shaping filtering to the frequency domain coefficients in the formation of the two or more transforms, wherein the frequency domain coefficients are consecutively arranged to be deinterleaved and the two or more transforms are spectrally concatenated accordingly. [Claim 7] 7. The frequency domain audio decoder of claim 1, wherein the inverse transformer is configured to support joint stereo coding with or without inter-channel stereo prediction and to use the frequency domain coefficients as sum (middle) or difference (outer) spectra or prediction residuals of the inter-channel stereo prediction, the frequency domain coefficients being arranged such that they are not deinterleaved regardless of the signaling. [Claim 8] 8. The frequency domain audio decoder of claim 1, wherein a number of the two or more transforms is equal to two, and a length of the first transform is twice a length of the second transform. [Claim 9] 9. A frequency domain audio decoder according to any one of claims 1 to 8, wherein the inverse transform is an inverse modified discrete cosine transform (MDCT). [Claim 10] 1. A frequency domain audio coder supporting transform length switching, comprising: a transformer (104) configured to transform a time domain portion of an audio signal to obtain frequency domain coefficients for frames of said audio signal; an inverse scaler (106) configured to inverse scale the frequency domain coefficients according to a scale factor; a frequency domain coefficient inserter (108) configured to insert the frequency domain coefficients of the frames of the audio signal, inverse scaled according to a scale factor, into the data stream; a scale factor inserter (110) configured to insert a scale factor into the data stream; the converter is configured to switch, for the frames of the audio signal, between performing at least one conversion of a first conversion length for each frame and performing two or more conversions of a second conversion length for each frame that is shorter than the first conversion length; the converter is further configured to signal the switching by signaling within the frame of the data stream; the frequency domain coefficient inserter is configured to insert, for each frame, a sequence of the frequency domain coefficients of the respective frame of the audio signal, inversely scaled according to a scale factor, into the data stream, independent of the signalling; the sequence of frequency domain coefficients is formed, in response to the signaling, by sequentially arranging the frequency domain coefficients of the one transform of each frame in a non-interleaved manner if one transform is performed for each frame, and by interleaving the frequency domain coefficients of the two or more transforms of each frame if two or more transforms are performed for each frame; The scale factor inserter is a frequency domain audio coder that operates independently of the signaling. [Claim 11] 1. A method for frequency domain audio decoding supporting transform length switching, comprising: extracting frequency domain coefficients of frames of an audio signal from the data stream; extracting scale factors from the data stream; inverse transforming the frequency domain coefficients of the frames, scaled according to a scale factor, to obtain a time domain portion of the audio signal; combining the time domain portions to obtain the audio signal; The inverse transforming step is responsive to a signaling within the frame of the audio signal, whereby, in response to the signaling, forming a transform by sequentially arranging the frequency domain coefficients of each frame in a non-deinterleaved manner, and performing an inverse transform of a first transform length on the transform; or forming two or more transforms by deinterleaving the frequency domain coefficients of the respective frames, and performing an inverse transform of each of the two or more transforms with a second transform length that is shorter than the first transform length; The method, wherein the extraction of the frequency domain coefficients and the extraction of the scale factors are independent of the signaling. [Claim 12] A method for frequency domain audio coding supporting transform length switching, comprising: performing a transform on a time domain portion of an audio signal to obtain frequency domain coefficients for frames of said audio signal; inverse scaling the frequency domain coefficients according to a scale factor; inserting the frequency domain coefficients of the frames of the audio signal, inverse scaled according to a scale factor, into a data stream; and inserting a scale factor into the data stream; the step of performing a transformation includes switching, for the frames of the audio signal, between performing one transformation of at least a first transformation length for each frame and performing two or more transformations of a second transformation length for each frame that is shorter than the first transformation length; The method further comprises signaling the switch by signaling within the frame of the data stream; the insertion of the frequency domain coefficients is performed by inserting, for each frame, a sequence of the frequency domain coefficients of the respective frames of the audio signal, inversely scaled according to a scale factor, into the data stream, independent of the signaling; the sequence of frequency domain coefficients is formed, in response to the signaling, by sequentially arranging the frequency domain coefficients of the one transform of the respective frame in a non-interleaved manner if one transform is performed for the respective frame, and by interleaving the frequency domain coefficients of the two or more transforms of the respective frame if two or more transforms are performed for the respective frame; A method, wherein said scale factor insertion is performed independently of said signaling. [Claim 13] 13. A computer program having a program code for performing the method according to claim 11 or 12, when the computer program runs on a computer.

Claims

1. 1. A frequency domain audio decoder supporting transform length switching, comprising: a frequency domain coefficient extractor (12) configured to extract frequency domain coefficients (24) of frames of an audio signal from the data stream; a scale factor extractor (14) configured to extract scale factors from the data stream; an inverse transformer (16) configured to inverse transform the frequency domain coefficients of the frames, scaled according to the scale factor, to obtain a time domain portion of the audio signal; a combiner (18) configured to combine the time domain portions to obtain the audio signal, The inverse transformer is responsive to a signalling in the frame of the audio signal, whereby in response to the signalling forming a transform by sequentially arranging the frequency domain coefficients of each frame scaled according to the scale factor in a non-deinterleaved manner, and performing an inverse transform of a first transform length on the transform; or forming two or more transforms by deinterleaving the frequency domain coefficients of the respective frames scaled according to the scale factor, and performing an inverse transform of each of the two or more transforms with a second transform length that is shorter than the first transform length; A frequency domain audio decoder, wherein the frequency domain coefficient extractor and the scale factor extractor operate independently of the signaling.

2. 2. The frequency domain audio decoder of claim 1, wherein the scalefactor extractor (14) is configured to extract the scalefactors from the data stream with a spectro-temporal resolution that is independent of the signaling.

3. 3. The frequency domain audio decoder of claim 1 or 2, wherein the frequency domain coefficient extractor (12) uses context-based or codebook-based entropy decoding to extract the frequency domain coefficients from the data stream by, for each frequency domain coefficient, assigning the same context or codebook to the respective frequency domain coefficient regardless of the signaling.

4. 4. The frequency domain audio decoder of claim 1, wherein the inverse transformer is configured to scale the frequency domain coefficients by the scale factor at a spectral resolution independent of the signaling.

5. 5. The frequency domain audio decoder of claim 1, wherein the inverse transformer is configured to perform noise filling on the frequency domain coefficients, the frequency domain coefficients being consecutively arranged in a manner that is not deinterleaved and with a spectral resolution that is independent of the signaling.

6. The inverter comprises: applying inverse temporal noise shaping filtering to the frequency domain coefficients in said forming one transform, wherein the frequency domain coefficients are sequentially arranged such that they are not deinterleaved; 6. The frequency domain audio decoder of claim 1 , further comprising: a decoder configured to apply an inverse temporal noise shaping filtering to the frequency domain coefficients in the formation of the two or more transforms, wherein the frequency domain coefficients are consecutively arranged to be deinterleaved and the two or more transforms are spectrally concatenated accordingly.

7. 7. The frequency domain audio decoder of claim 1 , wherein the inverse transformer is configured to support joint stereo coding with or without inter-channel stereo prediction and to use the frequency domain coefficients as sum (middle) or difference (outer) spectra or prediction residuals of the inter-channel stereo prediction, the frequency domain coefficients being arranged such that they are not deinterleaved regardless of the signaling.

8. 8. The frequency domain audio decoder of claim 1, wherein the number of the two or more transforms is equal to two, and the first transform length is twice the second transform length.

9. 9. A frequency domain audio decoder according to any one of claims 1 to 8, wherein the inverse transform is an inverse modified discrete cosine transform (MDCT).

10. 1. A frequency domain audio coder supporting transform length switching, comprising: a transformer (104) configured to transform a time domain portion of an audio signal to obtain frequency domain coefficients of frames of said audio signal; an inverse scaler (106) configured to inverse scale the frequency domain coefficients according to a scale factor; a frequency domain coefficient inserter (108) configured to insert the frequency domain coefficients of the frames of the audio signal, inverse scaled according to a scale factor, into the data stream; a scale factor inserter (110) configured to insert a scale factor into the data stream; the converter is configured to switch, for the frames of the audio signal, between performing at least one conversion of a first conversion length for each frame and performing two or more conversions of a second conversion length for each frame that is shorter than the first conversion length; the converter is further configured to signal the switching by signaling within the frame of the data stream; the frequency domain coefficient inserter is configured to insert, for each frame, a sequence of the frequency domain coefficients of the respective frame of the audio signal, inversely scaled according to a scale factor, into the data stream, independent of the signalling; the sequence of frequency domain coefficients is formed, in response to the signaling, by sequentially arranging the frequency domain coefficients of the one transform of each frame in a non-interleaved manner if one transform is performed for each frame, and by interleaving the frequency domain coefficients of the two or more transforms of each frame if two or more transforms are performed for each frame; The scale factor inserter is a frequency domain audio coder that operates independently of the signaling.

11. 1. A method for frequency domain audio decoding supporting transform length switching, comprising: extracting frequency domain coefficients of frames of an audio signal from the data stream; extracting scale factors from the data stream; inverse transforming the frequency domain coefficients of the frames, scaled according to a scale factor, to obtain a time domain portion of the audio signal; combining the time domain portions to obtain the audio signal; The inverse transforming step is responsive to a signaling within the frame of the audio signal, whereby, in response to the signaling, forming a transform by sequentially arranging the frequency domain coefficients of each frame in a non-deinterleaved manner, and performing an inverse transform of a first transform length on the transform; or forming two or more transforms by deinterleaving the frequency domain coefficients of the respective frames, and performing an inverse transform of each of the two or more transforms having a second transform length that is shorter than the first transform length; The method, wherein the extraction of the frequency domain coefficients and the extraction of the scale factors are independent of the signaling.

12. A method for frequency domain audio coding supporting transform length switching, comprising: performing a transform on a time domain portion of an audio signal to obtain frequency domain coefficients for frames of said audio signal; inverse scaling the frequency domain coefficients according to a scale factor; inserting the frequency domain coefficients of the frames of the audio signal, inverse scaled according to a scale factor, into a data stream; and inserting a scale factor into the data stream; the step of performing a transformation includes switching, for the frames of the audio signal, between performing one transformation of at least a first transformation length for each frame and performing two or more transformations of second transformation lengths for each frame that are shorter than the first transformation length; The method further comprises signaling the switch by signaling within the frame of the data stream; the insertion of the frequency domain coefficients is performed by inserting, for each frame, a sequence of the frequency domain coefficients of the respective frames of the audio signal, inversely scaled according to a scale factor, into the data stream, independent of the signaling; the sequence of frequency domain coefficients is formed, in response to the signaling, by sequentially arranging the frequency domain coefficients of the one transform of the respective frame in a non-interleaved manner if one transform is performed for the respective frame, and by interleaving the frequency domain coefficients of the two or more transforms of the respective frame if two or more transforms are performed for the respective frame; A method, wherein said scale factor insertion is performed independently of said signaling.

13. A computer program having a program code for performing the method according to claim 11 or 12, when the computer program runs on a computer.

Images